Читать книгу Social Work Research Methods - Reginald O. York - Страница 49

In this section are a few situations that will illustrate various aspects of the use of science to inform decisions. You will review each and decide if there is a problem.

Let’s examine the following situation:

A group of researchers started their study process with the purpose of determining the extent to which clients treated for depression have shown improvement in their levels of depression. They collected scores on a depression scale for a group of clients in treatment for depression. They collected these scores both before and after a service was provided that was designed to reduce depression. These scores showed improvement from the first to the last measurement. They were subjected to statistical analysis and were found to be statistically significant. The researchers concluded that these clients had experienced improvement with regard to depression, but they were not clear about the population to whom these results could be generalized because these clients had not been selected on a random basis from a larger population.

Do you see any problems with this situation? It is a summary you may have read from your search of the literature with regard to evidence about the treatment of a condition like depression. There are no major flaws here. The process started with a purpose, data were collected consistent with that purpose, and conclusions were drawn that were consistent with the data analyzed.

You saw a reference to statistical analysis, which will be discussed many times in this book, but you are not expected to fully understand it at present. But you surely have seen references to statistical analysis of data and concepts such as “statistical significance.” These clients showed improvement, and the level of their improvement cannot be easily explained by chance (because the data were found to be statistically significant). If your data can be explained by chance, you cannot conclude that they can be taken seriously.

A reference was made to the generalization of data results, another concept that will be discussed many times in the chapters of this book. It was noted that the clients had not been selected at random, a basis for scientific generalization. It is likely that you have seen numerous references to random samples in your review of the literature. That is the superior form of sampling and strengthens the extent to which you can generalize your findings to persons who were not in your study.

Now, let’s examine another situation.

A group of researchers had the purpose of determining if after-school tutoring was effective in the improvement of grades for middle school students. Grades were compared for the period of time before tutoring was offered and measured again after tutoring was completed. The results showed that the average student improved but that statistical significance was not achieved (i.e., these results can too easily be explained by chance). These researchers concluded that tutoring is an effective way to improve grades for middle school students.

Do you see a problem with this situation? You need an appreciation of the theme of chance to grasp the lesson provided here. These researchers treated the data seriously in their conclusions even though the data failed to be statistically significant. The proper conclusion would be that the study failed to find an improvement in grades. What they discovered was chance. If chance can explain your data, you cannot take them seriously with regard to your study conclusions.

What do you think of the following situation?

Some fellow students conduct a study of students in one social work research class and find that the majority of them are identified with the Democratic Party. They conclude that there are more people in this university who identify as Democratic than Republican.

The problem here is easy to identify. You cannot generalize from a small sample of people in one class in a university to the entire university. The issue of generalization is discussed with regard to the nature of the sample that you employ. A small class of students in one specific place is not a random sample. And you would need a much larger sample than this to generalize to the entire university.

Now examine this situation.

Paul is a social work research student who starts the process of conducting research by deciding that he would like to conduct interviews of his fellow students. He has not yet decided on the purpose of his study.

What is the problem here? Well, Paul has put the cart before the horse. He has decided on the methods he will use to conduct his study before he decides on the purpose of the study. The purpose should come first. Why? Different purposes suggest different types of methods to be used. You need to know the purpose before you can make decisions on your methods. Interviews serve some purposes better than others.

Finally, let’s examine this situation.

Paula is an individual and family counselor who has been trained in New Beginnings Therapy (hypothetical). Paula has used this treatment for the past 3 months when the objective was to reduce stress. She believes that it has been effective with some clients but has not collected evidence on the outcomes with her clients. She examined the literature on evidence regarding the effectiveness of this approach to counseling. She found 1 study that revealed that New Beginnings Therapy was effective in the reduction of stress, and she found 23 studies that showed that it is not effective in this regard. Paula concluded that this therapy is effective because she found a study that showed that it is.

Is there a problem here? Is it okay to use only 1 study that shows a certain outcome and ignore 23 studies that show an opposite outcome? No, not if you are being scientific. Science is comprehensive. You use all the evidence that you can find, not just the ones that support your predetermined conclusion. Perhaps Paula could report on all the evidence and conclude that she will conduct her own study given the fact that there is more evidence against this approach to therapy than evidence that supports it. Perhaps she will find it to be effective with the types of clients she serves. The lack of sound evidence should encourage Paula to measure client outcome with her individual clients or to search for other methods of treatment.