Читать книгу Economic Evaluation in Education - Henry M. Levin - Страница 28

As a final consideration before beginning the research, it is worth asking whether an economic analysis is needed. Thus far, it has been assumed that in most cases cost analysis should be done and that a lot of valuable information will be obtained from such analysis. However, this reasoning is not definitive. We have already suggested that if there are no alternatives or if a decision is already predetermined, the entire evaluative situation—including economic analysis—becomes moot. Further, if sufficient time or other resources are lacking or if CE types of data will not alter decisions, there is probably not a strong case for doing cost analysis. Even when all the prerequisites for implementing and using cost analysis are present, it is important that such an evaluation be worth doing in the first place.

Moreover, economic evaluation techniques might themselves be interrogated using efficiency criteria: What is the economic payoff to performing CE analysis? Do the benefits of performing BC analysis outweigh the costs? In response, we refer to Scriven’s (1974, pp. 85–93) notion of cost-free evaluation. In considering the choice of an analytic approach and supportive design for implementation, one must ask what will be gained by finding a better alternative. If the gains will be relatively small, only a small investment in evaluation would be merited. Indeed, if the potential gains of a good decision are minuscule, it is possible that no formal analytic study should be undertaken. That is, intuition and present knowledge should suffice, given that little will be gained from a more formal and extended evaluation. However, when the value of selecting a new alternative can be very great, it may be worth making a large investment in evaluation and analysis. Scriven would say that this situation meets the cost-free evaluation criterion if the probable gains from the evaluation will be in excess of the costs of the evaluation. It is cost-free in that more is saved by a good and appropriate evaluation than is expended in resources on that evaluation. Put simply, the more costly the intervention is to implement, the more likely it is that an economic evaluation will pay for itself by changing the allocation of resources. Prima facie, the economic case for economic evaluation appears reasonable: Society devotes a substantial amount of resources to education programs, and the information obtained from an economic evaluation is probably substantial and salient.

Nevertheless, in order to know how much economic evaluation to do, one needs a sense of what research effort is involved. We describe the research tasks in detail in the following chapters, so here we offer very general comments. First, it is important to recognize that costs are estimated, just in the same way that impacts are estimated. Cost estimation is therefore a substantial research activity, just as identification of impacts is. When estimating impacts, the researcher is immediately concerned with potential sampling error—that is, the likelihood that the sample chosen reflects the population. Sampling error is ameliorated by properly designing the sampling frame and drawing a sample of sufficient power. This logic also holds when estimating costs: The researcher should draw from the population of sites (e.g., schools or classrooms or students) to ensure that the sample reflects the population. A priori, it is difficult to say if the sample for estimating costs should be larger or smaller than the sample for estimating impacts; it depends on the variation of costs within the population. Overall, whatever research tasks are important for impact evaluations are likely to be required of economic evaluators.

Second, in most BC or CE evaluations, some expertise will be needed from persons who are specialists in these kinds of analyses. Although BC analysis is a standard component of the economics of public finance, many academic economists have not had practical training in either cost analysis or shadow pricing techniques. Also, one should not confuse expertise in accounting with expertise in BC or CE analysis. While BC and CE analysts must have a good understanding of cost accounting, cost accountants need have no understanding of BC and CE analysis. Often, the correct approaches for cost accounting for business firms are inappropriate for estimating the costs of social projects. In fact, one of the rationales of public economics is that, frequently, social costs and benefits differ considerably from private ones. Cost accountants are not usually trained to address social costs and benefits.

That said, economic evaluations do not just rely on economic principles. In order to perform CE analysis, one has to determine what effectiveness means in each educational context. For instance, for reading programs it might be comprehension or vocabulary or fluency; for dropout prevention programs it might be keeping students in school each year, getting them to finish 12 grades, or getting them to complete a high school diploma. The answers to these questions cannot be found in economic theory. Experts in the respective fields are needed to determine which specific outcomes are most valid.

Overall, there is no presumption that an economic evaluation is necessary nor is there a fixed amount of effort that an expert might contribute to perform such an evaluation. Resources would need to cover three discrete research tasks: (1) calculating costs, (2) identifying effects or estimating benefits, and (3) combining the two to report an economic metric. Potentially, such studies might require resources equivalent to those needed for an impact evaluation. The main guiding principle is the proportionality principle—that is, the idea that the amount of effort required should correspond to the value of the knowledge acquired and how that knowledge influences the results of the analysis. Suffice it to say, we hope to establish in this book that the knowledge gained from most economic evaluations is unlikely to be trivial and that more, rather than fewer, such evaluations are worthwhile.