Читать книгу Economic Evaluation in Education - Henry M. Levin - Страница 27

Once one has established the problem, the alternatives to be considered in addressing it, and the audiences, it is necessary to select the type of analytic framework that will be used. In the previous chapter, we identified these approaches as cost and CF analysis; CE and cost-utility (CU) analysis; and BC analysis. Here, we discuss which is appropriate across the range of educational interventions.

As noted in Chapter 1, CE and BC analyses are strictly intended to answer different questions about CE and efficiency respectively. However, the analyst may need to discover which form of analysis is appropriate for each intervention.

The best way to make this discovery is to make sure that the economic evaluation corresponds to the theory of change for each intervention. For our purposes, we can think of the theory of change in terms of (a) an educational intervention that is implemented (b) within a general context or set of existing conditions and that via a (c) connecting outcome or mechanism meets or is intended to meet (d) a set of longer-term goals (Weiss, Bloom, & Brock, 2014). Each of these elements (a–d) is relevant for deciding on the appropriate economic evaluation and how that evaluation is structured (Ludwig, Kling, & Mullainathan, 2011).

The intermediate or longer-term outcomes of some interventions explicitly can be measured in monetary values. For example, a training program may be intended to increase earnings for participants or a social emotional learning intervention may have savings to the school system from reduced conduct disorder as its goal. Given that BC analysis requires all amounts to be measured in monetary terms, this would be the most appropriate evaluative method. While the value of additional earnings and employment from an educational investment might be measured in dollars, some measures—such as positive feelings toward learning—cannot easily be converted into monetary units. Indeed, for many potential educational interventions, the use of BC analysis will be challenging. By contrast, CE and CU analyses can be applied under a wide variety of situations—most notably any intervention with achievement goals. However, if there are many outcomes—and especially if the longer-term goals are far into the future—then BC analysis will be more appropriate. Finally, if no outcomes or longer-term goals can be specified in quantitative terms, it may nonetheless be helpful to consider CF analysis to yield some indication of how much an intervention actually costs in relation to budgets.

The theory of change, as well as the economic method selected, will also affect how that method is applied. The cost analysis must clearly correspond to the scale, scope, duration, and intensity of the intervention that is being implemented. For instance, we can think of a high school support program such as Talent Search that is intended to boost high school completion and college enrollment (Bowden & Belfield, 2015). If the mechanism by which Talent Search is effective is that students successfully apply for college, then the analyst should cost out that application process. Alternatively, if the mechanism is continuous mentoring support through high school, then the costing exercise will have to follow students through their mentoring experiences, perhaps over multiple years. In turn, this means that the comparison group will have to be followed over the same time period. As another example, we can think of socioemotional learning interventions that are intended to change the climate across the school. The cost analysis must measure all changes in resources at the school level (see Long, Brown, Jones, Aber, & Yates, 2015). Cost per student may not be an accurate guide: In order to influence school climate, all (or most) of the students must receive services collectively, not individually, and it is the total cost that is salient.

Similarly, the economic analysis must be responsive to the context in which the intervention is delivered. On the cost side, the availability and prices of ingredients may vary across school districts: In low-income countries, some inputs (e.g., online computing systems) may not be available; in areas of high poverty, the analyst must take into account the opportunity cost of time for schoolchildren who may need to work to support their families. Context also matters on the benefit side. For example, a preschool program might reduce future youth crime—in fact, this effect is one of the main reasons why the HighScope Perry Preschool Program has such a high payoff. But this impact may only be salient for preschools in localities where the crime rate is high (a distinction noted in Barnett & Masse, 2007).

Finally, the analyst may have to infer longer-term outcomes from intermediate ones. In the case of preschool, for example, the researcher cannot wait 15 years for information on earnings; even freshman college programs can have very delayed impacts on future earnings. In these cases, it is important that the BC analyst choose to measure intermediate outcomes (e.g., high school graduation or first-semester credits) that are well correlated with longer-term outcomes. It is also important that any changes in behavior be measured when they occur and for how long they occur. Interventions where change is “fast-acting” and persistent are of higher value because of their immediacy, and the BC analysis should reflect this.

At this stage, we can cover only the main issues. As the theory of change for each intervention becomes more detailed, the economic evaluator should respond accordingly. Fundamentally, each economic analysis should adapt to fit with the theory of change for each specific intervention.