Читать книгу Multilevel Modeling in Plain Language - Karen Robson - Страница 15

In OLS, there is an assumption that the effects of independent variables are the same across contexts. For example, the effect of gender on school achievement is the same for everyone regardless of the region in which they go to school. We have many reasons to suspect that the effects of individual characteristics vary across contexts – that their impacts are not the same for everyone, regardless of group membership. We may find that the effects of gender are more pronounced in particular regions, for example. The context may influence the impact of gender on school achievement – for example, some regions may have an official policy around raising the science achievement of girls or the reading achievement of boys (as they are typical problems in the school achievement literature). Multilevel models allow regression effects (coefficients) to vary across different contexts (in this case, region) while OLS does not.

Table 1.5

Unstandardized regression coefficients. Standard errors in parentheses. POS – parental occupational status; ACT – Australian Capital Territory; NSW – New South Wales; VIC – Victoria; QLD – Queensland; SA – South Australia; WA – Western Australia; TAS – Tasmania; NT – Northern Territory.

* p < 0.05, ** p < 0.01, *** p < 0.001

We might now think that one possible solution would be to run separate individual-level OLS regressions for each group. Table 1.5 displays the results of such an exercise.

This may seem to solve the problem of examining how group differences affect the impact of independent variables on the outcome of interest. You can see, for example, that the effect of being male ranges from being –0.446 in New South Wales (NSW) to –0.246 in the Australian Capital Territory (ACT). Likewise with parental occupational status (POS), the coefficients range from 0.017 in South Australia (SA) to 0.024 in the ACT and Western Australia (WA). These results do not tell us if the values are statistically significantly different from each other, without further calculations, and they also do not tell us anything about group properties which may influence or interact with individual-level outcomes. In addition to being a poor specification, this technique can get unwieldy if you have a large number of groups. Here we have only eight and the presentation of results is already rather difficult.

Another possible solution in OLS to effects varying across contexts might be to run interaction terms. You probably learned in your statistics training about interaction effects or moderating effects. If we thought that an independent variable affects an outcome differentially based upon the value of another independent variable, we could test this by using interaction terms. Based on the criticism above of running separate regressions for each group, a reasonable solution may seem to be to create interaction terms between the region and the other independent variables. We were making a similar argument earlier when we suggested that gender might impact on student achievement depending on region. We create the interaction terms by multiplying gender by region and parental occupational status by region (gender * regions; parental occupational status * regions) and we add them to the OLS regression as a set of new independent variables. If the interaction terms are statistically significant, it means that there is evidence that the effect of gender on reading and/or parental occupational status is contingent on the regions in which students go to school. The results of this estimation are presented in Table 1.6.

Table 1.6N

b – unstandardized regression coefficients; s.e. – standard errors

^a Australian Capital Territory is the reference category

^* < 0.05, ** p < 0.01, *** p < 0.001

The results in Table 1.6 suggest that region impacts on how individual characteristics affect the dependent variable. There are many statistically significant interactions. To really unravel what they mean, we have to look at them along with the main effects of the variables in the model. We can graph the main effects with the interaction effects and demonstrate the overall effect. We’re not going to do that here, but we do address graphing interaction effects later. The main point from these results is that there are significant interactions. Perhaps we have just solved the problem of group effects?

Unfortunately not. There are still problems with this model. While there is no shortage of examples of this type of analysis in published work, one major problem with this approach is the nature of the cross-level interaction term. Cross-level interaction terms refer to interaction terms which have variables at different levels of aggregation. In this case, we have interacted individual characteristics (Level 1) with group characteristics (Level 2). This approach is fraught with problems. Treating group-level variables as though they are properties of individuals may result in flawed parameter estimations and downwardly biased standard errors (Hox and Kreft, 1994), and so we are more likely to find significant results. This also results in problems in the calculation of degrees of freedom, which leads to flawed estimates of the standard errors and faulty results. The problems associated with degrees of freedom are explained in more detail below.