Читать книгу Multilevel Modeling in Plain Language - Karen Robson - Страница 13

There are assumptions that relate to ‘independence of observations’ that we might think about less often. But it is this particular violation of the assumptions that multilevel modeling techniques are best suited to address. The assumption of independence means that cases in our data should be independent of one another, but if we have people clustered into groups, their group membership will likely make them similar to each other. Once people (Level 1) start having similar characteristics based on a group membership (Level 2), then the assumption of independence is violated. If you violate it, you get incorrect estimations of the standard errors. This isn’t just a niggly pedantic point. If you violate the assumptions, you are more likely to wrongly achieve statistical significance and make conclusions that are simply incorrect.

Perhaps an overlooked common-sense fact is that even if you don’t really care about group-level factors in your analysis (i.e. they aren’t part of your hypotheses), ignoring them does not make the problem go away. This is easy to demonstrate.

Suppose we are interested in how gender and parental occupational status influence academic achievement. Table 1.1 presents results from an OLS regression of reading scores on gender and parental occupational status. Reading scores are standardized within the sample to a mean of 0 and a standard deviation of 1. Gender is a dummy variable with 1 denoting males. Parental occupational status is a 64-category ordinal scale (the lowest value presenting low occupational status) that is treated as an interval-level variable here. The data come from the Australian sample from the 2006 Programme for International Student Assessment (PISA) organized by the Organisation for Economic Co-operation and Development (OECD, 2009). Data were obtained when all children were 15 and 16 years of age from schools in all eight states and territories of Australia.

Table 1.1N

b – unstandardized regression coefficients; s.e. – standard errors

* p < 0.05, ** p < 0.01, *** p < 0.001

Table 1.1 indicates that males (compared to females) have lower reading scores by 0.381 and that each unit increase in parental occupational status is associated with increases in reading scores of 0.020. These results are both statistically significant and have small standard errors. Our model has no group-level indicators, such as class or school. Just because we haven’t included group-level indicators, it does not mean that our problems of dependence among observations and thus correlated errors do not exist.

First, we need to predict our residuals from the regression equation whose coefficients we have just identified. Remember that the residuals are the difference between our predicted reading score based on these characteristics and the actual reading score we see in the data. Next, we can test if the assumption is violated if we run an analysis of variance (ANOVA) of these residuals by a grouping factor. Our grouping factor here is the region of Australia – the state or territory. If the residuals are independent of the regions, that is great – that means our errors are randomly distributed. This is not, however, the case in our data as the ANOVA returns a result of F = 55.8, df = 7, p < 0.001.

It might be helpful to think of it this way: we have several thousand students within the eight different regions in Australia. The ANOVA tells us that our individual-level results violate that assumption of uncorrelated errors because we find that the ANOVA gives statistically significant results. Table 1.2 shows the mean reading scores by region.

Table 1.2

We could assume that a ‘fix’ to this would be to add dummy variables to the model that represent the different regions. We add dummy variables to a model so that we can include nominal-level variables in our estimation. As the regions are nominal, we can then add them as a set of dummies with an omitted reference category. Table 1.3 shows how many students are in each region in Australia, while Table 1.4 displays the regression results for the model including the region dummy variables.

Table 1.3

As you can see in Table 1.3, students from this sample are dispersed among the eight regions of Australia. From reviewing the literature, we may have reason to believe that regional effects are determinants of academic achievement in Australia, as they are in many other countries around the world. For example, educational policies and resources are governed at the state level in Australia, and those regions containing the largest cities may offer the best resources for students (Australian Government, no date).

In Table 1.4 the gender and parental occupational status variables are the same as in Table 1.1. The region variable is entered as seven dummy variables, with the Australian Capital Territory as the omitted category.

Table 1.4N

b – unstandardized regression coefficients; s.e. – standard errors

^aReference category is Australian Capital Territory

^* < 0.05, ** p < 0.01, *** p < 0.001

Now is a good time to review the interpretation of coefficients as this will be important for understanding multilevel model outputs as well. The unstandardized coefficients (all in their own units of measurement) in Table 1.4 would be interpreted as:

 Compared to being in Australian Capital Territory, being in New South Wales is associated with a decrease in standardized reading scores by 0.075, controlling for the other variables in the model.

 Compared to being in Australian Capital Territory, being in Victoria is associated with a decrease in reading scores by 0.747, controlling for the other variables in the model.

 Compared to being in Australian Capital Territory, being in Queensland is associated with a 0.168 decrease in reading scores, controlling for the other variables in the model.

 Compared to being in Australian Capital Territory, being in South Australia is associated with a 0.089 decrease in reading scores, controlling for the other variables in the model.

 Compared to being in Australian Capital Territory, being in Western Australia is associated with a 0.222 decrease in reading scores, controlling for the other variables in the model.

 Compared to being in Australian Capital Territory, being in Tasmania is associated with a 0.189 decrease in reading scores, controlling for the other variables in the model.

 The coefficient for the Northern Territory is not statistically significant.

 Compared to being a female, being a male is associated with a 0.377 decrease in reading scores, controlling for the other variables in the model.

 Each unit increase in the parental occupational status scale is associated with a 0.019 increase in reading scores, controlling for all the other variables in the model.

 The constant is the reading score of –0.687, which is the value when all the independent variables have a value of zero. In this case it would be a female living in Australian Capital Territory whose parents have an occupational status score of zero – which isn’t possible with this particular occupational status measure.

Clearly there are ‘region’ effects here – the Australian Capital Territory (ACT) seems to be the best place for reading scores. The problem with this type of analysis is that children are nested within the regions. Furthermore, they are nested within schools, and even classrooms. While our analysis is at the individual level, the observations aren’t completely independent in the sense that there are eight regions into which pupils are divided.

An OLS regression assumes that the coefficients presented in the table are ‘independent’ of the effects of the other variables in the model, but, in this type of model, this assumption is false. If we believe that regions – an overarching structure – affect students differentially, the effect that regions have on reading scores is not independent of the effects of the other variables in the model. The structures themselves share similarities that we cannot observe but, nevertheless, influence our results. It is probably the case that parents’ occupational status and the gender of the student are also differentially associated with reading scores, depending on the region in which the student attends school. The OLS assumption of independent residuals is probably violated as well. It is likely that the reading scores within each region may not be independent, and this could lead to residuals that are not independent within regions. Thus, the residuals are correlated with our variables that define structure. We need statistical techniques that can handle this kind of data structure. OLS is not designed to do this.