For-profit schools in Sweden: a response to criticisms
22 December 2010
SUGGESTED
Uncategorized
21 December 2010
Uncategorized
23 December 2010
Government and Institutions
This is a response to Mr Francis Gilbert at the Local Schools Network who has questioned my research on Swedish for-profit schools with some relatively surprising arguments.
As the author of the paper, I am rather flattered by Mr Gilbert’s accusation, which appears to be that my models are “too sophisticated” – but if there is any problem, it is that the models I employ might not be sophisticated enough. On page 16 in the paper, I discuss the problems of “endogeneity” – it might be the case that even after employing a vast range of control variables, students in free schools may be better (or worse!) than students attending municipal schools, have more/less motivation etc. That is, if we fail to control for important uncontrollable variables that affect performance, our estimates will be biased. However, the most recent study (Tegle, 2010) suggests that not taking this into account significantly underestimates the effect of free schools – the effect is much stronger when using more sophisticated models that explicitly are supposed to take into account the problem of endogeneity. My paper is very clear on that point – the estimates presented are likely to err on the side of caution.
The reason I didn’t attempt to employ models accounting for endogeneity, which according to previous research would show even stronger effects of for-profit/non-profit schools, was three-fold. Firstly, I was mostly interested in analysing whether the “deterioration thesis” – that the profit motive drives down educational quality – held up to empirical scrutiny. As endogeneity was highly likely to bias the estimates against free schools, finding that my models displayed positive effects was thus enough to strongly indicate that the “deterioration thesis” was not accurate (especially since 65% of all free schools are for-profit). Secondly, I didn’t have enough time. Assembling the statistics for about 1,500 schools over five years, and coding all schools by ownership structure, was time-consuming. Employing more advanced models would necessitate further variables. Thirdly, as I analysed school-level evidence, I was concerned that it might be difficult to control for endogeneity entirely.
However, after I had completed the paper, I analysed this further by employing models explicitly designed to deal with endogeneity (so-called Instrumental-Variable models). As instruments, I use the percentage of children in each municipality attending privately run kindergarten (i.e. “free kindergarten”), and the percentage of “low-educated” people in each municipality. Since valid instruments are difficult to find, and since I need one instrument for each endogenous variable, I do not include the pre-reform dummy. However, removing all these schools from the sample renders almost exactly the same estimates, but the for-profit effect is slightly larger and the non-profit effect is marginally less significant (at the 10% level).
I find that, in line with Tegle’s findings, for-profit and non-profit schools, on average increase the GPA by about 34 points, which is much stronger than the effects I find in my paper (around 5 points in the model with the overall estimates). For this analysis I use estimates from 2009 due to time constraints. Using the same methodology as in my paper for this truncated sample renders the estimates basically the same as when I use data over five years.
The Hausman test strongly rejects the null hypothesis of no endogeneity. This suggests that the estimates in my paper are biased downwards, which confirms that these probably err on the side of caution. Furthermore, a test of weak instruments and an over-identification test, when lumping all free schools together (which renders an overall free school effect of 33.8 points), suggests that my instruments are valid. Also, this strongly suggests that for-profit and non-profit schools are equally good at raising standards.
The models employed in my paper thus probably underestimate the effect of for-profit schools – but this is something I anticipated. I was mainly interested in whether for-profit schools drove down quality and found no evidence that they did.
Mr Gilbert argues that “Many statisticians would question [the IEA’s] results because they have used ‘control mechanisms’ which claim to equalise children from different backgrounds in an attempt to compare ‘like for like’.” However, statisticians would not question the use of control variables (which is essential to statistical evidence); rather they might argue that I have not controlled for enough variables. But the statistical model described above is specifically designed to control for the variables which are difficult to control for (such as motivation, ability etc.). And when one does, the effects of for-profit schools are even more positive.
Thirdly, regarding the PISA results to which Mr Gilbert refers one must take into account other reasons for why countries with more competition do not perform better on average – again, we meet the problem of not controlling for enough variables. For example, the Swedish voucher programme in 1992 was introduced during an economic crisis. Residential segregation and unemployment increased during the 1990s while school budgets were cut. Meanwhile, we introduced a new grading system, which removed a “relative system” which was based on a bell-curve. Furthermore, we began to transfer more responsibility from the teacher to the student (allowing more students to work on their own), which the international research suggests would have had a negative effect. A perfect storm of causes contributed to problems in Swedish education during the 1990s.
Given that hitherto presented research – which I have extended to include for-profit free schools – finds positive effects of school competition in Sweden, these problems would probably have had a larger negative impact on educational performance had it not been for school competition. The evidence of declining performance in international ratings during the 1990s is bad news for those who argue that school competition is a panacea. But those of us who do not argue this would still conclude that competition has been beneficial.
The next step for those wanting to question my findings is to provide new research showing that for-profit free schools are in fact bad. So far, no such evidence has been presented, and I therefore conclude that fears of the profit motive in education reform, at least in the Swedish context, are unfounded.
As the author of the paper, I am rather flattered by Mr Gilbert’s accusation, which appears to be that my models are “too sophisticated” – but if there is any problem, it is that the models I employ might not be sophisticated enough. On page 16 in the paper, I discuss the problems of “endogeneity” – it might be the case that even after employing a vast range of control variables, students in free schools may be better (or worse!) than students attending municipal schools, have more/less motivation etc. That is, if we fail to control for important uncontrollable variables that affect performance, our estimates will be biased. However, the most recent study (Tegle, 2010) suggests that not taking this into account significantly underestimates the effect of free schools – the effect is much stronger when using more sophisticated models that explicitly are supposed to take into account the problem of endogeneity. My paper is very clear on that point – the estimates presented are likely to err on the side of caution.
The reason I didn’t attempt to employ models accounting for endogeneity, which according to previous research would show even stronger effects of for-profit/non-profit schools, was three-fold. Firstly, I was mostly interested in analysing whether the “deterioration thesis” – that the profit motive drives down educational quality – held up to empirical scrutiny. As endogeneity was highly likely to bias the estimates against free schools, finding that my models displayed positive effects was thus enough to strongly indicate that the “deterioration thesis” was not accurate (especially since 65% of all free schools are for-profit). Secondly, I didn’t have enough time. Assembling the statistics for about 1,500 schools over five years, and coding all schools by ownership structure, was time-consuming. Employing more advanced models would necessitate further variables. Thirdly, as I analysed school-level evidence, I was concerned that it might be difficult to control for endogeneity entirely.
However, after I had completed the paper, I analysed this further by employing models explicitly designed to deal with endogeneity (so-called Instrumental-Variable models). As instruments, I use the percentage of children in each municipality attending privately run kindergarten (i.e. “free kindergarten”), and the percentage of “low-educated” people in each municipality. Since valid instruments are difficult to find, and since I need one instrument for each endogenous variable, I do not include the pre-reform dummy. However, removing all these schools from the sample renders almost exactly the same estimates, but the for-profit effect is slightly larger and the non-profit effect is marginally less significant (at the 10% level).
I find that, in line with Tegle’s findings, for-profit and non-profit schools, on average increase the GPA by about 34 points, which is much stronger than the effects I find in my paper (around 5 points in the model with the overall estimates). For this analysis I use estimates from 2009 due to time constraints. Using the same methodology as in my paper for this truncated sample renders the estimates basically the same as when I use data over five years.
The Hausman test strongly rejects the null hypothesis of no endogeneity. This suggests that the estimates in my paper are biased downwards, which confirms that these probably err on the side of caution. Furthermore, a test of weak instruments and an over-identification test, when lumping all free schools together (which renders an overall free school effect of 33.8 points), suggests that my instruments are valid. Also, this strongly suggests that for-profit and non-profit schools are equally good at raising standards.
The models employed in my paper thus probably underestimate the effect of for-profit schools – but this is something I anticipated. I was mainly interested in whether for-profit schools drove down quality and found no evidence that they did.
Mr Gilbert argues that “Many statisticians would question [the IEA’s] results because they have used ‘control mechanisms’ which claim to equalise children from different backgrounds in an attempt to compare ‘like for like’.” However, statisticians would not question the use of control variables (which is essential to statistical evidence); rather they might argue that I have not controlled for enough variables. But the statistical model described above is specifically designed to control for the variables which are difficult to control for (such as motivation, ability etc.). And when one does, the effects of for-profit schools are even more positive.
Thirdly, regarding the PISA results to which Mr Gilbert refers one must take into account other reasons for why countries with more competition do not perform better on average – again, we meet the problem of not controlling for enough variables. For example, the Swedish voucher programme in 1992 was introduced during an economic crisis. Residential segregation and unemployment increased during the 1990s while school budgets were cut. Meanwhile, we introduced a new grading system, which removed a “relative system” which was based on a bell-curve. Furthermore, we began to transfer more responsibility from the teacher to the student (allowing more students to work on their own), which the international research suggests would have had a negative effect. A perfect storm of causes contributed to problems in Swedish education during the 1990s.
Given that hitherto presented research – which I have extended to include for-profit free schools – finds positive effects of school competition in Sweden, these problems would probably have had a larger negative impact on educational performance had it not been for school competition. The evidence of declining performance in international ratings during the 1990s is bad news for those who argue that school competition is a panacea. But those of us who do not argue this would still conclude that competition has been beneficial.
The next step for those wanting to question my findings is to provide new research showing that for-profit free schools are in fact bad. So far, no such evidence has been presented, and I therefore conclude that fears of the profit motive in education reform, at least in the Swedish context, are unfounded.
SHARE
