Opponents of school choice are misinterpreting the data
20 June 2012
SUGGESTED
Uncategorized
7 June 2012
Uncategorized
8 June 2012
Tax and Fiscal Policy
19 November 2024
In a recent column in the Guardian, Kevin Courtney lambasts the Secretary of State for Education, Michael Gove, for ‘fly[ing] a kite for free schools to be allowed to make a profit’. And he uses the Swedish experiment to justify his case. Let us ignore that free schools are already allowed to make a profit through procurement. The main question is: does the Swedish experience show that for-profit schools – and choice in general – have a deleterious impact on education? The short answer is no, and here is why.
Firstly, Courtney cites Sweden’s PISA record, which is not good. And neither is its TIMSS record. But does the fact that competition and choice have increased while international comparisons suggest falling achievement show that competition and choice are the cause of falling achievement? Of course not. This would be to confuse causation with correlation. So let us look at the evidence. All research from Sweden suggests some positive effects of the free school reform. The newest research from 2012 also indicates moderately positive short-term and long-term effects of competition in compulsory education, such as university attendance and mathematics grades in upper-secondary school. This is robust in controlling for grade inflation and other relevant changes in the education system, such as municipal school choice (which has an independent positive impact).
Nevertheless, it is sometimes difficult to discern exactly what the Swedish data are telling us because of the combination of decentralised grading practices with a heavily centralised admissions system that depends heavily on that same grading practice. Instead, therefore, let us look at the cross-national PISA research. This finds that private school competition increases achievement in mathematics, science and reading – though standards are falling across the Swedish education sector. It also drives down costs. Similar findings have been shown in TIMSS research. This contrasts strongly with the official PISA report, which is not surprising given its poor methodology.
This should be enough to refute the idea that more competition drives down standards in international comparison tests. Since we are comparing Nordic countries, however, let us go further. Interestingly, Norway and Sweden have fallen in international tests to the same degree since the 1990s, but only Sweden has increased choice significantly. Instead, the common denominator is that both implemented pedagogical ideas with no grounding in research. By emphasising ‘individualisation’, the role of teachers has been reduced in favour of pupils learning by themselves. The evidence suggests that this often has an outright detrimental impact on achievement. Furthermore, both countries have forced this idea into a national curriculum that practically all schools need to follow – there is no freedom for free schools here. In other words, the preponderance of evidence suggests that flawed pedagogical ideas are the key culprits behind Sweden’s fall in international tests, not choice and competition. I’m not alone in emphasising this: Björklund et al.(2010), discussing various potential factors, conclude that individualisation ‘probably is the most important explanation for the fall in achievement in Sweden’. In a follow-up post, I will explain how Finland fits the equation.
So much for international tests. Next, however, Courtney tells us that Jonas Vlachos, who works at the same organisation for which I’m working, finds a negative impact of attending a for-profit compulsory school for final grades in upper-secondary school. But we are not told what this (small) impact means. The truth is that, because of the Swedish upper-secondary school system’s peculiar features, the findings are not too illuminating. For example, it could equally be that children in municipal schools attend upper-secondary schools that suffer from higher grade inflation than children in for-profit compulsory schools. Another issue is the strong potential for selection bias. Pupils in for-profit schools may very well go on to vastly different upper-secondary school programmes, which affect their grades radically. Grades in science are compared with grades in hairdressing and building construction. In other words, apples are compared with oranges. Jonas Vlachos is a fine scholar, but in this case his findings have been grossly overstated.
Additionally, new research shows that the effects of for-profit and non-profit competition on grades in both municipal and free schools are equal: there are no differential general-equilibrium effects of for-profit and non-profit competition on achievement. It is therefore interesting that my own analysis of grades in ninth grade (equivalent to GCSEs) displays similar findings. While the comparison with municipal schools should be interpreted with caution, due to the potential for selection bias, differences between non-profit and for-profit free schools are less likely to be biased. In the preferred model, there is no difference in achievement between non-profit and for-profit free schools. Again, we do not know what this means, due to the decentralised grading practice, but grades are more comparable at this level than in upper-secondary school.
Additionally, the US and Chilean evidence does not find that for-profit schools are worse than non-profit schools. There is some heterogeneity, but, overall, the evidence generally indicates no general differences between for-profit and non-profit schools. In other words, there is very little evidence that the profit motive per se drives down quality. But, of course, the profit motive is very important in expanding choice because of the additional capital it can mobilise.
Finally, let me comment on the issue of segregation. Does the research at least indicate that the choice reform has increased inequality in achievement? Not at all. The evidence suggests that choice has not had any impact on the variation in achievement between pupils. While PISAdisplays no consistent picture of how equality of achievement has progressed since the early 2000s, according to TIMSS, high-achieving pupils have actually fallen more than low-achieving pupils in performance since 1995. Based on TIMSS, therefore, Swedish education has become more equal, but this is hardly anything to celebrate. Even the most recent report from the National Agency of Education concludes that the impact of socio-economic background on achievement has basically remained stagnant since the 1990s. In short, very little evidence indicates that the choice reform has had any impact on educational equality.
In conclusion, Courtney’s article – the themes of which have been taken up by the Labour shadow education minister – can tell us very little about the effects of for-profit schools and choice in general. This is because of a mix of confirmation bias – he simply leaves out research that does not support his thesis – and a failure to understand the very research he cites. Of course, Courtney is not alone. School choice proponents and opponents alike regularly commit similar mistakes. This strategy might be valuable if one’s sole objectives were (1) to confirm already held opinions and (2) to preach to the already converted on either side of the aisle. But it is useless as a strategy for trying to understand how we can improve education quality – which, I hope, is a goal we all share.
Firstly, Courtney cites Sweden’s PISA record, which is not good. And neither is its TIMSS record. But does the fact that competition and choice have increased while international comparisons suggest falling achievement show that competition and choice are the cause of falling achievement? Of course not. This would be to confuse causation with correlation. So let us look at the evidence. All research from Sweden suggests some positive effects of the free school reform. The newest research from 2012 also indicates moderately positive short-term and long-term effects of competition in compulsory education, such as university attendance and mathematics grades in upper-secondary school. This is robust in controlling for grade inflation and other relevant changes in the education system, such as municipal school choice (which has an independent positive impact).
Nevertheless, it is sometimes difficult to discern exactly what the Swedish data are telling us because of the combination of decentralised grading practices with a heavily centralised admissions system that depends heavily on that same grading practice. Instead, therefore, let us look at the cross-national PISA research. This finds that private school competition increases achievement in mathematics, science and reading – though standards are falling across the Swedish education sector. It also drives down costs. Similar findings have been shown in TIMSS research. This contrasts strongly with the official PISA report, which is not surprising given its poor methodology.
This should be enough to refute the idea that more competition drives down standards in international comparison tests. Since we are comparing Nordic countries, however, let us go further. Interestingly, Norway and Sweden have fallen in international tests to the same degree since the 1990s, but only Sweden has increased choice significantly. Instead, the common denominator is that both implemented pedagogical ideas with no grounding in research. By emphasising ‘individualisation’, the role of teachers has been reduced in favour of pupils learning by themselves. The evidence suggests that this often has an outright detrimental impact on achievement. Furthermore, both countries have forced this idea into a national curriculum that practically all schools need to follow – there is no freedom for free schools here. In other words, the preponderance of evidence suggests that flawed pedagogical ideas are the key culprits behind Sweden’s fall in international tests, not choice and competition. I’m not alone in emphasising this: Björklund et al.(2010), discussing various potential factors, conclude that individualisation ‘probably is the most important explanation for the fall in achievement in Sweden’. In a follow-up post, I will explain how Finland fits the equation.
So much for international tests. Next, however, Courtney tells us that Jonas Vlachos, who works at the same organisation for which I’m working, finds a negative impact of attending a for-profit compulsory school for final grades in upper-secondary school. But we are not told what this (small) impact means. The truth is that, because of the Swedish upper-secondary school system’s peculiar features, the findings are not too illuminating. For example, it could equally be that children in municipal schools attend upper-secondary schools that suffer from higher grade inflation than children in for-profit compulsory schools. Another issue is the strong potential for selection bias. Pupils in for-profit schools may very well go on to vastly different upper-secondary school programmes, which affect their grades radically. Grades in science are compared with grades in hairdressing and building construction. In other words, apples are compared with oranges. Jonas Vlachos is a fine scholar, but in this case his findings have been grossly overstated.
Additionally, new research shows that the effects of for-profit and non-profit competition on grades in both municipal and free schools are equal: there are no differential general-equilibrium effects of for-profit and non-profit competition on achievement. It is therefore interesting that my own analysis of grades in ninth grade (equivalent to GCSEs) displays similar findings. While the comparison with municipal schools should be interpreted with caution, due to the potential for selection bias, differences between non-profit and for-profit free schools are less likely to be biased. In the preferred model, there is no difference in achievement between non-profit and for-profit free schools. Again, we do not know what this means, due to the decentralised grading practice, but grades are more comparable at this level than in upper-secondary school.
Additionally, the US and Chilean evidence does not find that for-profit schools are worse than non-profit schools. There is some heterogeneity, but, overall, the evidence generally indicates no general differences between for-profit and non-profit schools. In other words, there is very little evidence that the profit motive per se drives down quality. But, of course, the profit motive is very important in expanding choice because of the additional capital it can mobilise.
Finally, let me comment on the issue of segregation. Does the research at least indicate that the choice reform has increased inequality in achievement? Not at all. The evidence suggests that choice has not had any impact on the variation in achievement between pupils. While PISAdisplays no consistent picture of how equality of achievement has progressed since the early 2000s, according to TIMSS, high-achieving pupils have actually fallen more than low-achieving pupils in performance since 1995. Based on TIMSS, therefore, Swedish education has become more equal, but this is hardly anything to celebrate. Even the most recent report from the National Agency of Education concludes that the impact of socio-economic background on achievement has basically remained stagnant since the 1990s. In short, very little evidence indicates that the choice reform has had any impact on educational equality.
In conclusion, Courtney’s article – the themes of which have been taken up by the Labour shadow education minister – can tell us very little about the effects of for-profit schools and choice in general. This is because of a mix of confirmation bias – he simply leaves out research that does not support his thesis – and a failure to understand the very research he cites. Of course, Courtney is not alone. School choice proponents and opponents alike regularly commit similar mistakes. This strategy might be valuable if one’s sole objectives were (1) to confirm already held opinions and (2) to preach to the already converted on either side of the aisle. But it is useless as a strategy for trying to understand how we can improve education quality – which, I hope, is a goal we all share.
SHARE
