Assessing the free-school effect at a system-wide level
6 December 2011
SUGGESTED
Uncategorized
6 December 2011
Uncategorized
7 December 2011
Tax and Fiscal Policy
19 November 2024
Evaluating the system-wide impact of choice and competition on educational outcomes is a difficult task. Last summer Rebecca Allen, in a paper for Research in Public Policy, offered a useful overview of the main studies of the Swedish evidence base, which has been referenced regularly by sceptics of school choice reform in the UK.
Allen was right to highlight the difficulties of constructing a long enough panel of data to account for both pre- and post-intervention trends as the main reason for the paucity of such research. She also commented on a methodological problem from which several studies in the early part of the decade suffered in failing to account for the non-random application of the policy across different municipalities and for the peculiar demographic characteristics of areas with many free schools (Allen, p. 5).
In assessing how effectively key studies have tackled these problems, Allen’s survey found that the most recent (Böhlmark and Lindahl, 2007; 2008) employed the most robust data and methods and required relatively few identification assumptions. They were able to construct a long panel of data, from 1988/89 to 2002/03, including three years of pre-intervention data. Böhlmark and Lindahl’s results showed ‘a moderately positive impact of free school growth on municipality academic performance at the end of ninth grade (the end of lower secondary school, when pupils are aged 15-16)’ (Allen, p. 3).
Allen went on to qualify this finding by emphasising that those that benefited most according to Böhlmark and Lindahl were the children of educationally aspirant parents. This subsequently much publicised interpretation of Böhlmark and Lindahl, however, has more recently been challenged by Gabriel Sahlgren of the IEA. Sahlgren clarified that while the positive effect for students with low-educated parents or an immigrant background was indeed insignificant, ‘students from low-income families benefit more than those from high-income families’ [my italics]. Furthermore, ‘the authors … emphasise that “even though there is some evidence of heterogeneous effects, none of these sub-groups are losing [my italics] from a higher private school share” (Böhlmark and Lindahl, 2007, p. 27).’
Sahlgren agrees with Allen’s portrayal of the results of Böhlmark and Lindahl’s 2008 study finding ‘a positive impact of competition on the percentage of students choosing academic programmes in upper-secondary school’ and that the positive effect did not seem to translate to higher grades in upper-secondary school or university attainment, but he includes the authors’ explanation: the effects of competition were not more marked because ‘the entry of new private schools not has been followed by the closing down of public schools’ (Böhlmark and Lindahl, 2008, p. 23, quoted in Sahlgren, pp. 7-8).
Sahlgren further finds that Allen’s analysis was more sober than it might have been had she included a study published (unfortunately only in Swedish) in April 2010. Tegle’s study analysed the educational achievement of all 9th grade students in 2006 (as opposed to just a sample, as in previous studies). He found ‘a significant effect of competition on municipal school students’ GPAs for students from all socio-economic backgrounds: a 10% increase of students in independent schools increases the municipal school average GPA by up to 2%, while increasing the performance on the standardised test in mathematics by up to 5.9%. Furthermore, Tegle [showed] that students in independent schools do significantly better than peers in municipal schools. The effect of attending an independent school equates to a 21% increase in the GPA and, even more astonishingly, a 33% increase in scores on the standardised mathematics test’ (Sahlgren, p. 8).
Taken together these studies indicate a more positive overall effect than has previously been suggested. In his too-often-overlooked study, Sahlgren argues that voucher reforms would have had far greater system-wide impact were it not for the adverse effects of the economic difficulties of the period, and those associated with other unsuccessful educational reforms.
James Croft is an IEA education research fellow and Director of the Centre for Market Reform of Education
Allen was right to highlight the difficulties of constructing a long enough panel of data to account for both pre- and post-intervention trends as the main reason for the paucity of such research. She also commented on a methodological problem from which several studies in the early part of the decade suffered in failing to account for the non-random application of the policy across different municipalities and for the peculiar demographic characteristics of areas with many free schools (Allen, p. 5).
In assessing how effectively key studies have tackled these problems, Allen’s survey found that the most recent (Böhlmark and Lindahl, 2007; 2008) employed the most robust data and methods and required relatively few identification assumptions. They were able to construct a long panel of data, from 1988/89 to 2002/03, including three years of pre-intervention data. Böhlmark and Lindahl’s results showed ‘a moderately positive impact of free school growth on municipality academic performance at the end of ninth grade (the end of lower secondary school, when pupils are aged 15-16)’ (Allen, p. 3).
Allen went on to qualify this finding by emphasising that those that benefited most according to Böhlmark and Lindahl were the children of educationally aspirant parents. This subsequently much publicised interpretation of Böhlmark and Lindahl, however, has more recently been challenged by Gabriel Sahlgren of the IEA. Sahlgren clarified that while the positive effect for students with low-educated parents or an immigrant background was indeed insignificant, ‘students from low-income families benefit more than those from high-income families’ [my italics]. Furthermore, ‘the authors … emphasise that “even though there is some evidence of heterogeneous effects, none of these sub-groups are losing [my italics] from a higher private school share” (Böhlmark and Lindahl, 2007, p. 27).’
Sahlgren agrees with Allen’s portrayal of the results of Böhlmark and Lindahl’s 2008 study finding ‘a positive impact of competition on the percentage of students choosing academic programmes in upper-secondary school’ and that the positive effect did not seem to translate to higher grades in upper-secondary school or university attainment, but he includes the authors’ explanation: the effects of competition were not more marked because ‘the entry of new private schools not has been followed by the closing down of public schools’ (Böhlmark and Lindahl, 2008, p. 23, quoted in Sahlgren, pp. 7-8).
Sahlgren further finds that Allen’s analysis was more sober than it might have been had she included a study published (unfortunately only in Swedish) in April 2010. Tegle’s study analysed the educational achievement of all 9th grade students in 2006 (as opposed to just a sample, as in previous studies). He found ‘a significant effect of competition on municipal school students’ GPAs for students from all socio-economic backgrounds: a 10% increase of students in independent schools increases the municipal school average GPA by up to 2%, while increasing the performance on the standardised test in mathematics by up to 5.9%. Furthermore, Tegle [showed] that students in independent schools do significantly better than peers in municipal schools. The effect of attending an independent school equates to a 21% increase in the GPA and, even more astonishingly, a 33% increase in scores on the standardised mathematics test’ (Sahlgren, p. 8).
Taken together these studies indicate a more positive overall effect than has previously been suggested. In his too-often-overlooked study, Sahlgren argues that voucher reforms would have had far greater system-wide impact were it not for the adverse effects of the economic difficulties of the period, and those associated with other unsuccessful educational reforms.
James Croft is an IEA education research fellow and Director of the Centre for Market Reform of Education
SHARE
