Recently, critics of low-cost private schooling in developing countries – and there are many – had a field day. A study published in Harvard’s prestigious Quarterly Journal of Economics appeared to put the nail in the coffin of those who of us thought that poor parents, by choosing private education, might be doing something sensible.

The study presented a randomized controlled trial (RCT) from a school choice experiment in rural Andhra Pradesh, India, (Muralidharan and Sundararaman, 2015). Development experts hailed the study as the ‘gold standard’ of research in this area.

It featured a two-stage lottery to allocate private school places (“vouchers”) to village children, and to create suitable control groups. Children were tested in Telugu (the regional language), mathematics and English at the end of two and four years, while tests in science/social studies and Hindi were also given after four years.

The results show that there was no significant difference in achievement between the voucher children in private schools and those left behind in the government schools, apart from in Hindi. But only the private schools taught Hindi, so that difference was unsurprising.

There was one crumb of comfort: private schools were doing the same as government schools for less than one third of the cost. But, rightly so in my view, that wasn’t enough to stop the critics gloating: The Times of India opined “The findings dispel a popular myth that private schools lead to better learning”. The Economic and Political Weekly suggested “The empirical evidence is increasingly pointing towards private schools not being able to add value as compared to government schools.’

However, this is not the end of the story.

It turns out that there was a rather serious flaw in the research design, which renders the results null and void.

The aim of creating a RCT is to ensure that participants in treatment and control groups are treated in exactly the same way, apart from the intervention, i.e., school vouchers. In this study however, they were not treated the same.

The problem was the language of the tests used for mathematics and science/social studies. The tests had instructions in Telugu for children in government schools, but in English for half of the children in private schools. Why? Because half of the private schools advertised themselves as English-medium.

However, in poor villages like those in the study, and especially in the lower school grades, ‘English-medium’ is more aspiration than reality. Children in lower grades are taught in their mother tongue – Telugu – just as they are in the Telugu-medium schools. English-medium schools do try to push English harder so that by the upper grades – higher than those in this study – children can be taught in English, assuming that English-speaking teachers are available.

So giving tests to voucher children in the private schools in English, whether or not they advertised themselves as English-medium, would likely skew the results against the private schools. The questions in mathematics and science were frequently very wordy, impossible to answer if you didn’t have a fair understanding of English.

Because of this violation of a fundamental principle of RCTs, we simply don’t know how the private school children performed in mathematics and science/social studies, as they were given different tests.

There is good news, however. Tucked away in the research paper are the results for the half of the private schools that were Telugu-medium, compared to the government schools – that is, where children were given identical tests, rather than the different tests in the English-medium private schools.

This time, children with vouchers in the private schools outperformed those in the public schools in all subjects after four years of the voucher program; the combined result shows a large, statistically significant difference in favour of private schools. This is a hugely positive finding for the school choice debate.

Not only is it agreed by critics and supporters alike that private schools are more efficient than government schools, it turns out that, when identical tests are given to those in government and private schools, voucher children in private schools outperform those left behind in government schools too.

It looks like the critics of low-cost private schools may have been celebrating too early.

 

This blog is based on Prof Tooley’s response to Muralidharan and Sundararaman (2015):   

Tooley, James (2016) ‘Extending access to low-cost private schools through vouchers: an alternative interpretation of a two-stage ‘School Choice’ experiment in India’. Oxford Review of Education.