Earlier this month, the OECD published some research on all-round education excellence. Specifically, this is the proportion of people who excel in all three of reading, maths and science. The research was covered in a blog by an OECD staff member from the Directorate for Education and Skills. The Telegraph reported the research and quoted Education Minister Liz Truss as saying: ‘This analysis highlights the importance of reading, maths and science for our future economic prosperity. But it also demonstrates that we lag behind the world leaders.’

This struck me as slightly strange because the Telegraph report focused on the very high level of all-round achievement by students in Chinese regions and China has a national income per head (on a purchasing power parity basis) of less than a quarter of the UK level and is 93^rd in the national income ‘league table’. China may be growing rapidly; and education may or may not have something to do with China’s growth; but China is mainly benefiting from ‘catch-up’ growth.

Having had my curiosity aroused by the idea that we would become more prosperous if we copied China, I thought it worth investigating the data and the OECD report further. First the data…



A quick scan did not reveal any obvious relationship between prosperity and all-rounder achievement. A scatter plot then only revealed that there was little to reveal. It would be just as easy to draw a line that suggested a negative relationship between all-rounder achievement and national income rank as one that suggested a positive relationship. It should be noted that the all-rounder rank is limited to those countries that were above the average score for OECD countries and the national income rank is the ranking within the same group of countries. Of course, you can miss a lot by looking at scatter plots, so the rank correlation coefficient was calculated. This was -0.07 on a scale that goes from -1 to 1 with 1 being perfect positive correlation and 0 being no correlation. Minus 0.07 is as close to zero as makes no difference.

Bi-variate relationships can hide a lot. This was one of the traps the authors of The Spirit Level fell into. For example, there might be a third factor which has a strong negative relationship with national income and which is coincidentally related to the all-rounder score whilst the ‘all-rounder’ score is positively related to national income.

Another common trap is to rely too much on a subset of the data. However, further examination suggested that these data would have confounded the statistical equivalent of a mediaeval torture chamber. Firstly, I suspect that the relationship between the level of national income (rather than the ranking of the countries) and the all-rounder score would be more strongly negative (because some very-low-income countries such as China would have greater weight). Secondly, if you look further into the data, you find some high income countries with very low scores. Thirdly, within income groups, countries such as Chile (which is a relatively rich middle income country) have much lower scores than countries with lower incomes such as Uruguay.

So, there is there little evidence of a link between the all-rounder score and prosperity. What about the minister’s claim that the research ‘highlighted’ the link between all-rounder score and future prosperity? Here, the minister might just scrape through if one were to be very pedantic about the OECD’s semantics. In fact, there is nothing in the research at all which examines the link between the all-rounder score and either current or future prosperity (and, as we have seen, this is hardly surprising). Interestingly, though, the OECD blog author covering the research asserts that there ought to be a link, as does the report itself. She says: ‘Why do – or should – countries care about the number of all-rounders they produce? Knowing the proportion of students who excel in these three subjects helps countries to determine the depth of their future talent pool, which has significant implications for a country’s ability to compete and grow in an increasingly information-based global economy.’

There is no evidence whatsoever for this in the published research and it is not even intuitively obvious. The opportunity cost of investments foregone resulting from more investment in education might be damaging to growth to a greater extent than better education increases growth – though education reform might generate better outcomes for lower cost. However, when it comes to the all-rounder study, we are looking at something very specific. For a given average level of attainment Liz Truss and the author of the blog are arguing that more all-round achievers will lead to greater prosperity. So, if there are 100 people in the country, it is better for prosperity if 10 of them are excellent at reading, science and maths and 90 mediocre at everything, rather than 10 being excellent at reading, 10 being excellent at science and 10 being excellent at maths with 70 being mediocre at everything. It might be better if medical doctors are brilliant at trigonometry too whilst we have fewer specialist mathematicians, but, in a world of free trade and the division of labour, this is by no means obvious. Clearly if everybody is better at everything then – all other things being equal – this may carry some advantage. However, this is not what the study is about.

Recently the IEA published Quack Policy. As it happens this area of economics is not a specialism of mine (clearly, I am not enough of an all-rounder). From beginning to end, though, this story is a risible example of Quack Science. The OECD published a study which is slightly interesting but seems to me to be published to benefit the bureaucracy rather than its taxpayer-funders. There is no evidence whatsoever in that study for the weak claim it makes between future prosperity and all-round educational achievement. The attempt to make such a link smacks of trying to justify the jobs of taxpayer-funded OECD policy advisers. The education minister than links all-round education performance with our future prosperity and claims that the OECD study highlights a link that is, in fact, simply asserted. Of course, there may be a link between today’s all-rounder achievement and future prosperity. We will not know for 30 years. However, all that has been produced is a bunch of figures in a four page report together with a claim of the data’s potential importance which is not even intuitively obvious and for which no evidence is presented.

The much-maligned GCSE history syllabus requires that pupils examine sources and weigh up evidence. Perhaps ministers are being too quick to replace this by traditional approaches to teaching history. We need more examination of sources before statements are made by politicians about those sources and more weighing up of evidence before policy conclusions are drawn. If the OECD report were a GCSE history source for a piece of coursework examining the relationship between all-round educational achievement and prosperity, it would simply have to be ignored by any student who aspired to a grade C.