Is The Spirit Level right about inequality and obesity?
27 July 2013
SUGGESTED
Uncategorized
27 July 2013
Uncategorized
Housing and Planning
Many have expressed the opinion that the very simple correlations shown in The Spirit Level alone can never show meaningful causality – this is correct. However, people can always claim the academic literature shows the same thing. Unfortunately this gulf will always exist, and the public (and even occasionally other academics) can be tricked into believing a relationship, because there is a so called academic ‘consensus’.
In this article I will present 4 simple questions everyone should ask before they believe the results of some revolutionary new study that claims x is causing y, including any of the claims made in The Spirit Level. In this context I will also explain the results of my recent paper in Economic Affairs, which is probably the first analysis to use UK data to test one of the claims of The Spirit Level. In particular, we investigated whether regional and district level inequality (Gini coefficient) determines individual likelihood of obesity.
% obese men vs. inequality measure
Firstly, is the study adequately designed to answer the question? In other words, does a graph like the one above, even if it accounts for all the problems I discuss below, help us discover whether living in a more unequal area increases your likelihood of being obese?
We know what roughly determines weight at the individual biological level (genetics, age, gender, energy consumption and energy expenditure) and the suggested relationship goes from higher inequality to feelings of low status, which are supposed to induce behaviours such as ‘comfort eating’, which then increases weight.
If this is the theorised causal sequence involved, then it makes far more sense to sample individuals and see how individual weight (i.e. BMI) differs according to the inequality in their geographical area. In fact as I explain in the paper, even the other academic studies that look at this issue using a cross-country approach suggest the next stage of research should be of this sort. This is what economists call multi-level models and this is the approach we take.
Secondly, is there a third (fourth? fifth? tenth?) variable driving the relationship? More precisely, are there ‘confounding’ factors determining x and y so that there only appears to be a causal relationship? Economists use a statistical tool called multiple regression analysis to help ‘control’ for these other factors. The most intuitive way to ‘control’ for stuff would be to go out and collect a sample of people that were identical in terms of all possible confounders and then see what relationship there is between x and y – this is very impractical so instead we collect a random sample and imitate this process with regression analysis.
The Wilkinson and Pickett paper from which the figure is taken, controls for gross national income per capita – but why not control for average aggregate versions of all our controls, among a variety other things (e.g. poverty levels)? Some of the nine multi-level studies we found lacked key controls: for example two studies did not control for individuals’ education and three seemed to be using imperfect proxies for individual or family income. Yet these are well known determinants of BMI and may be confounders – e.g. maybe being poor implies there are more poor people around you which translates into a higher regional/district Gini?
Informed by the decades of research into the individual determinants of obesity (and with some intuition), we chose a wide range of controls for our study which should help overcome some of these issues. There could always be confounding factors we have missed, but improving the number of controls helps us get as close as possible to a sense of causality.
Thirdly, is there reverse causation in the relationship? The simple case of this is that y is actually determining x, the converse of which should not show up if we have a good set of controls. Otherwise this is a particularly difficult issue statistically: even if we were to have perfectly controlled for everything we will not be able to isolate the effect of x on y, if y is also simultaneously determining x. Economists call this an equilibrium process and a classic example is the level of crime and police in an area. Fortunately, because it is very unlikely that an individual’s weight is determining inequality in their area, this should not be a problem for our study. But is it possible in aggregate studies, of the kind shown in the figure? It does not seem implausible.
Fourth: is there a statistically significant relationship? What if I repeated my study identically but instead of using a sample of around 10,000 we only sampled 10 people, and found that inequality has a very large effect on BMI (i.e. a high ‘economic’ significance). Would you be convinced the relationship holds for the population? Not really – this is why economists have to consider the somewhat subjective notion of statistical significance, as well as economic significance. The likelihood of statistical significance rises as the effect size increases or the sample size increases. Remember when observing any of the graphs in The Spirit Level, there is usually no indication of statistical significance.
All effects we found were statistically insignificant at commonly used benchmarks, except for some evidence of a district-level effect, but this was economically insignificant compared to other determinants of weight. Results across other studies were also mixed.
In conclusion, the reader should ask these 4 questions when confronted with any statistical claims – including any of the graphs from The Spirit Level. Further still, we can say that there is currently not enough evidence to suggest a meaningful relationship between inequality and obesity for the UK – or for that matter any country.
Darshan Zala is the author of ‘Challenging The Spirit Level: Is There Really a Relationship between Inequality and Obesity?, published in the June 2013 issue of Economic Affairs.
In this article I will present 4 simple questions everyone should ask before they believe the results of some revolutionary new study that claims x is causing y, including any of the claims made in The Spirit Level. In this context I will also explain the results of my recent paper in Economic Affairs, which is probably the first analysis to use UK data to test one of the claims of The Spirit Level. In particular, we investigated whether regional and district level inequality (Gini coefficient) determines individual likelihood of obesity.
% obese men vs. inequality measure
Firstly, is the study adequately designed to answer the question? In other words, does a graph like the one above, even if it accounts for all the problems I discuss below, help us discover whether living in a more unequal area increases your likelihood of being obese?
We know what roughly determines weight at the individual biological level (genetics, age, gender, energy consumption and energy expenditure) and the suggested relationship goes from higher inequality to feelings of low status, which are supposed to induce behaviours such as ‘comfort eating’, which then increases weight.
If this is the theorised causal sequence involved, then it makes far more sense to sample individuals and see how individual weight (i.e. BMI) differs according to the inequality in their geographical area. In fact as I explain in the paper, even the other academic studies that look at this issue using a cross-country approach suggest the next stage of research should be of this sort. This is what economists call multi-level models and this is the approach we take.
Secondly, is there a third (fourth? fifth? tenth?) variable driving the relationship? More precisely, are there ‘confounding’ factors determining x and y so that there only appears to be a causal relationship? Economists use a statistical tool called multiple regression analysis to help ‘control’ for these other factors. The most intuitive way to ‘control’ for stuff would be to go out and collect a sample of people that were identical in terms of all possible confounders and then see what relationship there is between x and y – this is very impractical so instead we collect a random sample and imitate this process with regression analysis.
The Wilkinson and Pickett paper from which the figure is taken, controls for gross national income per capita – but why not control for average aggregate versions of all our controls, among a variety other things (e.g. poverty levels)? Some of the nine multi-level studies we found lacked key controls: for example two studies did not control for individuals’ education and three seemed to be using imperfect proxies for individual or family income. Yet these are well known determinants of BMI and may be confounders – e.g. maybe being poor implies there are more poor people around you which translates into a higher regional/district Gini?
Informed by the decades of research into the individual determinants of obesity (and with some intuition), we chose a wide range of controls for our study which should help overcome some of these issues. There could always be confounding factors we have missed, but improving the number of controls helps us get as close as possible to a sense of causality.
Thirdly, is there reverse causation in the relationship? The simple case of this is that y is actually determining x, the converse of which should not show up if we have a good set of controls. Otherwise this is a particularly difficult issue statistically: even if we were to have perfectly controlled for everything we will not be able to isolate the effect of x on y, if y is also simultaneously determining x. Economists call this an equilibrium process and a classic example is the level of crime and police in an area. Fortunately, because it is very unlikely that an individual’s weight is determining inequality in their area, this should not be a problem for our study. But is it possible in aggregate studies, of the kind shown in the figure? It does not seem implausible.
Fourth: is there a statistically significant relationship? What if I repeated my study identically but instead of using a sample of around 10,000 we only sampled 10 people, and found that inequality has a very large effect on BMI (i.e. a high ‘economic’ significance). Would you be convinced the relationship holds for the population? Not really – this is why economists have to consider the somewhat subjective notion of statistical significance, as well as economic significance. The likelihood of statistical significance rises as the effect size increases or the sample size increases. Remember when observing any of the graphs in The Spirit Level, there is usually no indication of statistical significance.
All effects we found were statistically insignificant at commonly used benchmarks, except for some evidence of a district-level effect, but this was economically insignificant compared to other determinants of weight. Results across other studies were also mixed.
In conclusion, the reader should ask these 4 questions when confronted with any statistical claims – including any of the graphs from The Spirit Level. Further still, we can say that there is currently not enough evidence to suggest a meaningful relationship between inequality and obesity for the UK – or for that matter any country.
Darshan Zala is the author of ‘Challenging The Spirit Level: Is There Really a Relationship between Inequality and Obesity?, published in the June 2013 issue of Economic Affairs.
9 thoughts on “Is The Spirit Level right about inequality and obesity?”
Comments are closed.
SHARE
Darshan Zala,
The paper in the ‘J Epidemiol Community Health’, in the following link I provide gives all the necessary regression analyses and finds a positive association:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1733121/pdf/v059p00670.pdf
In addition, there are many other academically peer reviewed papers that support there is a positive association. Your article on the other hand only makes the claim that it does not with questions to ask; your paper is not freely accessible from Economic Affairs’ to test your claim, as in the above link I have provided.
Finally, you are not a specialist in the field of epidemiology and nor would there be in either Economic Affairs to peer review your work to the standards of epidemiology.
So, I would follow the opinions of experts in the field.
Hi Petrov
Thank you for the comment – sorry for the late reply, I had no idea anyone had commented.
1) That paper contains almost all the flaws I mentioned above.
2) Your right there are some papers – far better than the one you mentioned – that show a positive correlation but the point is that there are as many that show a negative correlation, which is what i explain in my paper.
3) Although only the contents and arguments of papers should matter I have been doing panel data econometrics for more than 4 years. And I am currently working as a health economist for a private consultancy.
Darshan Zala,
So, it would be nice if you could provide the source or the link to the far better paper(s) to the one I have provided the link to. Even better, please provide the many sources and or the links that you claim that show negative correlation – it is much easy to say that, that is the case, rather than show that such is the case.
In addition, The Spirit Level has been peer reviewed and is consistent with the high standards in that specialty which attests to its credentials in good research. There is scientific consensus that inequality in society produces poor outcomes in many areas in life – at the community or national level and at the individual level. This evidence has been amassed over the last 30 years.
The authors of the The Spirit Level look at the international data which is not there data and posed a theory based on that 30 or more years of peer reviewed research to explain and do explain such outcomes. Their theory shows it has predictive value and shows a remarkable consistency.
Regards
With all due respect Petrov you sound like you don’t know what your talking about or you have no experience with any of the econometric etc evidence in this area. I’m going to reply one last time…and this should lead you in the right direction in terms of reading more studies etc. But at the moment you sound very ideological.
1) There is whatsoever no academic consensus that inequality (not poverty) implies more obesity and weight status…ignoring the small review I conducted in my paper, the academics I have spoken to in Warwick economic dept and the Sheffield economics/health economic dept either say this is a very new area or the analysis presented in the spirit level and the paper we are talking about is not robust.
2) THERE IS NOT 30 YEARS OF STATISTICAL EVIDENCE on the claim that INEQUALITY determines weight etc.
3) The claims concerning published studies mentioned in the spirit level are themselves OBVIOUSLY NOT peer reviewed, but your right that it references peer reviewed papers like the one you mentioned.
4) BUT the paper you show is sooo simple that if you read it you will realise that they do not claim any strong causality. …If they did with such a simple analysis, I assure you no one would publish it.
5) Why can’t you look at the content of my blog? If you knew anything about stats you would know these concerns about the paper you mentioned are valid. The points I made are very basic aswell.
6) If you really want to see more papers – and usually more sophisticated individual level studies – search a database like medline or econlit. Most of these are not mentioned in the spirit level or the associated paper because neither claims to be at minimum even a thorough review. OR read my paper before your comment….BUT because I’m a gentleman if you give me your email address I will send you a table summarising these other studies.
Otherwise this conversation is going nowhere
Darshan Zala,
In relation to your leading paragraph on ideology, does not the IEA accepts and continue to receive large sums of money from the tobacco industry to which the IEA favours that industry’s position through its (IEA) publications? Isn’t casting doubt on the medical science linking smoking to cancer a strategy used by the tobacco industry for the last 30 to 40 years?
In relation to point (1), “In 2011 the Joseph Rowntree Foundation commissioned an independent review of the evidence about the impact of inequality, paying particular attention to the evidence and arguments put forward in “The Spirit Level”. It concluded that the literature shows general agreement about a correlation between income inequality and health/social problems…” see: https://en.wikipedia.org/wiki/The_Spirit_Level:_Why_More_Equal_Societies_Almost_Always_Do_Better . Though the review has its limitations (see: http://www.equalitytrust.org.uk/sites/default/files/jrf-response.pdf ), had the review been more thorough it would have supported a stronger conclusion that inequality was a causal factor. Secondly, a quick look at the internet, the following abstracts show like consensus to me:
• http://www.ncbi.nlm.nih.gov/pubmed/14723911
• http://www.bariatricnews.net/?q=news/11408/luxembourg-socioeconomic-inequalities-linked-obesity
• http://www.sciencedirect.com/science/article/pii/S1570677X10000663
• http://onlinelibrary.wiley.com/doi/10.1111/j.1467-789X.2007.00312.x/full
• http://www.sciencedirect.com/science/article/pii/S0277953612002961
• http://onlinelibrary.wiley.com/doi/10.1111/j.1751-9020.2010.00355.x/abstract
• http://ukcatalogue.oup.com/product/9780197264980.do
• http://www.britishacademypublications.com/view/10.5871/bacad/9780197264980.001.0001/bacad-9780197264980-chapter-11
• http://link.springer.com/article/10.1007/s10389-008-0185-x
• http://www.noo.org.uk/NOO_about_obesity/inequalities
• http://www.ncbi.nlm.nih.gov/pubmed/18280021
• http://papers.ssrn.com/sol3/papers.cfm?abstract_id=979189
In relation to point (2), I made the general claim in relation to the work in the Spirit Level. If you look in the reference section of the book you will find that the material they rely on go back that far – 30 years. But in their 2nd edition 2010, p91 they mention that the WHO set up a study in the 1980’s monitoring various diseases including obesity with measuring social gradients.
In relation to point (3) the claims they make are supported in the material they have referenced of which there are close to 400 peer reviewed papers. They themselves have published 100 articles in peer reviewed journals. Further, the Spirit Level is specifically a theory of problems which have social gradients, that is problems which become more common further down the social ladder and they test that theory on international data together with USA stats as the control. There is a lot of public health science that supports this position – you are just looking in the wrong places.
In relation to point (4) are we talking about the same paper? It has been published in the ‘J Epidemiology Community Health’ so it’s been peer reviewed – so it’s got standing – right!
In relation to your point (5) Students of statistics learn about those basic traps that you have listed in your blog. R. Wilkinson is an emeritus profession of epidemiology. No the issue is not because R. Wilkinson does not know the basics on statistics or that the Spirit Level lacks rigour. Rather, your blog is about confusing and creating doubt in the minds of the uninitiated on statistics, over a study that is well grounded and robust. Such criticisms are used by the tobacco industry of which the IEA has a strong financial relationship with.
In relation to point (6), Well, you do say in your earlier entry 08/02/2014 in response to my entry 05/10/2013 that there are better papers that show a positive correlation than the paper I have cited. Please make good on your promise. I see no reason why you can’t provide the details on your bog. However IEA has my email.
Regards
Petrov this discussion is very pointless: I cant decide if your being ideological or you have failed to understand the difference between poverty/low socieconomic status (low SES).
1) I cant comment on your claims about the IEA because im not affilliated with them i just published in that journal while I was an intern there. Im also going to ignore the general review you mentioned from the rowntree foundation becasue there are other general reviews from other think tanks (centre or centre right or whatever) that come to different conclusions. Im also going to ignore your point about wilkinson being a professor, there are many professors of economics and statistics that would be dubious about the statistics presented in the spirit level etc.
2) Anyway the papers you present have probably come up in my review (but its been a while since ive looked at mine), you understand why they were discounted? They are not about inequality causing obesity. They are about SOCIAL GRADIENTS having a correlation with obesity. All they are are saying is that if people have less SES (lower ed, income etc) then they are more likely to be obese…that is failry obvious now and most people dont disagree with that claim. Its also quite intuitive. I assume that the WHO study is also about that. THE SPIRIT LEVELS MAIN POSITION IS NOT THAT SOCIAL PROBLEMS HAVE SOCIAL GRADIENTS. ITS THAT INEQUALITY CONTROLLING FOR INCOME ETC CAUSES E.G. OBESITY. The claim is completely different (just look at the graphs in the book!): they are saying that if you have 2 people of the same objective SES (income etc) and 1 lives in a society with a higher gini and the other in a society with a lower gini, despite them both being equivelent on the absolute scales the one in the higher inequality society will be more likely to be fat (e.g. becasue there are 10 more billionairs in the city, say) – DO YOU UNDERSTAND THE DIFFERENCE? Perhaps the spirit level mixes these 2 positions up…it is quite common!
2) Yes that paper has been peer reviewed and published – BUT IT WOULDN’T HAVE BEEN IF THEY STATED IT IMPLIES REAL CAUSATION (i.e. more strongly as they probably do in the book).
3) yes im pretty sure none of the papers you mention are relevent to this debate (but ive only seen some of them…im a bit busy)…they are about ABSOLUTE not RELATIVE positions, but its confusing becasue sometimes they use the same language: they say “inequality” to the extent that if your poorer your more likely to be obese so that obviously if society was less unequal to the extent these people were less poor there would be less obesity. I agree with this position BUT this is not the position of the spirit level you must understand…they are saying CONTROLLING for income etc, inequality causes obesity…if that makes sense?
Anyway i dont know how I can get your email, you can give it to me now (shouldnt be a big deal)? or add me on facebook or something (you have my full name…i live in uk) and I can send you this table? This really would be easier.
I have your email address I’ll send you some information.
I have sent you some relevant literature to help you get feel for why the analysis in the spirit level and the associated paper I also mentioned in this blog is not very convincing – i,e. there are better studies some of which show a positive and some of which show negative or no relationships, so there is no clear conclusions.
But I think i understand why we disagree, I really think you have misunderstood the difference between absolute differences and relative differences in SES (or income) and the their relationship with obesity/weight. I will quickly explain, in order, why each of the 12 papers you have given links to are not relevant to this debate about RELATIVE income levels.
1) This 1st paper is about “socioeconomic inequality in obesity”, which is about how obesity is spread unequally in society…in the sense that poorer people or people with less education are more likely to be obese. So this is about how absolute difference in SES translate into differences in obesity – BUT the spirit level concerns how inequality controlling for these things effects obesity. 2) “The researchers performed a multivariate logistic regression analysis to measure the relationship between obesity and demographic, socioeconomic status and behavioural factors” This is not about how inequality determines obesity but about these absolute variables obviously. 3) Thisis in my review but show the opposite to picket et al and agree with mine (but again has the floor it is not individual level): ” Economic insecurity, measured in several different ways, was almost twice as powerful, while the impact of inequality was weak, and went in the opposite direction”…implying inequality they agree with my analysis. 4) This paper is about “socioeconomic inequalities in health”, they say things like ” socioeconomic inequalities in obesity are stronger in girls than boys”. This is abit confusing at first but they are saying that obesity is more likely or clustered at certain points along these SES-income type scales so that people are unequally effected by obesity, which is completely different to the idea that inequality around you is causing obesity controlling for your income etc. 5) Not sure how the analysis is done but is country level again (not in my review i dont think), but seems to agree with pickett et al: “Adjusting for GDP, both the Gini index and GII were associated with excess female obesity”. 6)Its about stress caused by absolute concepts (im stressed, so overeat because I struggle to pay bills…a far more intuitive and meaningful explanation then inequality controlling for income etc) but is about absolutes again: ” I highlight elements of disadvantage at the individual-, family-, school-, and neighborhood-level that are linked to energy intake and expenditure, which are directly related to obesity”. 7)Cant tell much about this doesnt seem to be statistical but doesnt seem related: “It indicates that important drivers of obesity arise from the interaction between the external ‘shock’ of falling food prices and the enduring normative assumptions that govern society as a whole”. 8) same book as 7 9) the title of this is “Income-related inequality in the distribution of obesity among Europeans” so obviously its about how obesity is distributed along the income scale…just look at the conclusion and it becomes obvious: “Our primary results attest to the existing literature, showing that a BMI with a value greater than 30 is most likely to be an encumbrance for those of low socioeconomic profile…Effective preventive policies should address the low socioeconomic status female population in Europe” THEIR RECOMENDATIONS ARE SIMILAR TO MINE…OBESITY REDUCTION (WRT) OBESITY SHOULD BE ABOUT RAISING THE BOTTOM NOT DECREASING THE GAP (seem related but the latter can lead to more poverty in the long run given certain underlying tradeoffs..but thats a seperate debate). 10) This is about SOCIAL GRADIENTS IN HEALTH that means health changes across certain social scales, but thats about absolutes. They say for example “There is a ‘social gradient’ in health – the lower a person’s position in society, the worse their health [1]. In the UK, socioeconomic inequalities have increased since the 1960s and this has led to wider inequalities in both child and adult obesity, with rates increasing most among those from poorer backgrounds”…i.e. inequality is unequally spread because its based on things like income and education (i.e. SES). THE GRAPHS MEASURE OBESITY AGAINST DEPRIVATION SCALE AND EDUCATION (ABSOLUTE CONCEPTS) THERE IS NOT GRAPH SHOWING OBESITY AMONG DIFFERENT COMMUNITIES WITH DIFFEREING GINIS. 11) This seems to be another country level analysis but seems better than the picket et al study (there are a few country level ones in my review) and seems to agree with it: “In multivariable analyses primarily limited to industrialized countries, countries in the middle and highest (vs. lowest) tertiles of income inequality had higher absolute age-standardized obesity prevalences in both sexes”. but in my opinion is likely to have the same flaws as other country level studies. 12) This is about stress associated with absolute factors effecting weight (a respectable hypothesis), but again it is about income not inequality differences: “Something about being poor makes people fat. Though there are many possible explanations for the income-body weight gradient”.
I have difficulty accepting the arguments of an academic who demonstrates that his basic English is defective. “your” instead of “‘you are”.
Nevertheless, I have enjoyed reading your exchange.