The bluff of ‘evidence-based policy’
22 August 2013
SUGGESTED
Uncategorized
Uncategorized
Economic Theory
25 June 2024
Jamie Whyte’s superb new book Quack Policy, published by the IEA this week, casts a sceptical eye over recent examples of alleged “evidence-based policy” in the UK, including minimum pricing for alcohol, banning smoking in enclosed spaces and various proposals based on the “science of happiness”. At the heart of his critique is a concept that is fundamental in economics but sadly absent when single-issue campaigners take the stage: trade-offs.
It may be true, for example, that raising the price of alcohol will reduce the number of alcohol-related injuries, just as it may be true that lowering the speed limit will reduce the number of traffic accidents. From this narrow perspective, demands for a minimum price of 50p for a unit of alcohol and a 20 mph limit on urban roads can be viewed as “evidence-based”. Politicians may feel inclined to act upon such evidence by passing fresh legislation. After all, drunken injuries and vehicular collisions are undeniably a bad thing.
However, enjoying a drink and getting from A to B are also good things. Having money in your pocket and the freedom to live your life as you wish are good things. If you want fewer drunken injuries, you can them, but it will cost billions of pounds in higher drink prices and might mean more moonshine and drug use. You can have fewer car accidents too, but it will involve slowing the country down to a crawl and being less efficient. In any policy, there are costs and benefits; winners and losers. Any cost-benefit analysis that ignores the good and focuses only on the bad is not worth the paper it is written on. The failure to acknowledge trade-offs and competing preferences is, Whyte argues, endemic in much of so-called evidence-based policy.
A minimum price of 50p might be an “evidence-based policy” if the only aim is to deter people from drinking, but so would a minimum price of 70p, £1.50, or—most effectively of all—£500. Similarly, a speed limit of 10 mph would be more evidence-based that a speed limit of 20 mph if the sole aim is to reduce car accidents, but a speed limit of 2 mph would be better still. Almost everybody would regard 2 mph speed limits and a £500 per unit minimum alcohol price as being so silly that they hardly need to bother explaining why. The reason, surely, is that such policies would deny people the pleasures of drinking and the benefits of driving.
The negative consequences are obvious in such extreme examples as a 2 mph limit and a £500 minimum price, but there are costs to be borne any level. The job of policy-makers is to set the right balance, but they are unlikely to do so if the evidence they are given is unbalanced from the start. Advocates of higher taxes are quick to put a price on the “emotional costs” of being distressed or injured by drunks, but they never balance the account by putting a price on the enjoyment of a night on the tiles. Well-being and consumer surpluses are not included in such analyses, presumably because it would weaken the case for action. But any serious attempt to study costs and benefits must show all the consequences of the policy, not only for health but for economic efficiency and life satisfaction. Alas—and this is Whyte’s point—serious attempts at cost-benefit analyses are seldom produced by policy-oriented campaigners and, therefore, serious evidence seldom appears. If, as he writes, “we choose to ignore the loss of benefits that also comes from reduced [alcohol] consumption, we know in advance the result we will get; we know the policy will appear to produce a benefit.”
Whyte saves his sharpest barbs for the risible realm of “happiness studies” which, he concludes, is “not even a pale imitation of proper sciences”. Buttressed by little more than self-reported survey data, happiness gurus such as Richard Layard advocate higher taxes, restrictions on people’s freedom of movement and bans on advertising as evidence-based policies to improve the nation’s well-being. The evidence they present is banal, rooted in a fatuous attempt to portray subjective expressions of happiness as if they were quantifiable units like a joule or a gramme, and the policies they espouse largely reflect the petty prejudices of wealthy, left-leaning, white males of a certain age. Only the woolliest mind could believe that the appeal to “happiness studies” makes the personal preferences of neo-utilitarians “evidence-based”.
Turning personal preferences into putative fact is what the worst of so-called evidence-based policy is all about. The evidence itself is often unsound. The infamous study that estimated the benefits of minimum pricing is riddled with faulty assumptions and has produced some extraordinarily sloppy errors. The happiness gurus ignore the obvious problem that their surveys rely on self-reporting in a changing context, and the epidemiology of passive smoking is a mess of conflicting studies that would have been put to bed years ago had the possibilities for social engineering been less profound. Although Whyte discusses some of the flaws in the evidence presented by campaigners, that is not the main purpose of his monograph. It is not necessary to debunk specific claims in order to expose the intellectual shortcomings of single-issue campaigners. With the exception of the “science of happiness”, which he regards as irredeemable flim-flam, Whyte generously assumes that the standard claims about passive smoking, minimum pricing and global warming are broadly correct. His argument is that the policies based on these claims are flawed because they fail to acknowledge that people have different preferences and are capable of offsetting risks against benefits. Paternalistic campaigners present a one-sided, idiosyncratic cost-benefit analysis and call it evidence. It is, as Whyte says, a “bluff”. He concludes that…
“Many people seek to replace your preferences and decisions with their own. They do not necessarily seek material gain. They may simply be overwhelmed by an exaggerated sense of their own morality and wisdom: ‘if only others acted by the light of my wisdom, the world would be such a wonderful place!’ Whatever their motivation, such people are always on the lookout for the latest ‘scientific breakthrough’ that appears to justify their meddling.”
Since most people are too busy or uninterested to delve into the evidence itself, the mere proclamation that a policy is evidence-based is enough to persuade them of its merit. They place their trust in those who have impressive-sounding credentials, even if those experts are speaking about subjects that fall far outside their area of formal expertise (Dr Noam Chomsky is probably very sound when discussing liguistics, for example, and Professor Ian Gilmore knows a lot about the workings of the human liver). Those who dare to question the policy or the evidence upon which it supposedly rests are regarded as enemies of science who clutch instead to morality, religion or—if all else fails—that indefinable beast, “ideology”. It was therefore unsurprising to see Whyte’s book being denounced on Twitter yesterday before anyone could possibly have had the time to read it. The IEA, it seemed, had finally gone too far. It had published a book criticising the cult of evidence-based policy and was therefore on the side of irrationality and unreason.
No fair-minded person who bothers to read Quack Policy could conclude that Whyte is anti-science. On the contrary, it is clear that he thinks that evidence-based policies would be a good idea. Alas, many of the schemes that are pitched under that banner are nothing of the sort.
Evidence can be very good at showing us whether a policy has achieved its aim, but it is not so good at showing us what the policy should be and is quite useless when it comes to telling us what the aims of society should be in the first place. Much so-called evidence-based policy starts from a view that something must be done about a perceived problem and then gathers the data to show how to go about it. The subjective preferences that underlie the desire for political action are seldom questioned and are rarely rooted in science, or even philosophy. The belief that people in Britain drink too much, for example, can only be an opinion. Insofar as it can tested by evidence—by, for example, comparing alcohol consumption in the UK with our European neighbours—it is not compelling even as opinion. The number-crunchers behind the minimum pricing estimates are unable to tell us what the scientifically determined optimum level of drinking is and cannot give a credible answer to the question of why a 50p unit price is more evidence-based than 40p or 60p. It’s all a bluff. Whyte’s witty and thought-provoking book provides a valuable service in reminding us that evidence can be found almost anywhere for almost anything and that partial and biased evidence should be treated with profound scepticism.
It may be true, for example, that raising the price of alcohol will reduce the number of alcohol-related injuries, just as it may be true that lowering the speed limit will reduce the number of traffic accidents. From this narrow perspective, demands for a minimum price of 50p for a unit of alcohol and a 20 mph limit on urban roads can be viewed as “evidence-based”. Politicians may feel inclined to act upon such evidence by passing fresh legislation. After all, drunken injuries and vehicular collisions are undeniably a bad thing.
However, enjoying a drink and getting from A to B are also good things. Having money in your pocket and the freedom to live your life as you wish are good things. If you want fewer drunken injuries, you can them, but it will cost billions of pounds in higher drink prices and might mean more moonshine and drug use. You can have fewer car accidents too, but it will involve slowing the country down to a crawl and being less efficient. In any policy, there are costs and benefits; winners and losers. Any cost-benefit analysis that ignores the good and focuses only on the bad is not worth the paper it is written on. The failure to acknowledge trade-offs and competing preferences is, Whyte argues, endemic in much of so-called evidence-based policy.
A minimum price of 50p might be an “evidence-based policy” if the only aim is to deter people from drinking, but so would a minimum price of 70p, £1.50, or—most effectively of all—£500. Similarly, a speed limit of 10 mph would be more evidence-based that a speed limit of 20 mph if the sole aim is to reduce car accidents, but a speed limit of 2 mph would be better still. Almost everybody would regard 2 mph speed limits and a £500 per unit minimum alcohol price as being so silly that they hardly need to bother explaining why. The reason, surely, is that such policies would deny people the pleasures of drinking and the benefits of driving.
The negative consequences are obvious in such extreme examples as a 2 mph limit and a £500 minimum price, but there are costs to be borne any level. The job of policy-makers is to set the right balance, but they are unlikely to do so if the evidence they are given is unbalanced from the start. Advocates of higher taxes are quick to put a price on the “emotional costs” of being distressed or injured by drunks, but they never balance the account by putting a price on the enjoyment of a night on the tiles. Well-being and consumer surpluses are not included in such analyses, presumably because it would weaken the case for action. But any serious attempt to study costs and benefits must show all the consequences of the policy, not only for health but for economic efficiency and life satisfaction. Alas—and this is Whyte’s point—serious attempts at cost-benefit analyses are seldom produced by policy-oriented campaigners and, therefore, serious evidence seldom appears. If, as he writes, “we choose to ignore the loss of benefits that also comes from reduced [alcohol] consumption, we know in advance the result we will get; we know the policy will appear to produce a benefit.”
Whyte saves his sharpest barbs for the risible realm of “happiness studies” which, he concludes, is “not even a pale imitation of proper sciences”. Buttressed by little more than self-reported survey data, happiness gurus such as Richard Layard advocate higher taxes, restrictions on people’s freedom of movement and bans on advertising as evidence-based policies to improve the nation’s well-being. The evidence they present is banal, rooted in a fatuous attempt to portray subjective expressions of happiness as if they were quantifiable units like a joule or a gramme, and the policies they espouse largely reflect the petty prejudices of wealthy, left-leaning, white males of a certain age. Only the woolliest mind could believe that the appeal to “happiness studies” makes the personal preferences of neo-utilitarians “evidence-based”.
Turning personal preferences into putative fact is what the worst of so-called evidence-based policy is all about. The evidence itself is often unsound. The infamous study that estimated the benefits of minimum pricing is riddled with faulty assumptions and has produced some extraordinarily sloppy errors. The happiness gurus ignore the obvious problem that their surveys rely on self-reporting in a changing context, and the epidemiology of passive smoking is a mess of conflicting studies that would have been put to bed years ago had the possibilities for social engineering been less profound. Although Whyte discusses some of the flaws in the evidence presented by campaigners, that is not the main purpose of his monograph. It is not necessary to debunk specific claims in order to expose the intellectual shortcomings of single-issue campaigners. With the exception of the “science of happiness”, which he regards as irredeemable flim-flam, Whyte generously assumes that the standard claims about passive smoking, minimum pricing and global warming are broadly correct. His argument is that the policies based on these claims are flawed because they fail to acknowledge that people have different preferences and are capable of offsetting risks against benefits. Paternalistic campaigners present a one-sided, idiosyncratic cost-benefit analysis and call it evidence. It is, as Whyte says, a “bluff”. He concludes that…
“Many people seek to replace your preferences and decisions with their own. They do not necessarily seek material gain. They may simply be overwhelmed by an exaggerated sense of their own morality and wisdom: ‘if only others acted by the light of my wisdom, the world would be such a wonderful place!’ Whatever their motivation, such people are always on the lookout for the latest ‘scientific breakthrough’ that appears to justify their meddling.”
Since most people are too busy or uninterested to delve into the evidence itself, the mere proclamation that a policy is evidence-based is enough to persuade them of its merit. They place their trust in those who have impressive-sounding credentials, even if those experts are speaking about subjects that fall far outside their area of formal expertise (Dr Noam Chomsky is probably very sound when discussing liguistics, for example, and Professor Ian Gilmore knows a lot about the workings of the human liver). Those who dare to question the policy or the evidence upon which it supposedly rests are regarded as enemies of science who clutch instead to morality, religion or—if all else fails—that indefinable beast, “ideology”. It was therefore unsurprising to see Whyte’s book being denounced on Twitter yesterday before anyone could possibly have had the time to read it. The IEA, it seemed, had finally gone too far. It had published a book criticising the cult of evidence-based policy and was therefore on the side of irrationality and unreason.
No fair-minded person who bothers to read Quack Policy could conclude that Whyte is anti-science. On the contrary, it is clear that he thinks that evidence-based policies would be a good idea. Alas, many of the schemes that are pitched under that banner are nothing of the sort.
Evidence can be very good at showing us whether a policy has achieved its aim, but it is not so good at showing us what the policy should be and is quite useless when it comes to telling us what the aims of society should be in the first place. Much so-called evidence-based policy starts from a view that something must be done about a perceived problem and then gathers the data to show how to go about it. The subjective preferences that underlie the desire for political action are seldom questioned and are rarely rooted in science, or even philosophy. The belief that people in Britain drink too much, for example, can only be an opinion. Insofar as it can tested by evidence—by, for example, comparing alcohol consumption in the UK with our European neighbours—it is not compelling even as opinion. The number-crunchers behind the minimum pricing estimates are unable to tell us what the scientifically determined optimum level of drinking is and cannot give a credible answer to the question of why a 50p unit price is more evidence-based than 40p or 60p. It’s all a bluff. Whyte’s witty and thought-provoking book provides a valuable service in reminding us that evidence can be found almost anywhere for almost anything and that partial and biased evidence should be treated with profound scepticism.
SHARE
