Why the government should abolish the Research Excellence Framework

Work is proceeding within universities on the assumption that in 2019 they will have to submit another Research Excellence Framework (REF) exercise. For those not totally up to speed with this exercise, it calls for a full peer evaluation of all universities’ research performance in three areas – publications, impact and ‘research environment’. Publications are graded by whether the publishing journals are ‘3’ or ‘4’ rated, these being the top two notches on a scale of 0-4; and it is proposed that the notches below 3 (even on some reports below 4) are to be ignored in the 2019 ‘round’; alternatively the publications may be read by an assessor which dubiously substitutes subjectivity for journal ranking. ‘Impact’ measures the effect of research on the wider world outside academia. ‘Environment’ measures such things as the ability to run PhD programmes and the support given to academics and PhD students in their research.

These research exercises date back to the early 1980s when they were instituted by Sir Keith Joseph in an effort to improve the research performance of UK universities at a time when the state essentially funded all of universities’ budgets, well before the advent of large-scale foreign student arrivals and the introduction of UK undergraduate student fees. Without a doubt they succeeded in their purpose. The question today, over three decades later, is whether the state’s involvement in organising university competition is still justified. Effectively the state is now driving the agenda in research and its organisation within universities in a way unheard of in the USA where the UK’s main university competitors are located. Indeed, the USA was the model Sir Keith had in mind that the UK, then a state monolith, should get good enough to emulate and he hoped that, once fit, it would become a market-driven sector where reputation and quality would get their market rewards. But as a result of the exercises still carrying on, a huge bureaucracy has been created both in the government agency for universities, HEFCE, and its satellites in Scotland, Wales and Northern Ireland, and also in the universities themselves. This bureaucracy is devoted to corralling academics to meet the aims of the exercise.

As usual with such grand centrally planned operations, form takes over from the substance of research. Quality is trampled underfoot in the rush to generate suitable scores for the exercise. Academic promotion or sackings are linked to these scores.[i] Administrators who are entirely ignorant of research regulate academics with orders and targets. All they know about is numbers – numbers of publications of suitable ‘grade’, numbers of students, pass rates, criteria of ‘impact’ and so on. These administrators have become the enemy of academic quality and the essential academic freedom that can generate it. Why bother with freedom, markets or competition when Soviet-style planning is sorting everything out and doling out the prizes?

In my view it is time to return universities to the disciplines of the market in higher education. The government no longer supplies university finance except in minor ways that will presumably in time disappear with the rest. Indeed even the exercises, originally justified by large-scale funding for research, no longer produce more than modest funding; and this remaining amount could easily be disbursed by the research councils as occurs in the US. We would be astonished and appalled if the government had a REF for the pharmaceutical industry, on the basis that it helps it directly or indirectly through R&D support. Yet higher education is a huge industry, on a par with pharmaceuticals and other research-orientated industries; government support and interest is welcome, dirigiste planning is not.

The market is infinitely preferable to bureaucracy and planning in its encouragement of research. The main reason for this is that the market permits competition and diversity. New journals spring up to challenge the existing wisdom as purveyed by the currently dominant journals, the 3 and 4 ones, whose conservatism prevents the free entry of new ideas. I know of academics who later got the Nobel Prize – such as Bob Lucas or Ed Prescott – or did equivalent pioneering work, whose careers would never have got off the ground in the current UK state-planned environment, simply because their ideas were unacceptable to prevailing opinion as represented by ‘top journals’. Research requires long horizons and a willingness to do things that are not currently fashionable. In economics as elsewhere the best researchers do not generally produce ‘top’ articles every year, as the REF requires; more like one or two a decade, together with a fair bit of minor commentary.

Unfortunately the exercise leads to degradation of the originality and long-term orientation of research. To get their four ‘top’ publications in some five years academics must constantly sacrifice quality for ability to please conservative editors with lookalike work. Ronald Coase, the great British economist at Chicago University, published just two great papers in his life for which he got the Nobel Prize; both charted quite new directions. Imagine how the REF would have treated him, and how administrators would have harried him!

As for impact, what on earth is the justification for this as a guide to research? If funders want ‘impact’ that is their privilege. But why should the government ask for ‘impact’ for research it no longer funds and indeed even for research it directly funds (since to demand impact is foolishly to direct what the findings should be)? Let academics decide on the trade-offs here; after all, it is not as if they too do not want their work to have maximum impact in the world at large. For them impact is a natural thing to seek since they do their research with social goals (such as improvement of the economy, the fighting of disease, the understanding of the universe) actually or subliminally in their minds. They also grasp that ‘impactful events’, such as media discussions, blogs and public debate, are all a way of promoting their ideas – advertising if you like.

Measuring ‘environment’ seems harmless enough until you ask why it is necessary. There are plenty of market-based measures of success in training PhD students – such as the sorts of jobs they get, and their completion rates – without some extra bureaucratic accolade.

Which brings one to the massive cost of this exercise. For years academics are displaced into ‘peer review’ committees for the exercise, on top of the government and university bureaucrats involved. They could all doing something worthwhile.[ii]

Once upon a time the REF was needed to shake up a government-dominated sector which had become ossified. Its job has been done. There is nothing to fear any more in letting UK universities compete in the global market place for academic output, for research as much as for students. There is much also to gain, in getting rid of layers of government and university bureaucracy, time-wasting, and administrative ignorance.

[i] In a recent IEA briefing Professors Len Shackleton and Philip Booth emphasised the way in which the REF distorts academic priorities and leads universities to ‘game’ the system.

[ii] According to the REF Accountability Revew the cost of this exercise was £250 million. The funds distributed by the REF were £1.5 billion in 2014.

Shadow Monetary Policy Committee

Patrick Minford has been the Professor of Economics at Cardiff Business School, Cardiff University from October 1997. Between 1967 and 1976 he held economic positions in: the Ministry of Finance, Malawi; Directors' staff, Courtaulds Limited; HM Treasury; HM Treasury's Delegation to Washington, DC; Manchester University; and the National Institute for Economic and Social Research. From 1976-1997, he was a Professor of economics at Liverpool University. Patrick Minford was also a member of the Monopolies and Mergers Commission between 1990 and 1996 and one of HM Treasury's Panel of Forecasters (the 'Six Wise Men') between January 1993 and December 1996. He was awarded a CBE for services to economics in 1996 and is the author of: books, articles and journalism on exchange rates, unemployment, housing and macroeconomics. Patrick Minford currently directs the Julian Hodge Institute of Applied Macroeconomics at Cardiff University Business School.

2 thoughts on “Why the government should abolish the Research Excellence Framework”

  1. Posted 03/08/2015 at 15:38 | Permalink

    I thank Prof Minford in raising this issue. I am a successful academic from REF perspective who has always been RAE or REF returned with a high GPA since the early 1990s. REF has done good for my academic promotion. But the fact of the matter is that REF has neither encouraged nor garnered research; it has confused research with replication. It has essentially killed deep thinking amongst academics at least in Social Sciences where I belong to. REF has also compromised the quality of teaching – REF has inflicted a huge cost on teaching quality. REF has turned universities into bureaucratic hub. It is high time that all stakeholders reconsider this issue. The REF 2020 rehearsal has already begun across UK universities. Above all REF is divisive.

    I recall a recent conversation where one university bureaucrat said: Even if you have won a Nobel prize in 2013 that is now useless because it now predates the REF 2020 census period. Isn’t this a wonderful outcome of REF?

  2. Posted 05/08/2015 at 00:23 | Permalink

    I hesitate to comment on Patrick Minford’s very thoughtful blog about the REF, since in my ‘academic career’ I have published only two refereed journal articles — which seems to be what the REF is demanding. The first (on how inflation accounting affected the accounts of nationalised industries) I only discovered afterwards had been refereed (though I never received any comments from the referees!); and the second was an article I was commissioned to write by a journal. I have preferred (a) to concentrate on teaching — which I see may be about to come back into fashion — and (b) to write books on controversial subjects —
    probably just the sort that (if they were articles) respectable’ journals might be unwilling to publish. I do agree with Patrick’s point about the long-term nature of some very worthwhile research (a la Coase) as compared with the essentially short-term nature of the REFs demands. I have witnessed the unfortunate effects of the REF at Cranfield (at least in the School of Management). I went into academic life so that I could think for myself, study whatever I liked and publish (or not) as I saw fit. I did not intend to be subject to government orders or bureaucratic hassle — which the REF provides in spades.

Comments are closed.