Scrap the REF
24 July 2015
SUGGESTED
Government and Institutions
Education
Lifestyle Economics
15 November 2024
British universities are tightly regulated. But, as often happens with regulation, bureaucratic procedures do not guarantee that things work well. They do, however, generate vested interests, tend to produce distorted outcomes, and become so much part of the furniture that quite sensible people can’t imagine life without them.
The allocation of government research funding is a case in point. There is a separate issue about how much government funding should go to research, but what concerns me here is how this funding (which largely comes via the Department for Business Innovation and Skills) is allocated.
The UK has a ‘dual funding’ system. The larger part (around £2.6 billion this year) is allocated through the various Research Councils to support specific projects which have been through a rigorous competitive process. But somewhere between £1 billion and £1.5 billion is awarded directly to universities as a result of something called the Research Excellence Framework (REF). This is the latest title for a series of exercises which began in the 1980s. Every few years, the research work done by universities is peer-assessed, and funds are awarded by the various Higher Education Funding Councils (HEFCE in England) on the basis of REF rankings.
When assessment exercises of this kind were first done, they provided useful information about what universities were doing with what was then entirely public money. It also acted as a spur to institutions which had become lazy. However times have changed.
It is now abundantly clear that most high-quality research is concentrated in a limited number of leading universities, which receive the bulk of government funding. This is apparent without the need for a costly and lengthy procedure. REF 2014 took a year and cost at least £250 million. It involved reviewing the work of over 50,000 academics, with hundreds of panel members reading and evaluating almost 200,000 research ‘outputs’, yet a roughly similar allocation could have been reached by using publicly available metrics such as citations, research grant incomes, PhD completions and so on. The REF-based funding that goes to universities outside the Russell Group of top universities represents a trivial proportion of their income.
But it’s not simply that the cost of the REF is excessive. It also distorts the allocation of resources towards the particular type of research (short articles in a handful of key academic journals) encouraged by the assessment process, at the expense of more fundamental work. It leads to game playing, where institutions submit for consideration only a small proportion of their staff, and temporarily import research ‘stars’ with no real connection to the institution, to boost rankings for publicity purposes rather than funding per se. Such stratagems may seriously mislead students, overseas partners and the general public.
More fundamentally, it has a detrimental effect on teaching, with many researchers doing little or no teaching despite tuition fees paying their wages, while students are fobbed off with doctoral students or part-time visiting lecturers as their main academic contacts. It has led to a damaging obsession with ‘planning for the REF’ at the expense of more sensible academic priorities. It has also fuelled lecturer fantasies about the importance and significance of sometimes banal research in the humanities and social sciences. The self-interest and self-obsession of academic assessment panels has led to implausible ‘grade inflation’ with the proportion of ‘world-leading’ (the REF term, not mine) research outputs rising from 14 per cent in 2008 to 30 per cent in 2014.
The REF no longer serves a useful purpose and should not be repeated. We should gradually move away from institution-based funding towards project-based funding via the Research Councils. Any remaining need to allocate research support to institutions, for instance for capital purposes, should rely on available metrics rather than the REF.
Len Shackleton is co-author, with Philip Booth, of Abolishing the higher education Research Excellence Framework (the REF).
The allocation of government research funding is a case in point. There is a separate issue about how much government funding should go to research, but what concerns me here is how this funding (which largely comes via the Department for Business Innovation and Skills) is allocated.
The UK has a ‘dual funding’ system. The larger part (around £2.6 billion this year) is allocated through the various Research Councils to support specific projects which have been through a rigorous competitive process. But somewhere between £1 billion and £1.5 billion is awarded directly to universities as a result of something called the Research Excellence Framework (REF). This is the latest title for a series of exercises which began in the 1980s. Every few years, the research work done by universities is peer-assessed, and funds are awarded by the various Higher Education Funding Councils (HEFCE in England) on the basis of REF rankings.
When assessment exercises of this kind were first done, they provided useful information about what universities were doing with what was then entirely public money. It also acted as a spur to institutions which had become lazy. However times have changed.
It is now abundantly clear that most high-quality research is concentrated in a limited number of leading universities, which receive the bulk of government funding. This is apparent without the need for a costly and lengthy procedure. REF 2014 took a year and cost at least £250 million. It involved reviewing the work of over 50,000 academics, with hundreds of panel members reading and evaluating almost 200,000 research ‘outputs’, yet a roughly similar allocation could have been reached by using publicly available metrics such as citations, research grant incomes, PhD completions and so on. The REF-based funding that goes to universities outside the Russell Group of top universities represents a trivial proportion of their income.
But it’s not simply that the cost of the REF is excessive. It also distorts the allocation of resources towards the particular type of research (short articles in a handful of key academic journals) encouraged by the assessment process, at the expense of more fundamental work. It leads to game playing, where institutions submit for consideration only a small proportion of their staff, and temporarily import research ‘stars’ with no real connection to the institution, to boost rankings for publicity purposes rather than funding per se. Such stratagems may seriously mislead students, overseas partners and the general public.
More fundamentally, it has a detrimental effect on teaching, with many researchers doing little or no teaching despite tuition fees paying their wages, while students are fobbed off with doctoral students or part-time visiting lecturers as their main academic contacts. It has led to a damaging obsession with ‘planning for the REF’ at the expense of more sensible academic priorities. It has also fuelled lecturer fantasies about the importance and significance of sometimes banal research in the humanities and social sciences. The self-interest and self-obsession of academic assessment panels has led to implausible ‘grade inflation’ with the proportion of ‘world-leading’ (the REF term, not mine) research outputs rising from 14 per cent in 2008 to 30 per cent in 2014.
The REF no longer serves a useful purpose and should not be repeated. We should gradually move away from institution-based funding towards project-based funding via the Research Councils. Any remaining need to allocate research support to institutions, for instance for capital purposes, should rely on available metrics rather than the REF.
Len Shackleton is co-author, with Philip Booth, of Abolishing the higher education Research Excellence Framework (the REF).
2 thoughts on “Scrap the REF”
Comments are closed.
SHARE
There is so much more that’s wrong with the way universities are funded. I am more concerned by the £9000pa fees paid by undergraduates. How are these spent? Are they value for money? Does it really cost £1m to run a course for 111 students each year? A cartel has effectively been created with few universities charging less than £9k. Hardly a free market, and not a transparent one either, as we have no idea how the money is allocated. My guess is that much of the money cross subsidises other university activities which don’t have to account for themselves like businesses might.
I could not agree more with this report! As an academic I have seen first hand the distortions of behaviour that the REF has produced and the “gaming” of the system by universities.
The whole system is flawed and should be scrapped immediately. And I can assure readers that over 90 of my colleague agree!