Liz Morrish, December 12th
2014
This blog is a reflection on a
panel discussion on The REF and the State
of Higher Education Today. The panel was convened and supported by SAGE
publishers to celebrate the launch of their new series SAGE Swifts, ‘a new short
series of polemical texts’. The first two books in the series are Professor
Derek Sayer’s (University of Lancaster) Rank
Hypocrisies: The Insult of the REF, and Professor Thomas Docherty’s (University
of Warwick) The University at War.
The battleground, as was
immediately evident, is the REF and concomitant manifestations of managerialism
and audit culture in UK universities. Accompanying Thomas Docherty and Derek
Sayer on the panel were David Sweeney, Director of Research and Innovation at
HEFCE – and overseer of the REF process, and The Right Honourable David
Willetts MP, former Minister for Higher Education under the Coalition
government.
I blog this encounter, firstly
because it occurred just ten days before the release of the REF results on 18th
December 2014, and secondly because it was a unique occasion; I am not aware of
any other opportunity for the architects of the REF to debate with those who experience
its effects. It is very clear that support for the REF is waning among those
who supply the ordnance for their institution’s assault. This week’s Times
Higher (11th December 2014) contains a leader, a lead article, and
several other commentaries - all critical of the current methodology of the
REF. The REF is part of the culture of UK universities,
and it is time for a sober reflection on the cost –financial, and also
intellectual and collegial – of REF 2014.
The strictures of the REF (Research
Excellence Framework) 2014 are well known to UK academic readers, but may not
be widely appreciated further afield. Preparing a research unit for entry into
the REF is rather like trying to make your way around London with a map of
Paris. The assessment cycle takes place every 5-7 years, and yet priorities,
expectations and reporting mechanisms have changed with each iteration. The
2014 exercise has required all HEFCE-funded research units to submit for
evaluation four ‘best’ pieces of research for each research-active academic in
post on October 31st 2013. These can be monographs, refereed journal
articles, book chapters, etc. Pieces are given a ranking by what is claimed to
be a process of ‘expert peer review’. Each of these research ‘outputs’ is
graded 1-4 according to a judgement of: nationally recognised, internationally
recognised, internationally excellent, or world-leading quality. Two other
measures contribute to the unit/department’s overall score. The research
‘environment’ is evaluated according to whether it is conducive to producing quality
research of in terms of its vitality and sustainability. The other measure is
‘impact’, and for each ten scholars entered, there must be at least one case
study which details how their work has had impact – either economic or social. Paradoxically,
the sort of scholarly influence desired by most academics carries no weight.
David Willetts and David Sweeney
laid out their contention that the REF was a necessary part of universities’
responsibility to account to society for their performance. David Willetts
assured the panel that the growth of the REF, and its current complex methodology,
have been driven by academics in order to make sure that talent was not
overlooked by the research establishment. He was keen to stress that if there
was a cheaper way of organising the audit, his successors in BIS would be
willing to consider it, and that it might be time to refocus on collaboration,
rather than competition. How he will wean Vice Chancellors and Universities UK
away from their preoccupation with rankings and competition, he did not reveal.
David Sweeney of HEFCE cautioned the
audience, largely comprised of practising academics, that universities cannot
be the closed institutions they were fifty years ago, and that they must be
open to society. This did not seem a controversial proposition. Sweeney did, though, enter into the polemical spirit of the SAGE Swifts series by taunting academics
with the suggestion that it ill behove us to condemn a process we have taken
part in. Perhaps Dr Sweeney imagines the REF to be voluntary, with some of us able
to navigate our careers outside of its dominion, but his suggestion seemed
rather like deflecting complaints from travellers queuing at Heathrow
immigration by saying ‘well, you stood and waited , didn’t you’?
Derek Sayer, who lobbied to have
his work excluded from his department’s REF return, took issue with the subjectivity
and fallibility of the assessment methods, not with the necessity and
desirability for assessing research. He has bolstered his critique of the REF
methods and structures with a hugely impressive grasp of REFonomics. The
government declares that the £60 million spend is good value for money, a
figure disputed by Sayer who estimates the cost to be nearer £200 million when
we factor in ‘opportunity costs’ such as time spent by university departments
gaming the REF, and new appointments such as departmental research directors
which were previously unnecessary.
Sayer went on to argue that the REF
also fails in its claim to offer international benchmarking of UK research,
since so few of the judging panellists are from overseas. Furthermore, the
reduction in the number of panels in 2014 has meant that members of broad
subject-themed panels are attempting to make judgements of significance,
originality and rigour on research which may not even lie within their primary
field of expertise. The lack of sufficient breadth and depth on panels means
that the exercise amounts to nothing more than skim reading masquerading as rigorous
judgement, and the resulting subjectivity certainly does not match the rigour
of the peer review demanded for publication.
Also in Sayer’s gunsights was the
claim of the REF to encourage ‘selectivity’, and he identified this as
particularly destructive of collaborative and collegial relations between
researchers, as much worthy research has been excluded from submissions. In
some departments, these REFugees have been placed under various degrees of ‘performance
management’, even though they may have produced quantities of excellent
research, as endorsed by international peer review. In other departments, harmony
between colleagues has been replaced with antagonism as REFable scholars promise
to scale the institutional hierarchy at the expense of those rejected.
Thomas Docherty reiterated
that universities were not contesting the necessity of the REF. He asserted
that universities have a duty to taxpayers and to students, but what is
important is the way they fulfil that. Academics need to be aware of the history
of universities, and their wider duty as stewards of the past, the present and
the future. In monetizing those functions, by the charging of tuition fees,
Docherty argues that the current regime breaks the generational bond. And
further down the production line for the ‘global citizens’ which universities
are claiming to produce for a 'fast changing' world, there lies the outcomes of
growing inequality, which means these citizens will ‘get shafted’. Docherty
argues that academics are not just servants of this state of affairs - we
should instead be shaping it. The problem that Docherty identifies with the REF
is that it drives academics towards conformity and safe research topics –
delivering static ‘outputs’ rooted in a present disconnected from its past, which
makes countering those forces less likely. At its best, the REF distorts
research agendas and priorities. However, a graver hazard is that it will not be
research selectivity and competition which is delivered, but a new selective
and competitive academic will be formed, whose research trajectory is entirely
determined by a regime peripheral to their own intellectual curiosity and
academic judgement. Academics take a huge step in making themselves vulnerable
when they submit work for publication. This is on the understanding that it
will receive the full attention of a qualified expert in the particular
specialism. They do not benefit from having this rigour superseded by some other,
subjective and possibly under-informed, evaluation, with career determining consequences.
In the discussion of the panel
papers, several questioners mentioned that the REF has created a culture of
anxiety (see this blog for a compilation of sources on the effects of university-as-anxiety-machine).
One distinguished professor, admitted to being ‘terrified’, echoing Derek
Sayer’s statement, “I think that fear is integral to the way that British
universities work and I think the REF is the key instrument of that.” http://www.timeshighereducation.co.uk/news/universities-need-more-collaboration-less-competition/2017464.article The prevailing view was that the REF
has become a disciplinary tool which has left academics exposed as the targets
of potential institutional bullying. If it is a tool of accountability, we do not lack mechanisms for monitoring academics: NSS, QAA,
module evaluations, teaching observations, internal audits, and even hourly
‘space utilisation’ surveys. The
danger is that such instruments will exceed their original objectives
and become weaponized in the hands of what Docherty calls ‘managerial
fundamentalists’.
We are in danger if we persist in allowing ourselves to be
subject to this discipline, in the face of critiques like Docherty’s and
Sayer’s – in the face of inconsistency and falsification – of moving from a
culture of anxiety into a culture of psychosis. Sayer quoted Sir Peter
Swinnerton-Dyer, former Chair of the University Grants Committee (later
Universities Funding Council), as saying that the rot set in when Vice Chancellors
saw the REF primarily as a tool of
discipline and reputation building, not for research funding. This view of the utility of the RAE was echoed,
in the Times Higher in 2007, by David Eastwood, now Vice Chancellor of the
University of Birmingham, but written when he was Chief Executive of HEFCE, “The
RAE has also been the key instrument for performance management in institutions”.
Given the flaws and dangers
outlined in the papers and the discussion, what surprised me was that there
remained an appetite for some kind of REF research ranking and accountability
exercise. From what I heard from all the
speakers and comments around the room, and from what I read in a selection of
blogs, it seems that there is support for a research quality audit based on metrics
(citations, journal impact factors, h-index etc.). To me, this reinforces the norms
of academic capitalism, and those canons of the neoliberal academy:
quantification, finacialization, competition, the creation of crisis, and the
insistence that universities and individuals display ‘productivity’, and that what
they research must be ‘improved’, ‘reformed’ and ‘held to account’.
In my opinion, we should just
accept that measuring research and measuring ‘excellence’, are like trying to apprehend
a mirage. It will always evade us. Some things are just not measurable, or not
measurable enough to satisfy the demands of rationality, equity and
tolerability. It would be better ‘value for money’ if QR funding was
distributed partly as developmental ‘seed’ money, and partly in response to
bids from individuals and groups. To precis the arguments above – the REF is a
colossal waste of money, and does not deliver on its claims.
But I am not optimistic. To use an
analogy favoured by economist Paul Krugman, these are zombie ideas, which sheer
logic cannot kill – they just keep shambling forward. No, indeed, it will take
more than just disagreement to end REFonomics, especially when it cements the
authority of those who rather like academics to be controlled.
