In the UK, research outputs from universities are assessed every five years to determine future funding allocations from government. In 2014, for the first time, the Research Excellence Framework (REF) included an assessment of research impact. This component was worth 20 per cent of the score awarded to each institution. The assessment was performed by panels of academics and research users.
The importance of research user engagement throughout the process of REF 2014 cannot be overstated. Research users, or those benefitting from publicly funded university research, played key roles throughout REF 2014 in several ways. Their evidence was needed to substantiate academics' claims about the wider impact of university research — conveyed through impact case studies and strategies. Secondly, representatives from beneficiary organisations, like the British Library, the Overseas Development Institute, the BBC, Royal Museums Greenwich, Oxfam, BT, BAE Systems and the Bank of England served on the panels that assessed the impact of university research. So their role in engaging with REF 2014 has been vital to its success and important to the future of the process.
At the request of the UK's higher education funding bodies, RAND Europe conducted two evaluations related to the impact element of REF 2014. The first evaluation reviewed universities' preparation of submissions describing impacts — in particular we examined their case studies and impact strategies. The research users we interviewed as part of this evaluation felt that the two hours on average it took them to submit testimonials on the benefit of a specific piece of research to their organisation was not overly burdensome.
Moreover, research users felt that it was important to contribute to this exercise — recognising the value of university research and the need to describe the impact on wider society in order to ensure further funding. They also saw benefits in engaging in the assessment process in terms of creating new relationships and networks.
The second evaluation undertaken by RAND Europe reviewed the process of assessing impact case studies and templates. REF required all impact case studies to be reviewed by at least one academic and one research user. Twenty-seven per cent of those assessing impact in REF 2014 were research users. Academics and research users alike emphasised the importance of the combination of both of these perspectives in assessing impact beyond academia. Research users were seen to be particularly important in qualifying the value of some impact statements; for example web traffic statistics, audience appreciation and testimonials from specific geographical contexts.
The balancing of perspectives was highlighted in particular during the calibration process that panellists undertook as a sub-panel before starting the scoring. The involvement of both parties in the process allowed everyone to understand others' interpretations of impact and to develop a shared understanding.
The tension now is around how to maintain an acceptable level of engagement with research users in the process — in future REF exercises it is possible that the submission process will take up even less time for research users, because evidence will be collected on an ongoing basis. However, sustained engagement will still be required in the assessment phase. To support this, more needs to be done by the UK's higher education funding bodies to raise awareness of the process and its purpose outside of the academic sector, and to ensure support from those benefiting from academic research.
Catriona Manville is a senior analyst at RAND Europe in Cambridge. @CMManville
This article was originally posted on PolicyWonkers, a blog hosted and run by the Policy Institute at King's, on March 31, 2015.
