RAND Europe, together with the University of Cardiff and the University of Sheffield, conducted a real-time evaluation of the Research Excellence Framework. The study assessed attitudes, perspectives and behaviours towards REF 2021 across the sector; and explored how REF policies and changes are embedded in the way submissions are prepared and delivered. The study aims to provide clarity and perspective to inform future rounds of the REF.

Cover: Understanding perceptions of the Research Excellence Framework among UK researchers
Read Online

Understanding perceptions of the Research Excellence Framework among UK researchers

The Real-Time REF Review

by Catriona Manville, Camilla d'Angelo, Andreas Culora, Emily Ryen Gloinson, Cagla Stevenson, Netta Weinstein, James Wilsdon, Geoff Haddock, Susan Guthrie

Download eBook for Free

FormatFile SizeNotes
PDF file 1.4 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What are the attitudes, perspectives and behaviours towards REF 2021 across the sector?
  2. How are REF policies and changes embedded in the way submissions are prepared and delivered?
  3. How have views towards the REF changed over time?

The Research Excellence Framework (REF) is an evaluation system for assessing the quality of research in UK higher education institutions (HEIs). Despite its importance in shaping research cultures, there is little systematic evidence about perceptions and attitudes towards the REF across the sector, and which aspects are viewed favourably or unfavourably. Research England, on behalf of the four UK higher education funding bodies, commissioned RAND Europe, together with the University of Cardiff and the University of Sheffield, to conduct a real-time evaluation of attitudes and perspectives of the sector towards REF 2021. The specific objectives of this study are to: (1) assess attitudes, perspectives and behaviours towards REF 2021 across the sector; (2) explore how REF policies and changes are embedded in the way submissions are prepared and delivered; and (3) expand and build on the findings of a pilot study conducted in 2018. By conducting the review in real time, alongside preparations for the exercise, the study aims to help accelerate the process of learning and revision post-REF 2021 and provide more clarity and perspective on the future of the REF to institutions at an early stage for the next round of assessment. The study also aims to provide an early contribution to the evidence base in anticipation of a potential reform of the REF that was requested by the UK government.

Key Findings

  • The majority of views on the influence of the REF on academic research and the research community are mixed.
  • The majority of researchers stated that, at an individual level, the REF has not influenced their own research.
  • The majority of academics perceive that changes to the rules for REF 2021 are positive for them.
  • Early career researchers feel, more than more-established researchers, that changes to the REF affect the expectations, both positive and negative, placed on them.
  • There are different approaches to preparing for the REF at an institution level, and some are seen as helpful and others as harmful. Helpful approaches were highlighted as being carried out more commonly.
  • Researchers have varied views on how positive and negative the beneficial and detrimental qualities of the REF are for individual researchers.
  • The majority of researchers perceive that, overall, the REF has a negative influence on UK researchers, however some academics and institutional leaders raised a number of positive aspects of the REF.
  • There is a lack of clarity on the purpose of the REF, and there are a range of views on what the purpose of the REF should be.
  • There are divergent views on what the future of the REF should be.

Recommendations

  • It would be important to set out clearly the purpose and desired outcomes of the REF (and align the approach of the exercise to that), as a better understanding of this purpose might help mitigate the perceptions of burden.
  • In future rounds [of the REF], it would be important to act quickly and provide clarity to the sector as early as possible, to make the rules as simple as possible, and ensure rules and changes are communicated clearly.
  • There is a need for further research and consultation with the sector on the ways to take the exercise forward — for example, looking at international comparisons to learn from other systems to allocate funding and conduct national assessments, as well as considering the options and associated trade-offs of the alterations listed above.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    The influence of the REF on researchers and the research community

  • Chapter Three

    Changes to the REF for 2021 and preparations for assessment

  • Chapter Four

    General attitudes towards the REF

  • Chapter Five

    Looking to the future of the REF

  • Chapter Six

    Concluding thoughts

  • Annex A

    Overview of methodological approach

  • Annex B

    National survey questionnaire

  • Annex C

    Longitudinal survey questionnaire

  • Annex D

    Interview protocol: Institutional leadership

  • Annex E

    Focus group protocol: Research Managers

  • Annex F

    Focus group protocol: Academics

  • Annex G

    NVivo code book for analysing qualitative research

  • Annex H

    List of recommendations from the Stern Review

Research conducted by

The research described in this report was commissioned by Research England and conducted by RAND Europe.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.