After recruitment issues had led to the cancellation of a randomised controlled trial on the impact of incentivised pay on teaching in the UK, RAND Europe conducted a follow-up investigation exploring the reasons behind the recruitment difficulties. The study found that a combination of teachers being resistant to the principle of financial incentives and a lack of awareness about the trial were key reasons why recruitment had been unsuccessful.

Cover: 'Motivating teachers with incentivised pay and coaching' randomised control trial (ICR trial)
Read Online

'Motivating teachers with incentivised pay and coaching' randomised control trial (ICR trial)

Closure report: Understanding factors influencing participant recruitment failure

by Alex Sutherland, Rob Prideaux, Julie Belanger, Miriam Broeks, Yulia Shenderovich, Simon Van Der Staaij

Download eBook for Free

FormatFile SizeNotes
PDF file 0.6 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What are the factors that influenced the low recruitment of teachers in the study?
  2. Why was the recruitment of teachers to take part in the study unsuccessful?

While performance-based compensation systems in primary and secondary education are no longer a novelty in the US, they are still not yet common in the UK. Research from the US finds mixed results regarding the effects of performance-related pay on pupil achievement, but there is limited evidence around this concept from the UK.

Therefore, RAND Europe, in collaboration with the University of Cambridge, was commissioned by the Education Endowment Foundation to conduct a randomised controlled trial to explore the impact of incentivised pay and peer coaching in the teaching of mathematics. The incentive payment would have been in addition to normal teacher pay, with the final amount being dependent on the performance of pupils in the end of year tests. Coaching was included as part of the study on the basis that some teachers might be motivated by pay incentives, but might require support to improve their performance.

However, the study faced difficulty recruiting teachers for the study and due to this reason was subsequently cancelled. A follow-up investigation of the factors associated with the difficulties with recruitment then took place.

The follow-up study found that a combination of teachers being resistant to the principle of financial incentives and a lack of awareness about being able to participate in the trial were key reasons why the recruitment strategy had failed initially.

Key Findings

  • There was resistance to the principle of financial incentives. Teachers who opted out of the trial tended to be resistant to the principle of financial incentives, but these views were not universally held.
  • There were concerns about the perceived unfairness and impact of a financial incentive which could lead to some teachers 'gaming the system'. Some of the teachers and both headteachers expressed serious concerns about the perceived unfairness and impact of a financial incentive that, as a consequence of the research design, would have been available to some teachers but not others. Respondents expressed concern about teachers in receipt of the incentive payment 'gaming the system' by focusing more attention on children whose test performance would influence the incentive award.
  • Factors motivating teachers to participate in the trial included an interest in coaching, the attraction of receiving additional pay, and the contribution the trial would make to education research.
  • The communications strategy for the trial could have been refined, but this was not the primary reason for under-recruitment. Among those who made an active decision to participate or not, there was a good understanding of the trial, with teachers believing that they had enough information to make an informed decision. However, not all teachers were aware that they had the opportunity to participate and not all teachers had read the emails intended for them.

Recommendations

  • Recruitment strategies in schools should carefully consider how to secure headteacher or senior staff buy-in. This is crucial in setting the general mood for acceptability among teachers in a school.
  • Developers and evaluators should consider placing greater emphasis on the involvement of school practitioners in designing trials and the way they are communicated.
  • The widening of participation in trials, where possible to include more than one region or academy trust, should be considered.
  • For research projects as complicated as this trial, it is critically important for funders, developers and evaluators to meet iteratively to discuss and resolve questions.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Methodological approach

  • Chapter Three

    Findings from survey, document review and interviews

  • Chapter Four

    Conclusion and recommendations

Research conducted by

The research described in this report was funded by the Education Endowment Foundation (EEF) and conducted by RAND Europe.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.