Developing tools to assess evidence for policymakers and practitioners
Hero Images/GettyImages
At the request of the College of Policing and the Education Endowment Foundation (EEF), researchers reviewed existing challenges in the assessment of social policy evidence and outlined the emerging best practices in the field.
After reviewing and piloting a number of existing tools and approaches to appraising evidence, the team recommended three tools and provided guidance on how to apply them.
Background
Using evidence to inform policymaking and practice is incredibly important and there has been growing emphasis on critical appraisal of research in social policy. Recognising this, organisations are developing processes and tools to assess research evidence and communicate its findings and quality to policymakers and practitioners.
Goals
RAND Europe was commissioned by the College of Policing and the Education Endowment Foundation (EEF) to advise on their systems for appraising evidence in the context of preparing recommendations for practice in policing and education. The research aimed to provide recommendations on how to assess evidence from individual studies and groups of studies and how to make evidence-informed recommendations for practitioners and policymakers.
Methodology
To inform the work of the College of Policing and the EEF on preparing guidelines for practitioners, RAND Europe researchers reviewed and piloted a number of existing tools and approaches to appraising evidence, focusing primarily on tools for experimental and quasi-experimental programme evaluations. Out of the many study assessment tools available, the focus of the project was on reviewing and piloting the most commonly used and well-established approaches, in order to select tools to be recommended to the College of Policing and the EEF.
Having selected the recommended approaches, we assessed two sample sets of evaluations (one in education and one in policing), first appraising each study and then the overall bodies of evidence. Based on discussions with the teams at the College of Policing and the EEF, we then provided guidance on applying these approaches to the work of these two organisations.
Findings
The project reviewed existing challenges in the assessment of social policy evidence, such as translating tools from medicine to social policy. It also outlined the emerging best practices in the field, such as providing reviewers with detailed guidance and training in the recommended tools. The key tools and approaches recommended as a result of the project were:
Revised Tool for Risk of Bias in randomized trials (RoB 2.0)
Risk Of Bias in Non-randomized Studies - of Interventions (ROBINS-I) tool
Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the GRADE Evidence to Decision frameworks