Transforming health and care for older people is complex. Evaluating such efforts requires a range of approaches, and a reflection on ways to improve how evaluations are commissioned, completed and used in a changing policy landscape. This report summarises workshops undertaken with evaluators, commissioners of evaluations and services, and those delivering services that are evaluated with the explicit aim of addressing these questions.

Cover: Better evaluations to support the needs of older people in the UK
Read Online

Better evaluations to support the needs of older people in the UK

by Tom Ling, Alex Sutherland, Jack Pollard

Download eBook for Free

FormatFile SizeNotes
PDF file 0.7 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Question

  1. How can evaluations of complex interventions, such as services for older people in the UK, be improved?

Transforming health and care for older people is complex and demanding. Evaluating such efforts requires a range of approaches and ideas, and a reflection on ways to improve how evaluations are commissioned, completed and used in a changing policy landscape. This report by researchers at RAND Europe summarises three workshops undertaken with evaluators, commissioners of evaluations and services, and those delivering services that are evaluated with the explicit aim of addressing these questions.

Key Findings

Relevance

  • Evaluations were too often not focused on major challenges and consequently findings were 'unsurprising' or even trivial.

Timeliness

  • Evaluation findings were often not available when decisions had to be made.

Replication and lack of cumulative building of knowledge

  • Evaluations were often designed as stand-alone pieces rather than building on previous evaluations and contributing to future evaluations.

Reluctance to share knowledge

  • Competition, a lack of clarity in communication, and poor knowledge management/mobilisation led to a reluctance to share information, pool data, or drew on the evaluations of potential competitors.

Efficiency

  • Evaluations often failed to use routine data and ignored the costs imposed on service providers and users.

Recommendations

  • For commissioners of evaluations: Commissioners from multiple organisations should collaborate routinely to identify key cross-cutting challenges, jointly fund evaluations addressing common concerns, engage the evaluation community earlier when developing an evaluation specification and ensure budgets and timelines match the scale of the question(s) asked.
  • For evaluators: Evaluators should engage earlier with commissioners and better balance competitive with collaborative behaviours to inform commissioners, develop standardised data sets, and make better use of routine data.
  • For service providers: Service providers can be a great source of tacit knowledge, but they often experience evaluations as something to be endured rather than as something to shape. Providers should work with evaluators to develop better routine monitoring and data collection.
  • For national policymakers in NGOs and government: Support the conditions under which the 'tragedy of the commons' can be avoided by: identifying and communicating priorities for evaluation research locally and nationally; monitoring the evaluation landscape; encourage and reward organisations for engaging with national priorities; and help nourish the 'commons' through sharing and collaborating while also ensuring adequate competition and variety.

Table of Contents

  • Chapter One

    What is the problem?

  • Chapter Two

    What should be done differently?

  • Chapter Three

    Conclusions and recommendations

  • Annex A

    Responses from the warm-up question at each workshop

Research conducted by

The research described in this report was funded by Age UK and conducted by RAND Europe.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.