Better Evaluations to Support the Needs of Older People in the UK

Transforming health and care for older people is complex and demanding. Evaluating such efforts requires a range of approaches and ideas, and a reflection on ways to improve how evaluations are commissioned, completed and used in a changing policy landscape. This study by researchers at RAND Europe summarises three workshops undertaken with evaluators, commissioners of evaluations and services, and those delivering services that are evaluated with the explicit aim of addressing these questions. It was broadly agreed that the areas to address were:

• Relevance: evaluations were too often not focused on major challenges and consequently findings were “unsurprising” or even trivial.
• Timeliness: evaluation findings were often not available when decisions had to be made.
• Replication and lack of cumulative building of knowledge: evaluations were often designed as stand-alone pieces rather than building on previous evaluations and contributing to future evaluations.
• Reluctance to share knowledge: competition, a lack of clarity in communication, and poor knowledge management/mobilisation led to a reluctance to share information, pool data, or drew on the evaluations of potential competitors.
• Efficiency: evaluations often failed to use routine data and ignored the costs imposed on service providers and users.

Following from this, the workshops generated key recommendations:

For commissioners of evaluations: Commissioners from multiple organisations should collaborate routinely to identify key cross-cutting challenges, jointly fund evaluations addressing common concerns, engage the evaluation community earlier when developing an evaluation specification and ensure budgets and timelines match the scale of the question(s) asked.

For evaluators: Evaluators should engage earlier with commissioners and better balance competitive with collaborative behaviours to inform commissioners, develop standardised data sets, and make better use of routine data.

For service providers: Service providers can be a great source of tacit knowledge, but they often experience evaluations as something to be endured rather than as something to shape. Providers should work with evaluators to develop better routine monitoring and data collection.

For national policymakers in NGOs and government: Support the conditions under which the “tragedy of the commons” can be avoided by: identifying and communicating priorities for evaluation research locally and nationally; monitoring the evaluation landscape; encourage and reward organisations for engaging with national priorities; and help nourish the “commons” through sharing and collaborating while also ensuring adequate competition and variety.