Initial evaluation of the impact of the Transformation Support Unit of Eastern Academic Health Science Network. We developed a logic model, surveyed stakeholders and analysed 4 case studies of innovations supported. Signs are that the Unit is improving implementation of innovations in the National Health Service in eastern England. We recommended collection of more data to enable estimation of the return on investment.

Read Online

Evaluating the impact of the Eastern Academic Health Science Network Transformation Support Unit

by Miaoqing Yang, Jack Pollard, Bryn Garrod, Sonja Marjanovic, Jon Sussex

Full Document

FormatFile SizeNotes
PDF file 0.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Question

  1. What has been the impact so far of Eastern Academic Health Science Network's Transformation Support Unit?

The Eastern Academic Health Science Network (Eastern AHSN) Transformation Support Unit (TSU) started work in November 2016 with the objective 'to find and help implement innovative solutions for the critical challenges that health and care providers face'. The Eastern AHSN commissioned RAND Europe in October 2017 to undertake a rapid initial evaluation of the impact of the TSU. The assessment of impact is necessarily preliminary and our review focused on identifying early signs that could indicate future potential for impact, as much as examining existing achievements. The main elements of the study were: a workshop with TSU staff and key stakeholders; an online survey of stakeholder perceptions; case studies of four of the projects supported by the TSU to date; a review of TSU and Eastern AHSN materials and discussions with TSU staff.

Most respondents believe that the TSU has a positive impact on the outcomes of their projects, helping them to reach more people and speed up implementation of the innovation. TSU support for pilots, implementation and network building are considered to be particularly useful. There is room for improvement in terms of helping to identify scale-up opportunities and requirements, and creating an evidence base on impact to support shared learning and spread of good practice. Given its early stage, it is understandable that the TSU has not yet had much impact on NHS cost savings and the wider economy, and most respondents believe that these positive effects will come in the near future.

Key Findings

  • Most survey respondents believe that the TSU has a positive impact on the outcomes of their projects, especially in helping them to reach more people and speeding up implementation of the innovation. Support for pilots, implementation and network building are particularly useful.
  • There is room for improvement in terms of helping to identify scale-up opportunities and requirements, and creating an evidence base on impact to support shared learning and spread of good practice.
  • Given its early stage, it is understandable that the TSU has not yet had much impact on NHS cost savings and the wider economy, and most respondents believe that these positive effects will come in the near future.
  • From the four case studies we undertook, we do not have strong evidence to indicate the extent to which the TSU's inputs are necessary to achieve the potential beneficial impacts of supported interventions, or to achieve them to a greater extent or sooner than would otherwise have been the case. Such evidence should in principle be drawn from experimental or quasi-experimental studies with appropriate comparison groups that tell what would happen without support of the TSU. A weaker but more pragmatic approach would be future surveys of stakeholders to obtain their views on the additional impact (if any) brought about by the TSU's activities.
  • The TSU appears to be contributing to worthwhile innovations but it is not possible to know with certainty how much of the net benefits of the innovations can be attributed to the TSU's role.

Recommendations

  • We recommend that the TSU considers which types of evidence can be collected for all of the projects it supports on a case-by-case basis (considering criteria such as relevance, feasibility and resource implications for data collection) and proactively embeds this into project delivery from the outset of each project.
  • It will also be desirable to obtain from the partner organisations participating in each project the extent to which they incur additional costs in implementing it (e.g. set-up and training costs); this information is generally lacking from the evidence we have seen so far.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    The intervention logic for the TSU

  • Chapter Three

    Survey of stakeholders' perceptions

  • Chapter Four

    Case studies

  • Chapter Five

    Reflecting on progress and implications for the future

Research conducted by

The research described in this report was prepared for the Eastern Academic Health Science Network (Eastern AHSN) and conducted by RAND Europe.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.