In recent years there has been a boost to the United Kingdom's earth observation (EO) market, driven by advancements in technology, increased public investment, and a critical mass of EO-focused companies. Satellite-based EO technologies play a critical role across the world, from monitoring climate change, to assessing the situation during crises such as extreme weather events. As of 7 September, the United Kingdom has re-entered the European Union (EU) Copernicus EO Programme, a significant win for the UK EO community.
Important questions remain linked to government-funded space technology programmes, namely in relation to public interest and the value of such initiatives to Britain's strategic aims and overall prosperity. Evaluation studies help understand the contribution of such programmes to the public good. RAND Europe's ongoing evaluation of the Centre for EO Instrumentation programme for the UK Space Agency considers good practice for determining the public value of EO and other space technologies.
The RAND Europe research team conducted a literature review exploring approaches for evaluating publicly funded EO research and development programmes. Findings indicate that UK EO programmes were generally reviewed positively as having delivered on their objectives, notably in terms of value for money through economic assessment. In addition to analysing assessments of both specific funding programmes and memberships to collaborative EO infrastructures, the team is also analysing programmes to explore the links with advancement of EO technology and innovation, the capabilities and skills of the workforce, sector growth, and end-user benefits.
Satellite-based EO technologies play a critical role across the world, from monitoring climate change, to assessing the situation during crises such as extreme weather events.
Share on TwitterAs a result, we have so far identified a list of key good practices and challenges for EO research and development evaluation.
Good Practices for Demonstrating Value in the UK EO Market
- Development of EO technology and innovation remains a common objective for many programmes and is hence central to the evaluation. Some useful indicators for this theme include progression of technology readiness level, number of projects awarded, de-risking of technology for operability or commercialisation, intellectual property, and success rate in joining international missions.
- Developing capabilities and fostering EO sector growth is a common objective of publicly funded programmes. Capability and skills tend to be measured by outputs such as new and/or improved products and services, number of publications, and talent support. EO sector growth can be measured by tracking involvement of small-and-medium-sized enterprises, number of collaborations, follow-on sales, access to new markets, and job creation.
- End-user benefits and the wider social and environmental impact are useful for programme leaders to understand the downstream effects of EO programmes. Common measuring metrics for end-user benefits include number of potential new users and the involvement of end-users in the programme design. For the wider impact, some indicators include global influence of the United Kingdom in the EO sector, support of climate-change moderation, better data access for non-space sectors, and gender equality in funded projects.
Challenges in Measuring the Value of Publicly Funded Space Research and Development in the United Kingdom
- Challenges due to time lags between research and development, and realisation of impact: While a scientific impact is expected in many EO programmes, it has yet to materialise until missions launch and begin observing. In some cases, projects may still not be underway when the evaluation is conducted. Additionally, general challenges in measuring the space economy also apply to evaluating space research and development funding. Time lags will affect how long it takes to realise products from initial space research and development. Unless repeat studies with the same indicators are conducted, there are risks of inflating results and challenges to measure benefits chronologically. Further, attributing an organisation's success to a single funding programme has particular limitations in the space sector, as firms often need support from several sources to commercialise their technologies.
- Attribution of and contribution to space sector impacts: Although not explicitly mentioned, this theme emerged in relation to proving additionality in interventions compared to other surrounding influences, such as comparator interventions, the broader research and development finance climate, and capability factors such as the buoyancy of the job market.
- The lack of monitoring data and programme-indicator targets for space research and development programmes: For indicators with no targets (PDF), the programme's performance would need to be evaluated on a case-by-case basis, identifying mitigation actions taken or evolution over time. Further, examples such as the evaluation of UK membership in the European Southern Observatory (PDF) revealed gaps in the evidence base, recommending additional periodical data collection rounds, bibliometric analyses, and more precise cataloguing of improvements to remedy this.
Assessing the value of publicly funded EO programmes in the United Kingdom and internationally remains a challenging but essential task. By adopting best practices in evaluation and focusing on the uniqueness of the UK EO market, we can better understand the return on investment and communicate the societal and scientific benefits of such programmes. As the United Kingdom continues to invest in space technology and participate in international agreements such as Copernicus, effectively demonstrating the value of these investments remains a top priority.
Katie O'Brien and Theodora Ogden are analysts, and Michelle Qu is a research assistant at RAND Europe. Billy Bryan is RAND Europe's evaluation and research leader.
