Investigating the Relationship Between R&D Funding and Performance

Coins and calculator on a British newspaper

Photo by Pixelbliss/Fotolia

To understand more about the implications of long-term funding restrictions, as well as the potential benefits from increasing research spending, researchers examined the relationship between the level of research spending and research performance, and how changes in spending relate to changes in performance.

The team found that there is a statistically significant relationship between changes in spending and changes in performance, but it takes around two to four years for changes in research spending to be reflected in research performance. The team also analysed ‘nonlinearities' in the relationship between funding and performance, particularly when a small decrease in funding results in a significant decrease in some or all measures of performance, and when a small increase in funding improves performance significantly.

Background

The UK is often said to ‘punch above its weight’ in terms of research excellence, but with UK research spending only around 1.7% of GDP (compared to around 2.8% in the US and Germany) could the UK’s future research base be under threat?

The current performance indicators show the UK has a strong level of research performance and demonstrate that the UK research base had some resilience to be able to survive short-term constraints in funding. However there is a risk that long-term restrictions in funding may lead to erosion of some of the research base capacity that will transfer in to diminished performance after a certain time lag.

Goals

To understand more about the implications of these long-term funding restrictions, as well as the potential benefits from increasing research spending, RAND Europe was commissioned by the Department of Business, Innovation and Skills to examine — and thus to improve the government's understanding about — the relationship between the level of research spending and research performance, and particularly how changes in research spending relate to changes in performance.

The goal of this project was to deepen the understanding of the relationship between research inputs and research performance both at national and sub-national level. A key focus of the project was on the impact or potential impact of long-term erosion of funding which creates pressure points that could lead to adverse research performance.

Findings

The team found that there is a statistically significant relationship between changes in spending and changes in performance, when country factors are taken into account. This implies that changes in spending are correlated with changes in performance, meaning that a substantial increase or decrease in spending will have an effect on performance as measured in the number of publications produced.

The average time-lag in this relationship is likely to be between two and four years. This means that it will take around two to four years for changes in research spending to be reflected in research performance as measured by bibliometric estimates of the quality and quantity of academic outputs.

The United Kingdom remains one of the most productive countries in terms of highly cited papers, producing a high proportion of the world’s top 1% most highly cited publications. In relation to a set of comparator countries, it is also the most efficient in the production of the world’s most cited publications.

Through five case studies, the team analysed ‘nonlinearities' in the relationship between funding and performance, focusing on two particular cases: ‘cliff-edges’, points where a small decrease in funding results in a significant decrease in some or all measures of performance; and ‘take-off points’, points where a small increase in funding improves performance significantly.

Five general themes were identified across the five case studies. The themes are:

  1. Cliff-edge and take-off depend on perspective: In several cases it was found that what was perceived as a cliff-edge by some stakeholders, generally academia, was not perceived as such by other stakeholders, such as policy makers.
  2. The evolution of a research field can lead to different interpretations: To some a field may evolve, to others it is in decline. Important in these judgements is how the field is defined: a broad conception of a field may yield a different judgment than a narrow conception.
  3. It is crucial to understand the wider context around changes in performance: In any study of change, context is important, and the case studies reconfirm the importance of understanding the context of wider policy issues, other changes in industry, economic factors, and the assessment of cliff-edges and take-offs.
  4. The mechanisms through which spending decisions are translated into disproportionate changes in research capacity and excellence can vary.
  5. Early warning signs of disproportionate effects are not easy to identify: The team noted that some signals currently used, such as individual reports and signals from areas outside of the research (e.g. in industry), should be treated with caution.