The Problem: How Does Basic Research Translate Into Clinical Application?
There is a common perception that mental health research has been poorly translated from bench to bedside (Gould & Manji, 2004; Insel, 2009). The past two decades have seen tremendous advances in basic sciences—the human genome has been decoded, molecular biology is unravelling the basic structure of how cells function, new imaging technologies are unveiling the intricate functions of the brain. But how does basic research translate into clinical application? This question seems to come into particular focus in the field of mental health research where, despite significant advances in the biomedical understanding of mental health and brain function, these are yet to have much practical impact on the diagnosis and treatment of schizophrenia. Still there are no diagnostic blood tests, imaging is not clinically valuable or routine, treatments are chosen largely on a trial-and-error basis, and there are no objective biochemical markers to follow (Grant & Wooding, 2010).
Our Aim: Identifying Where and How Mental Health Research Has Translated Into Advances in Schizophrenia Treatment
This project forms part of the growing field of 'the science of science' and seeks to identify success in research translation and impact, and to understand how that success occurred and how it might be replicated (Grant & Wooding, 2010). It is no longer enough simply to campaign for more funding for science. At a conceptual level, we need to understand what attributes lead to research success. For example, what kind of science, what kind of scientists, what kind of settings and what kind of funding mechanisms are most effective in promoting discovery and translating the most promising research findings from bench to bedside?
This project aimed to identify where translation of mental health research has occurred, with a particular focus on schizophrenia. We set out to:
- identify the long-term clinical benefits or 'payback'* that arise from the translation of mental health research into practice;
- identify attributes that are associated with the successful translation of research to produce different types of 'payback';
- provide insights to inform funding policy.
Our Approach: Using Retrosight to Identify Attributes of Successfully Translated Research
The word 'retrosight' in the project's title refers to the idea that we can learn from the past to inform our current and future practice in science and funding policy. Clearly today's research questions are different from those of the past, but there has been less radical change in the funding and governance of research, and in the social structures of science and their interactions with healthcare and society.
In this project we both followed forwards from research carried out around 20 years ago and traced backwards from current interventions to understand their origins—often over an even longer period. In tracing forwards we focused on research in three countries—Canada, the UK and the USA—thereby providing ourselves with a variety of national contexts in which to explore the development and adoption of research.
Forward-Tracing Case Studies
We selected 18 pieces of research for the forward-tracing case studies, evenly distributed across the three countries and distributed across basic, clinical and interventional research. We picked research that was considered important at the time, by selecting papers that were highly cited in the five years following their publication. We then contacted the corresponding authors of these papers and interviewed them to construct the 'research cloud' around each paper. A research cloud is a body of research carried out by a particular research group that corresponds to a particular discovery, advance or insight. The concept of research clouds seeks to align the unit of analysis in this study with researchers' conceptions of the nature of their work.
Having identified a research cloud, we then constructed a case study tracing the development of the research, its impacts and the individuals involved through to the present day. This involved interviews with researchers and those who used the research, a review of published papers, an examination of archive material and a bibliometric analysis. To structure these case studies, we used the Payback Framework (Buxton & Hanney, 1996)—widely used to examine research impact. The Payback Framework provided both a logic model to structure the narrative of the case study and five categories into which we classified the various impacts of the research: knowledge production; research targeting and capacity building; informing policy and product development; health and health sector benefit and broader economic benefits (Table 1).
Table 1
Definitions of payback categories
| Payback Categories |
|---|
| Knowledge production: publication of papers and reports. |
| Research targeting and capacity building: (i) better targeting of future research; (ii) development of researcher skills, research infrastructure, etc |
| Informing policy and product development: (i) improved information bases on which to take policy decisions; (ii) informing product development |
| Health and health sector benefits: (i) health or quality of life gains; (ii) cost reduction in the delivery of services; (iii) qualitative improvements in the process of service delivery and effectiveness of services |
| Broader social and economic benefits: (i) wider economic benefits from, e.g., increased employment or commercial exploitation of research; (ii) social benefits from societal change, e.g., attitude change |
To ensure the accuracy of our case studies, both the investigators who carried out the research and those who had acted as interviewees reviewed the drafts. The case studies were then independently peer reviewed by two researchers with knowledge of the field, one from the same country and one from another country to provide international context.
Having identified the impacts arising from each of the 18 research clouds, we asked a nine-member international panel comprising researchers, research funders, practitioners and patient and family representatives to provide a numerical rating of the extent of impact for each case study in each payback category. Using these ratings we identified case studies of high and low performance in each category. Because we used five different measures of impact, it was possible for cases to have high impact in some areas (e.g. knowledge) and low impact in others (e.g. health).
We then compiled a list of attributes that might influence the translation of research—characteristics of the research, the researchers and the research setting. These were identified from our previous research, from the research of others and through an initial examination of the case studies in this project.
In the final stage of analysis we examined in detail those attributes that occurred more often in high-impact than in low-impact case studies in particular payback categories. Taking these attributes, we returned to the case studies to check consistency and tease out their meaning in more detail, alongside reviewing the literature available on the area to place our observation in context.
Backward-Tracing Perspectives
To provide a supplementary source of data to look at the question of research translation in mental health, we selected the six interventions (Table 2) to trace backwards from clinical advance to the antecedent research. We identified the interventions through the combination of a survey, a comparison of clinical guidelines over time and across the three countries of the study, and the input of our subject-expert advisors.
Table 2
The Six Interventions Examined
| The Six Interventions |
|---|
| The use of cognitive behavioural therapy, a form of psychosocial or 'talking' therapy, as a treatment for schizophrenia. |
| The recognition of the value of early intervention in schizophrenia and the realisation that treatment was often delayed. |
| The use of supported employment as a way to help people with schizophrenia gain and maintain employment. |
| The development, introduction, removal and reintroduction of clozapine, the first of the second-generation antipsychotic drugs. |
| The recognition of, and moves to address, the metabolic side effects of second-generation antipsychotics. |
| The recognition of the role of cognitive deficits in schizophrenia, and efforts to develop drugs to address them. |
Having selected the interventions, we then tried to develop narratives for each through a combination of telephone and face-to-face interviews, review of archived material and literature reviews. As we developed these, it became clear that the scope of the subjects covered and the contested nature of the fields meant it was not feasible to construct a single, definitive narrative for each intervention. Instead we concentrated on consulting a variety of sources and establishing a balanced perspective. This is why we refer to these narratives throughout this report as 'perspectives' rather than as narratives or case studies.
To analyse the perspectives we took a narrative approach, in which a researcher not involved in compiling the perspectives read and reviewed them, taking an iterative approach to extracting and describing the important attributes that appeared to have either promoted or hindered translation.
In the final stage of our analysis, we took the conclusions emerging from both the case studies and the perspectives and looked at how they reinforced and complemented one another to draw up a list of our overall findings and conclusions from the study.
Strengths and Weaknesses of These Methods
The approach we have taken, as with all approaches in the science of science, brings with it its own strengths and weaknesses (Guthrie et al., 2013). The strength of our approach lies in its ability to uncover details about the process of research and translation into patient benefit. The weaknesses lie particularly in the relatively small sample size, which we attempted to mitigate through stratified selection for the forward-tracing case studies and expert-guided selection for the backward-tracing perspectives. The robustness of our conclusions depends on the accuracy with which we have been able to construct our case studies and perspectives and the fidelity with which we have been able to distil the essence of these, through quantifying the impacts in the case studies, identifying the attributes that might drive translation, and extracting the key insights from the perspectives. To enhance the solidity of our findings we attempted to insulate the study observations from our conscious and unconscious biases, through randomising forward-tracing case study selection, using an external panel to rate the impact of the case studies, using an objective approach in shortlisting our attributes and through combining a series of inputs to select topics for our perspectives.
Our Findings and Their Potential Implications for Policymakers
The diagram (Figure 1) presents the complete list of findings of this report, colour coded to indicate the research strand from which they are derived, and links them forwards to the policy provocations they raise. We use the term 'provocation' rather than 'recommendation' because of the relatively small sample size of this project. Below we describe each of the top four 'headline' findings and give an assessment of the strength of the evidence supporting them. Reflecting this remaining uncertainty, we have also presented in the diagram the key questions alongside the provocations, as an emerging research agenda.
Figure 1: Findings
Headline Findings
Headline Finding 1: The case studies and perspectives support the view that mental health research has led to a diverse and beneficial range of academic, health, social and economic impacts over the 20 years since the research was undertaken.
The forward-tracing case studies captured a wide range of impacts across the five payback categories from knowledge production through health benefits to economic impacts. The backward-tracing perspectives also demonstrated the role of research in improving health as well as demonstrating the breadth of research types that contributed to these benefits.
Headline Finding 2: Clinical research has had a larger impact on patient care than basic research has over the 20 years since the research was undertaken.
The analysis of case studies and the long time lags between research and impact seen in our backward- tracing perspectives suggests that clinical research has a larger payback than basic research in terms of health, social and economic benefit over periods up to 20 years.
This finding aligns with our previous case study research that has shown clinical research having a larger health, economic and social impact than basic research over shorter timescales—10 years for arthritis, 15 years for cardiovascular research.
There are two ways in which we could be wrong about this conclusion—in other words, two ways in which basic research could be having a larger impact than clinical research over a 20-year timescale which we don't see in the data that we have examined: the impact of basic research may come from a small number of large successes, which we are less likely to have included; or basic research may be having impacts that we cannot trace or are too diffuse for us to follow. It is also possible that basic research may be necessary as the foundation of future clinical research, or that basic research may go on to have a larger impact than clinical research over longer timescales. This study does not address either of those possibilities.
Headline Finding 3: Those involved in mental health research who work across boundaries are associated with wider health and social benefits.
Our analysis of both the perspectives and the case studies shows that individuals with broad disciplinary breadth and/or the ability to bridge the gap between the research world and the policy and care spheres (or the reverse) were frequently key in driving change forwards. They often did this by getting directly involved in the implementation of their research findings. Likewise the analysis of case studies showed that researchers who worked across disciplinary boundaries were associated with research that had a higher impact. This suggests that funders should support individuals who work across boundaries—both disciplinary boundaries and stages of the translation pathway—possibly by providing soft 'expenses' type accounts and facilitating face-to-face networking activities.
Headline Finding 4: Committed individuals, motivated by patient need, who effectively champion research agendas and/or translation into practice are key in driving the development and implementation of interventions.
The analysis of case studies tentatively suggests that the motivation of researchers to improve the lives of patients may be more important than funding mechanism or research setting, but as noted this needs further investigation. The examples of committed individuals from the perspectives also tended to be motivated by patient needs in driving either the research agenda or the incorporation of advances into routine practice. Funders may benefit from giving priority to the support of researchers motivated primarily by patient need, although the identification of such individuals may not be straightforward.
References
Buxton, M., & Hanney, S., "How can payback from health services research be assessed?" Journal of Health Services Research and Policy, Vol. 1, 1996, pp. 35–43.
Gould, T.D., & Manji, H.K., "The molecular medicine revolution and psychiatry: bridging the gap between basic neuroscience research and clinical psychiatry," Journal of Clinical Psychiatry, Vol. 65, 2004, pp. 598–604.
Grant, J., & Wooding, S., In search of the Holy Grail: understanding research success, Cambridge, UK: RAND Europe, 2010.
Guthrie, S., Wooding, S., Pollitt, A., Pincus, H., & Grant, J., Mental Health Retrosight: Understanding the returns from research (methods), Cambridge, UK: RAND Europe, 2013.
Insel, T.R., "Translating scientific opportunity into public health impact: a strategic plan for research on mental illness," Archives of General Psychiatry, Vol. 66, No. 2, 2009, pp. 128–133.
Note
* We define 'payback' to mean those academic, health, social and economic benefits of research as commonly used in research impact assessment and defined by Buxton and Hanney (1996) in the development of the 'payback model'. We do not imply a numerical estimate of the return on investment, as sometimes used in economic analysis.
The project described in this article was supported in Canada by the Graham Boeckh Foundation, Alberta Innovates Health Solutions, and the Canadian Institutes of Health Research; in the UK by the National Institute for Health Research; and in the USA by the National Institute of Mental Health. The research was conducted by RAND Europe.
