In Search of the Holy Grail

Understanding Research Success

by Jonathan Grant, Steven Wooding

This Article

RAND Health Quarterly, 2012; 1(4):14


This article considers the continuing challenges facing research funders when trying to allocate research money. It focuses on the area of research policy in mental health research funding, with a particular emphasis on funding for schizophrenia research, and provides an overview of research policy in the last 20–25 years. It then goes on to consider what approaches funders could take to build an evidence base to support future decisions about funding.

For more information, see RAND OP-295-GBF at

Full Text

A major and recurring challenge facing those involved in science and science policy is how best to spend research money. This is even more pressing in fields with great diversity of science and opinion, such as mental health. An earlier version of this article was used to stimulate thinking prior to a workshop hosted by the Graham Boeckh Foundation in Montreal on 21 and 22 April 2009 to discuss the evidence base for mental health research funding, with a special interest in funding for schizophrenia research. The primary purpose of the workshop was to engage other research funders (both government and philanthropic), research practitioners and policy researchers in a discussion on mental health research funding and how to facilitate the translation of research into patient benefit. The workshop examined whether other funders are interested in policy research into the effectiveness of research funding in the field, and would derive value from it. The outcome of the workshop was the establishment of SoS Mental Health*—a network that will convene funders of mental health research in Canada, US, UK and elsewhere, along with mental health scientists and practitioners, and policy researchers interested in the science of science.

The network will identify a “living” portfolio of policy research that will lead to improvements in the effectiveness and efficiency of research funding. This portfolio will be delivered through a series of “science of science” projects. The first project to be funded through the network is Project Retrosight, a multinational study which will evaluate the translation and payback from basic or early clinical mental health research into clinical application and community practice. Building on successful methodologies used to evaluate diabetes, arthritis and cardiovascular research funding, this project will provide a long term view of the factors involved in successful research.

The Graham Boeckh Foundation was created by Tony Boeckh and his family. Tony is a Canadian philanthropist with an interest in mental health, in honour of his late son, who suffered from schizophrenia. The private family foundation supports research and programmes in the area of mental health and has previously funded a Chair in Schizophrenia Studies, based at the Douglas Hospital Research Centre of McGill University. Tony and his family are interested in improving the lives of patients, supporting families, and ultimately, finding a cure for schizophrenia. In other words, how to improve the translation of discovery to patient benefit, and whether it is possible to identify some key factors to accelerate the process.

Science policy addresses four broad questions: Which areas of science to invest in? How best to make that investment? What are the returns from those investments? And are there different roles for different types of funder? Various meetings and discussions are aimed at determining which scientific areas to fund. This is not the focus of this project. Our focus is on the evidence base for how funding actually works and whether it can inform subsequent decisions. That is, we are interested in measuring the return, or payback, from research funding with the objective of identifying what works. We want to identify bottlenecks that are hindering research translation and use funding to stimulate improving the process.

We provide a brief overview of the mental health and schizophrenia research field, the challenge of translating advances in neuroscience into patient benefit, and the current state of clinical and community practice. The purpose of this information is to provide a primer for non–mental health research specialists.

This study does not claim to be a systematic or unbiased review of the relevant literature—it inevitably draws on research in which we have been involved. Rather, it provides a selective background, drawing on our own work to supply a common starting point for discussion and to highlight key issues for debate.

The Mental Health Research Translation Gap

The last two decades have seen tremendous advances in basic sciences—the human genome has been decoded, molecular biology is unravelling the basic structure of how cells function, new imaging technologies are unveiling the intricate functions of the brain. But, how does basic research translate into clinical application?

There is a general concern that the very significant investments in basic biomedical research made over the last 20 years are not providing a dividend in terms of improvements in healthcare. This seems to come into specific focus in the field of mental health research, where, despite significant advances in the biomedical understanding of mental health and brain function, little of this has directly impacted on day-to-day clinical practice. There are still no diagnostic blood tests, imaging is still not clinically valuable or routine, treatments are still chosen largely on a trial-and-error basis, and there are no objective biochemical markers to follow. Few truly innovative treatments have been developed, and several promising new treatments (for example, second generation antipsychotics) have had disappointing results when disseminated into practice. At the same time the global burden of disease resulting from neurological disorders looks set to increase, which entails significant social and economic costs both in developed and developing economies. Two broad directions lie ahead of the field. At one level, basic neuroscience discovery has to continue because one cannot second-guess advances that are just around the corner. At another, a special effort has to be made to translate existing and new discoveries into clinically meaningful applications, recognising that this takes time and effort (Contopoulos-Ioannidis et al., 2008).

This is especially the case for schizophrenia. Schizophrenia is a chronic, severe and disabling brain disorder. It is characterised by symptoms like hallucinations, delusions, disordered thinking, movement disorders, social withdrawal and cognitive deficits. Today, schizophrenia affects approximately one per cent of the population worldwide. There is substantial burden associated with the disease, which usually has its onset in early adulthood (15–35 years) and, despite the best available treatments, approximately two-thirds of affected individuals have persistent but fluctuating symptoms.

The causes of schizophrenia are still unknown, so current treatment focuses on eliminating the symptoms of the disease using antipsychotic medications and psychosocial interventions (National Institute of Mental Health, 2007). There were no efficacious treatments for schizophrenia until the beneficial effects of chlorpromazine were discovered in the early 1950s. These treatments enabled patients to leave hospitals and function moderately well in society at large (Saha et al., 2005). However, this first generation of typical antipsychotic medications could cause side effects, such as rigidity, persistent muscle spasm, tremors and restlessness. In the early 1990s, a second generation of atypical antipsychotics with fewer side effects were developed, although trials, such as the CATIE study,** have questioned their clinical superiority, and there is now widespread concern about the adverse effects of these medications.

A number of psychosocial interventions are also used in the management of schizophrenia. These include supported employment, family psychoeducation, cognitive behaviour therapy, assertive community treatment, peer support programmes, and skills training, to name only a few (NHS Centre for Reviews and Dissemination, 2000; Keyser et al., 2008). Over the past few years, the nature of research on psychosocial interventions has been changing, including expectations that studies should resemble randomised controlled drug trials and that a more standardised process of care should be developed that assesses constituency in the implementation of the intervention.

In other words, despite substantial investments in both basic and applied mental health research, improvements in patient outcomes have been modest.*** At the same time, the burden of disease continues to increase. It is thus hardly surprising that the condition weighs heavily on the research funding agendas of governments, research councils, charities, philanthropists and industry.

A key challenge for funders, therefore, is to improve the efficiency and impact of their investments in this important area of mental health. The first step towards tackling such a challenge is to understand what the key enablers and impediments are to knowledge advancement and to translating schizophrenia-related research into effective prevention, diagnosis and treatment strategies. An evidence base is needed to answer these policy relevant questions, and to inform future funding decisions. What kind of research support, given when and to whom, delivers the best translation? What are the barriers to translation? Why did some exciting scientific discoveries not translate? In a nutshell, what are the factors that lead to success or failure in efforts to promote research translation from bench to bedside?

This is not just an academic question. Funding agencies worldwide are trying to evaluate the impact of health research in order to inform their research strategies, justify public expenditure on research, and engage stakeholder communities in illuminating the research process as well as the research itself (Grant, Hanney, and Buxton, 2004; National Institutes of Health, 2000; Smith, 2001; World Health Organization, 2002).

Concluding Comments

Researchers, including those in mental health, seek to secure more resources by identifying new scientific opportunities to fund. We believe that there should be a parallel undertaking to understand better the mechanisms for spending those funds, based on developing evidence about how funding actually works and how that evidence should inform decisions in the future. The examples we have described here are steps towards this end but are not sufficient. In order to identify correlates of research success, it is necessary to build a stock of evidence-based research that can be derived from or applied to mental health, and specifically schizophrenia, research funding. In one sense this is like chasing the Holy Grail—a never-ending quest for an illusory formula that will predict research outcomes. In another sense, it is a statement of the obvious; just as science is the effort to discover and increase human understanding of how physical reality works, science policy should be about understanding how science works.


Contopoulos-Ioannidis, D. G., G. A. Alexious, T. C. Gouvias and J. P. A. Ioannidis, Life Cycle of Translational Research for Medical Interventions. Science, 321, 2008, 1298–99.

Grant, J., S. Hanney and M. Buxton, Academic medicine: time for reinvention. Research needs researching (Letter). British Medical Journal, 328, 2004 (3 January), 48.

Keyser, D., J. K. Houtsinger, K. Watkins and H. A. Pincus, Applying the Institute of Medicine Quality Chasm Framework to Improving Health Care for Mental and Substance Use Conditions. Psychiatric Clinics of North America, 31, 2008. 43–56.

National Institute of Mental Health, Schizophrenia. Bethesda, Maryland: US Department of Health and Human Services, National Institutes of Health, 2007.

National Institutes of Health, FY 2000 Research Program Outcomes Assessment Material. US Department of Health and Human Services. Bethesda, Maryland: National Institutes of Health, 2000.

NHS Centre for Reviews and Dissemination, Psychosocial interventions for schizophrenia. Effective Health Care, 6 (3), 2000, 1–8.

Saha, S., D. Chat, J. Welham and J. McGrath, A systematic review of the prevalence of schizophrenia. PLoS Medicine, 2 (5), 2005, 0413–33.

Smith, R., Measuring the social impact of research. British Medical Journal, 323, 2001, 528.

World Health Organization, National Health Research Systems—Report of an International Workshop. Geneva: World Health Organization, 2002.


* In this context SoS abbreviates “science of science.”

** The Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Study, funded by the National Institute of Mental Health (NIMH) from December 2000 to December 2004, is a US nationwide public health–focused trial comparing the effectiveness of older and new antipsychotic medications used to treat schizophrenia. It is the largest, longest and most comprehensive trial ever carried out to examine existing therapies for this disease. See (as of 19 March 2009).

*** For example, the total amount of grants awarded from the National Institute of Mental Health, the primary funding agency for mental health research in the US has increased from around US $714 million in 1999 to over US $1.1 billion in 2008.

RAND Health Quarterly is produced by the RAND Corporation. ISSN 2162-8254.