Generalizability of Studies on Mental Health Treatment and Outcomes, 1981 to 1996

Published in: Psychiatric Services, v. 56, no. 10, Oct. 2005, p. 1261-1268

Posted on RAND.org on December 31, 2004

by Joel T. Braslow, Naihua Duan, Sarah L. Starks, Antonio Polo, Elizabeth Bromley, Kenneth B. Wells

Read More

Access further information on this document at ps.psychiatryonline.org

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: This study operationalized and measured the external validity, or generalizability, of studies on mental health treatment and outcomes published in four journals between 1981 and 1996. METHOD: MEDLINE was searched for articles on mental health treatment and outcomes that were published in four leading psychiatry and psychology journals between 1981 and 1996. A 156-item instrument was used to assess generalizability of study findings. RESULTS: Of more than 9,000 citations, 414 eligible studies were identified. Inclusion of community sites and patients from racial or ethnic minority groups were documented in only 12 and 25 percent of studies, respectively. Random or systematic sampling methods were rare (3 percent), and 75 percent of studies did not explicitly address sample representativeness. Studies with funding from the National Institute of Mental Health (NIMH) were more likely than those without NIMH funding to document the inclusion of patients from minority groups (30 percent compared with 20 percent). Randomized studies were more likely than nonrandomized studies to document the inclusion of patients from minority groups (28 percent compared with 17 percent), include patients with comorbid psychiatric conditions (31 percent compared with 19 percent), and attend to sample representativeness (28 percent compared with 15 percent). Modest improvements were seen over time in inclusion of patients from minority groups, inclusion of patients with psychiatric comorbidities, and attention to sample representativeness. CONCLUSIONS: Generalizability of studies on treatments and outcomes, whether experimental or observational, remained low and poorly documented over the 16-year period.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.