Development Quality Criteria to Evaluate Nontherapeutic Studies of Incidence, Prevalence, or Risk Factors of Chronic Diseases

Pilot Study of New Checklists

Published in: Journal of Clinical Epidemiology, v. 64, no. 6, June 2011, p. 637-657

Posted on RAND.org on June 01, 2011

by Tatyana Shamliyan, Robert L. Kane, Mohammed T. Ansari, Gowri Raman, Nancy D. Berkman, Mark Grant, Gail Janes, Margaret A. Maglione, David Moher, Mona Nasser, et al.

Read More

Access further information on this document at Journal of Clinical Epidemiology

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. STUDY DESIGN AND SETTING: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. RESULTS: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. CONCLUSION: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/principles.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.