Self-study from Web-Based and Printed Guideline Materials

A Randomized, Controlled Trial Among Resident Physicians

Published in: Annals of Internal Medicine, v. 132, no. 12, June 2000, p. 938-946

by Douglas S. Bell, Gregg C. Fonarow, Ron D. Hays, Carol Mangione

Read More

Access further information on this document at

This article was published outside of RAND. The full text of the article can be found at the link above.

BACKGROUND: On-line physician education is increasing, but its efficacy in comparison with existing self-study methods is unknown. OBJECTIVE: To compare knowledge, learning efficiency, and learner satisfaction produced by self-study of World Wide Web-based and print-based guidelines for care after acute myocardial infarction. DESIGN: Randomized, controlled trial. SETTING. 12 family medicine and internal medicine residency programs at four universities. PARTICIPANTS: 162 residents. INTERVENTIONS: In proctored sessions, participants were randomly assigned to study from printed materials or from SAGE (Self-Study Acceleration with Graphic Evidence), a Web-based tutorial system. Both methods used identical self-assessment questions and answers and guideline text, but SAGE featured hyperlinks to specific guideline passages and graphic evidence animations. MEASURERMENTS: Scores on multiple-choice knowledge tests, score gain per unit of study time, and ratings on a learner satisfaction scale. RESULTS: Immediate post-test scores on a 20-point scale were similar in the SAGE and control groups (median score, 15.0 compared with 14.5; P > 0.2), but SAGE users spent less time studying (median, 27.0 compared with 38.5 minutes; P < 0.001) and therefore had greater learning efficiency (median score gain, 8.6 compared with 6.7 points per hour, P = 0.04). On a scale of 5 to 20, SAGE users were more satisfied with learning (median rating, 17.0 compared with 15.0; P < 0.001). After 4 to 6 months, knowledge had decreased to the same extent in the SAGE and control groups (median score, 12.0 compared with 11.0; P = 0.12). CONCLUSIONS: On-line tutorials may produce greater learning efficiency and satisfaction than print materials do, but one self-study exposure may be insufficient for long-term knowledge retention. Further research is needed to identify instructional features that motivate greater final learning and retention.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.