Design of the Evaluation

The evaluation was built around a before-and-after comparison of two different patient groups. Changes in the patient population targeted for early change (the pilot group) were compared with changes in a control population (usually patients at a different site in the same organization). Data for the comparison came from the following:

Health Care Organization Surveys

Key staff and senior leaders were asked to complete brief organizational surveys. In addition, RAND reviewed existing administrative and organizational documents that helped characterize the site and provided data on the organization and market environment.

Patient Telephone Survey

At the end of the study, RAND contacted a sample of patients with the designated condition by phone and asked about their care and health status. All sampled individuals were sent an advance letter informing them about the study, inviting them to participate in an initial phone survey and a follow-up survey one year after. Patients were informed that their answers to the interview may be combined with information from their health records. Patients were paid $10 for completing each interview. Patient confidentiality was assured.

Patient Chart Review

Near the end of the study, a sample of patient charts was reviewed retrospectively to assess the quality of care before and after the site's participation in the Chronic Illness Care Collaborative.

Senior Leader Reports and Telephone Interviews

Data submitted as part of the collaborative were analyzed to see exactly which activities were undertaken and the indicators tracked and reported by the sites were reviewed. A final phone call reviewed these activities and collected final thoughts on participation, and follow-up plans.

Cost Surveys

A final brief survey collected data on the costs of participation in the collaborative and of making and implementing changes.

For more details on the design, see Shan Cretin, Stephen M. Shortell, Emmett B. Keeler, An Evaluation of Collaborative Interventions to Improve Chronic Illness Care: Framework and Study Design (PDF), Evaluation Review, February 2004. Also see Costs and Effects of Participation in Collaboratives to Improve Chronic Illness Care, RAND Corporation, WR-269, 2005.