- A team from RAND and the University HealthSystem Consortium (UHC) developed a toolkit to help hospitals enhance their quality improvement efforts using quality indicators from the Agency for Healthcare Research and Quality.
- The RAND-UHC evaluation found that most of the participating hospitals judged the tools to be useful in supporting quality improvement. They also provided feedback on the tools, which was used to revise the toolkit to enhance its usefulness and usability.
- Improved performance depended on leadership buy-in: Hospitals with board and executive support were able to do more.
- The toolkit also helped achieve staff consensus on the extent of quality gaps and supported the use of evidence-based practices.
Quality of care in U.S. hospitals is a significant problem. The pace of quality improvement efforts in hospitals has picked up in recent years, with a focus on monitoring and public reporting of quality information. Prominent among such tools are quality indicators (QIs) developed by the Agency for Healthcare Research and Quality (AHRQ), including two sets of indicators for inpatient settings: the Inpatient Quality Indicators (IQIs) and Patient Safety Indicators (PSIs). However, if hospitals lack the needed knowledge, skills, or resources, their efforts to improve performance may have little effect on quality outcomes. Many hospitals may need help to enhance their capacity to use QIs for effective quality improvement.
To meet this need, AHRQ contracted with a team of researchers from RAND and the University HealthSystem Consortium (UHC), a large collection of academic medical centers, to develop a toolkit for hospitals to support their quality improvement efforts based on the AHRQ QIs. This toolkit provides a roadmap to guide quality improvement. It includes a sequence of steps designed to help hospitals calculate desired rates for the QIs, set priorities for improvement, develop specific strategies and goals, implement these strategies, and sustain improvements. This research highlight describes this toolkit, reports findings from RAND-UHC's field-testing of the toolkit in hospital settings, and highlights ways hospitals can use the toolkit to improve their quality of care.
Developing the Toolkit
To develop the toolkit, the RAND-UHC team first reviewed an extensive literature on hospital quality improvement. First, the review assessed the characteristics and performance of AHRQ's QIs. Second, it documented the many ways hospitals are already using the QIs. Third, the review identified key factors that influence success in implementing improvements.
The RAND-UHC team convened an expert panel to help identify and develop tools. Based on the literature review and the panel guidance, the RAND-UHC team produced an alpha version of the toolkit (see table).
Field-Testing the Toolkit
To assess the usability of the candidate tools, the RAND-UHC team field-tested the kit to evaluate its helpfulness for hospitals for implementing performance improvement interventions based on the AHRQ QIs. Six hospitals participated in a formal evaluation; four others contributed informal feedback. Hospitals represented a mix of characteristics, including bed size, academic/community status, geographic diversity (regional and urban/rural), and safety net status. RAND collected primary data from the participating hospitals on improvement experiences and toolkit usability. Data were collected using pre- and post-test interviews, follow-up interviews, and usability testing interviews.
Most hospitals reported that the tools helped support improvement processes. They varied in the tools they used, and some reported using none of the tools because they already had well-established quality improvement processes and tools. Here are other key findings from field-testing:
- All of the hospitals focused on patient safety (the PSIs) rather than more traditional approaches to quality improvement (the IQIs). These decisions appeared to be responses to performance expectations from external parties, such as Medicare, other payers, and regulatory entities.
- For many hospitals, their first challenge was uncertainty about the accuracy of rates calculated for their PSIs, due to issues in documentation and coding of health records. Confidence in the accuracy of PSI rates was a prerequisite because a loss of credibility with clinical staff can irreparably damage a quality improvement project.
- Improved performance depended on leadership buy-in: Hospitals with board and executive support could do more. Hospitals cited external incentive programs based on the AHRQ QIs (such as public reporting or pay for performance) as powerful motivators for hospital boards.
- Several hospitals found that the toolkit helped achieve staff consensus on the extent of quality gaps and supported the use of evidence-based practices.
Hospitals provided feedback on existing tools and also requested additions to the toolkit. They asked for (1) a tool that provides guidance on documenting and coding medical record data for use in calculating PSIs; (2) examples of how to set priorities; and (3) guidance on best practices for additional PSIs (beyond what the alpha toolkit contained). Using this feedback, the RAND-UHC team revised existing tools and added new tools to the toolkit (see table).
A Toolkit for Hospital Quality Improvement
|Improvement Steps for Hospitals||Key Tools|
|Determining readiness for change||
|Applying QIs to hospital data||
|Identifying priorities for quality improvement||
|Monitoring progress for sustainable improvement||
|Analyzing return on investment||
|Using other resources||
a Added in response to hospital feedback.
b The alpha toolkit addressed three PSIs; five more were added with revisions.
SOURCE: "Introduction and Roadmap" and "Webinar" in Farley D, Weinick R, Mayer L, Cerese J, Burns R, and Hussey P, AHRQ Quality Indicators™ Toolkit for Hospitals, Rockville, Md.: Agency for Healthcare Research and Quality, March 2012 (http://www.ahrq.gov/qual/qitoolkit/index.html).
This project confirmed that hospitals face challenges in quality improvement and found that the toolkit was helpful in addressing them. Hospitals judged many of the tools to be user-friendly, and the RAND-UHC team employed their feedback to enhance the tools' utility. In using the QIs, however, hospitals should be aware of known validity issues and not overinterpret results. As their name suggests, the QIs are "indicators" that point to areas of potential strengths or problems. Hospitals should supplement them with additional examination of care processes and data analysis before deciding on improvement actions.