When Should Measures Be Updated?

OBJECTIVE: To document current practices on long-term maintenance of quality measures and to develop a consensus framework to guide the design of maintenance systems. STUDY DESIGN: Survey of 10 organizations developing measures and selected researchers in the USA about current policies and procedures and desirable properties for a comprehensive system for measures maintenance. Panel discussions with all respondents to arrive at consensus recommendations for a framework for maintenance of measures. PARTICIPANTS: Five measures developers, two provider and three purchaser organizations. Six were private sector organizations, two were governmental agencies, and two were accreditation institutions. PRINCIPAL FINDINGS: All organizations had procedures for measures maintenance, but the degree of formalization of the procedures varied. Three key functions for a measures maintenance system emerged: ad hoc review to deal with unexpected problems; annual maintenance to incorporate changes in coding conventions; and regular re-evaluation to thoroughly review measures at predefined intervals. Importance, scientific soundness, feasibility and usability were universally used as evaluation criteria. The panel discussions yielded a consensus set of recommendations for relationships between maintenance functions, evaluation criteria and measures disposition. CONCLUSIONS: A sufficient degree of implicit consensus was found among leading measures developers to arrive at a consensus framework for policies and procedures for measures maintenance. Although organizations may choose to implement the framework in a way that is most consistent with their mission and structure, it provides guidance regarding which components should be included.