Deciding Not to Measure Performance

BACKGROUND: No Health Plan Employer Data and Information Set (HEDIS) performance measures evaluate health plans for possible overuse or inappropriate use of antibiotics. Acute otitis media (AOM), or infection of the middle ear, is one of the most common infections in children. The antibiotic resistance of the bacteria that cause AOM and the general overuse and inappropriate use of antibiotics for this condition are taking center stage as a major public health threat. An effort was undertaken to develop a new HEDIS performance measure that evaluates appropriate antibiotic use in children with AOM. THE MEASURE DEVELOPMENT PROCESS: The measure development process has three major phases: the initial development phase, the field-testing and analysis phase, and the measure refinement phase. With AOM, the measure development process could not proceed beyond the first phase for reasons that are be discussed in detail. CONCLUSIONS: Additional difficulties beyond feasibility issues may arise in developing a quality of care performance measure. The measure development process discussed in this article failed primarily because of issues related to relevance as well as the scientific soundness of the proposed measures. If the evidence base related to the diagnosis and management of a particular condition is sparse, no standard can be developed against which to measure performance, and the criterion of scientific soundness cannot be met. This is the case with AOM. Unfortunately, the evidence base will likely remain inadequate to support the development of such a measure in the future.