Development and Evaluation of Candidate Standardized Patient Assessment Data Elements

Findings from the National Beta Test (Volume 8: Observational Assessments of Cognitive Function, Mental Status, and Pain)

by Maria Orlando Edelen, Anthony Rodriguez, Sangeeta C. Ahluwalia, Emily K. Chen, Catherine C. Cohen, Sarah Dalton, Jason Michel Etchegaray, Wenjing Huang, Brenda Karkos, David J. Klein, et al.

Full Document

FormatFile SizeNotes
PDF file 0.7 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What is the feasibility, reliability, and validity of these candidate SPADEs?
  2. What subset of candidate SPADEs can be standardized across PAC settings?
  3. What are the strengths and weaknesses of collecting these data elements in practice, based on the perspectives of research nurses and facility and staff assessors?

The Centers for Medicare & Medicaid Services (CMS) contracted with the RAND Corporation to evaluate candidate standardized patient assessment data elements (SPADEs) in a national field test titled the National Beta Test. The National Beta Test was conducted to evaluate the performance of candidate SPADEs in the clinical categories of (1) cognitive function and mental status; (2) special services, treatments, and interventions; (3) medical conditions and comorbidities; (4) impairments; and (5) other categories, for use in four post-acute care (PAC) settings: home health agencies, inpatient rehabilitation facilities, long-term care hospitals, and skilled nursing facilities.

This is Volume 8 of the final report on the National Beta Test, which includes the identification and testing of candidate SPADEs developed specifically for patients and residents who are unable to communicate (staff assessments of mental status, mood, and pain). This volume provides results and significance tests on the feasibility, reliability, validity, stability and change over time, and sensitivity to national representativeness of the candidate SPADEs. The data elements tested include the Observational Assessment of Cognitive Function, the Staff Assessment of Patient/Resident Mood, and the Observational Assessment of Pain or Distress. All three data element sets performed reasonably well, with some variability in performance.

Key Findings

  • In terms of feasibility, rates of missing data were high for many of the data elements in the Staff Assessment of Mental Status and Staff Assessment of Patient/Resident Mood but were reasonable for the Observational Assessment of Pain or Distress. The Staff Assessment of Patient/Resident Mood took the longest to complete.
  • Interrater reliability was good to excellent for all data elements, although some data elements performed better than others. The variability in data element performance seemed to align with the content of the data elements, such that those symptoms that were more readily observed were documented more reliably.
  • Assessor feedback for these three data elements was limited, and available feedback was somewhat mixed for the three observational assessments. The data elements were all deemed to be at least somewhat to moderately clinically useful but with a moderate to high burden by the clinical assessors in this study.

Research conducted by

The research described in this report was sponsored by the Centers for Medicare & Medicaid Services (CMS) and conducted by the Quality Measurement and Improvement Program within RAND Health Care.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.