The Improving Medicare Post-Acute Care Transformation Act of 2014 mandates that the Centers for Medicare & Medicaid Services (CMS) use standardized patient assessment data elements across post-acute care settings. CMS tasked RAND with developing and testing the data elements. This report, Volume 4 of a series, presents quantitative and qualitative data gathered during testing and interpretations of the results for cognitive status data elements.

Development and Evaluation of Candidate Standardized Patient Assessment Data Elements

Findings from the National Beta Test (Volume 4: Cognitive Function)

Maria Orlando Edelen, Anthony Rodriguez, Sangeeta C. Ahluwalia, Emily K. Chen, Catherine C. Cohen, Sarah Dalton, Jason Michel Etchegaray, Wenjing Huang, Brenda Karkos, David J. Klein, et al.

ResearchPublished Nov 15, 2019

The Centers for Medicare & Medicaid Services (CMS) contracted with the RAND Corporation to evaluate candidate standardized patient assessment data elements (SPADEs) in a national field test titled the National Beta Test. The National Beta Test was conducted to evaluate the performance of candidate SPADEs in the clinical categories of (1) cognitive function and mental status; (2) special services, treatments, and interventions; (3) medical conditions and comorbidities; (4) impairments; and (5) other categories, for use in four post-acute care (PAC) settings: home health agencies, inpatient rehabilitation facilities, long-term care hospitals, and skilled nursing facilities.

This is Volume 4 of the final report on the National Beta Test, which includes the identification and testing of candidate SPADEs in the cognitive status clinical category. This volume provides results and significance tests on the feasibility, reliability, validity, stability and change over time, and sensitivity to national representativeness of the candidate SPADEs. The data elements tested include the Brief Interview for Mental Status (BIMS), the Confusion Assessment Method (CAM), Expression and Understanding, and Behavioral Signs and Symptoms. All four cognitive status data elements performed reasonably well, showing feasibility, acceptable reliability, and substantial support from assessors.

Key Findings

All four data elements performed well, with some key differences

  • In terms of feasibility, missing data were very low for all four tested data elements, but there was some variability in time to complete. The BIMS, which consists of seven questions, took the longest to complete.
  • Interrater reliability was acceptable for all data elements, although some performed better than others. Overall kappas for the BIMS were excellent. Kappas were good to moderate for Expression and Understanding. Kappas, where calculated, were also good to moderate for the CAM. However, several kappas could not be calculated for the CAM, and no kappas were calculable for Behavioral Signs and Symptom, because of low prevalence rates.
  • Results for repeat assessment (performed three, five, and seven days after admission and at discharge) generally showed very little variability in data element scores, reinforcing the overall stability of the data elements.
  • As with the quantitative results, assessor feedback was generally supportive of all the cognitive status data elements. That is, the data elements were all deemed at least moderately clinically useful and to have reasonably low burden by the clinical assessors in this study.

Topics

Document Details

Citation

RAND Style Manual
Edelen, Maria Orlando, Anthony Rodriguez, Sangeeta C. Ahluwalia, Emily K. Chen, Catherine C. Cohen, Sarah Dalton, Jason Michel Etchegaray, Wenjing Huang, Brenda Karkos, David J. Klein, Jaime Madrigano, Monique Martineau, Terry Moore, Patrick Orr, Jessica Phillips, Ben Schalet, Victoria Shier, Susan M. Paddock, and Denis Agniel, Development and Evaluation of Candidate Standardized Patient Assessment Data Elements: Findings from the National Beta Test (Volume 4: Cognitive Function), RAND Corporation, RR-3004/3-CMS, 2019. As of September 7, 2024: https://www.rand.org/pubs/research_reports/RR3004z3.html
Chicago Manual of Style
Edelen, Maria Orlando, Anthony Rodriguez, Sangeeta C. Ahluwalia, Emily K. Chen, Catherine C. Cohen, Sarah Dalton, Jason Michel Etchegaray, Wenjing Huang, Brenda Karkos, David J. Klein, Jaime Madrigano, Monique Martineau, Terry Moore, Patrick Orr, Jessica Phillips, Ben Schalet, Victoria Shier, Susan M. Paddock, and Denis Agniel, Development and Evaluation of Candidate Standardized Patient Assessment Data Elements: Findings from the National Beta Test (Volume 4: Cognitive Function). Santa Monica, CA: RAND Corporation, 2019. https://www.rand.org/pubs/research_reports/RR3004z3.html.
BibTeX RIS

Research conducted by

The research described in this report was sponsored by the Centers for Medicare & Medicaid Services (CMS) and conducted by the Quality Measurement and Improvement Program within RAND Health Care.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.