This report presents results of the first Alpha 1 feasibility test of a proposed set of items for measuring each of seven areas of health status for Medicare beneficiaries: (1) vision and hearing; (2) cognitive status; (3) depressed mood; (4) pain; (5) care preferences; (6) medication reconciliation; and (7) bladder and bowel continence.

Development and Maintenance of Standardized Cross Setting Patient Assessment Data for Post-Acute Care

Summary Report of Findings from Alpha 1 Pilot Testing

Maria Orlando Edelen, Barbara J. Gage, Adam J. Rose, Sangeeta C. Ahluwalia, Amy Soo Jin DeSantis, Michael S. Dunbar, Shira H. Fischer, Wenjing Huang, David J. Klein, Steven C. Martino, et al.

ResearchPublished Apr 27, 2017

The Centers for Medicare & Medicaid Services (CMS) contracted with the RAND Corporation to identify and/or develop standardized items to include in the post-acute care patient assessment instruments. RAND was tasked by CMS with developing and testing items to measure seven areas of health status for Medicare beneficiaries: (1) vision and hearing; (2) cognitive status; (3) depressed mood; (4) pain; (5) care preferences; (6) medication reconciliation; and (7) bladder and bowel continence. This report presents results of the first Alpha 1 feasibility test of a proposed set of items for measuring each of these health status areas. Conducted between August and October 2016, the test is one of two Alpha tests that will be completed by mid-2017 to assess the feasibility of proposed items. The results of these small-scale feasibility tests will inform a national Beta test designed to determine how well the measures perform when implemented in post-acute care settings. The Alpha 1 testing phase was successfully completed, in that all items were pilot tested among 133 patients. Items from all content areas were assessed on interrater reliability and feasibility; items from some content areas were assessed on other metrics. Items have now been revised, when necessary, based on the findings of the Alpha 1 test. Alpha 2 testing is under way with the updated, revised items.

Key Takeaways

Patient/Resident Factors

  • When patients/residents were cognitively impaired or fatigued, it was noticeably more difficult to complete the assessment.

Staff Experience with the Assessment

  • Many research nurses stated that facility staff members became more fluid with the assessment as they gained practice.
  • Conversely, when facility staff members had not done an assessment recently, they found it more difficult.
  • Debriefing after interviews to discuss challenges and questions had been helpful in improving skills.

Technical Glitches

  • The research nurses commented that sometimes questions would be skipped accidentally because pages would stick together or facility staff would skip the "testing only" items.
  • One pair of assessors mitigated these errors by asking each other at the end of a section if there was anything else they needed to cover.

Skill with Electronic Health Record (EHR) Use

  • Research nurses highlighted the importance of easy access to the EHR. Those who needed assistance said it created extra hassle and scheduling, and often felt rushed while searching it because someone else was involved in the process.

Challenge of Timing Each Section Separately

  • Research nurses shared that it was difficult to time each section separately. After a section had been completed, additional information frequently arose. However, this is unique to the testing phase and will not be a problem when these items are actually in use.

Difficulty Finding Family Members

  • Several research nurses noted that it was often difficult to find family members if they were not in the room, which made answering caregiver response items difficult.

Recommendations

  • The first category assessed in Alpha 1 testing, which included depressed mood, pain, and bladder and bowel continence, worked nearly perfectly: The items were both reliable and feasible to implement. They will require little if any modification and can proceed to Beta testing in their current form.
  • The second category — for vision and hearing, cognitive status, and care preferences — evidenced minor issues with reliability or feasibility. Some of the items in these content areas will require minor adjustments, and they may need to be retested in the Alpha 2 phase to prepare them for Beta testing.
  • The third category, which consisted of medication reconciliation, was being tested for the first time. Not surprisingly, the medication reconciliation items demonstrated the most limitations with respect to reliability and feasibility. This content area will require considerable refinement and retesting in the Alpha 2 phase to prepare for Beta testing.

Topics

Document Details

Citation

Chicago Manual of Style

Edelen, Maria Orlando, Barbara J. Gage, Adam J. Rose, Sangeeta C. Ahluwalia, Amy Soo Jin DeSantis, Michael S. Dunbar, Shira H. Fischer, Wenjing Huang, David J. Klein, Steven C. Martino, Francesca Pillemer, Tepring Piquado, Victoria Shier, Regina A. Shih, Cathy Donald Sherbourne, and Brian D. Stucky, Development and Maintenance of Standardized Cross Setting Patient Assessment Data for Post-Acute Care: Summary Report of Findings from Alpha 1 Pilot Testing. Santa Monica, CA: RAND Corporation, 2017. https://www.rand.org/pubs/research_reports/RR1895.html.
BibTeX RIS

The research described in this report was sponsored by the Centers for Medicare & Medicaid Services (CMS) and conducted by RAND Health

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.