Independent Evaluation of the New York State 1115 Waiver Amendment: The Children's Design
Interim Findings
RAND Health Quarterly, 2022; 9(3):2
RAND Health Quarterly is an online-only journal dedicated to showcasing the breadth of health research and policy analysis conducted RAND-wide.
More in this issueAbstract
In an effort to streamline care for children and youth under age 21 with needs for home- and community-based services (HCBS), New York State consolidated several prior 1915(c) waivers into a single Children's Waiver and amended their Section 1115 Medicaid Redesign Team waiver. The 1115 waiver amendment allows the state to move services covered by the consolidated Children's Waiver from fee for service to Medicaid managed care (MMC) and to target eligibility to medically needy family-of-one children who meet clinical criteria and are enrolled in the consolidated Children's Waiver but do not qualify for Medicaid due to family income. Together, these waiver amendments are called the “Children's Design,” which was implemented in 2019. In this interim evaluation, the authors identify facilitators of, and barriers to, implementation of the Children's Design and describe baseline trends in outcomes of interest to its future evaluation. The authors found that: (1) providers, advocates, MMC plan representatives, and government partners perceived the transition to the Children's Design as challenging and were particularly concerned about the burden of accessing care on children's families and reductions in service availability; (2) prior to the Children's Design implementation, parents of children with chronic conditions had high levels of satisfaction with their primary care providers but were less satisfied with their ability to access special equipment and therapies and with coordination efforts among multiple providers; and (3) at baseline, the levels of quality indicators for children did not change significantly, with the exception of some primary care indicators for young children, which improved.
For more information
Full Text
Purpose of Evaluation
As part of ongoing redesign efforts, New York State (NYS, or the State) proposed, and the Centers for Medicare & Medicaid Services (CMS) approved, concurrent amendments to the 1115 Medicaid Redesign Team (MRT) waiver and the 1915(c) Children's Waiver. The six prior 1915(c) waivers for children were consolidated into a new 1915(c) waiver, which aimed to streamline care for children and youth under age 21 who have needs for physical and behavioral health services and home- and community-based services (HCBS). The 1115 waiver amendment allows the state to move the services covered by the consolidated 1915(c) Children's Waiver from fee for service (FFS) to Medicaid managed care (MMC) and to target eligibility to medically needy Family-of-One (Fo1) children who meet clinical criteria and are enrolled in the consolidated 1915(c) Children's Waiver. Together, these waiver amendments are called the children's Medicaid system transformation, i.e., the “Children's Design.” The consolidated 1915(c) Children's Waiver was implemented in April 2019, and the 1115 waiver was implemented in October 2019, except for children and youth in foster care, for whom mandatory MMC enrollment will start in July 2021.
To meet the requirements for the MRT 1115 waiver renewal application and to determine the impact of the Children's Design demonstration on both providers and clients during the transition to the program, RAND Corporation conducted an evaluation to inform both the NYS Department of Health (DOH) and CMS of the results thus far. Since this interim study is for the renewal of the 1115 waiver, the evaluation has an emphasis on the 1115 waiver component of the Children's Design, although it is not possible to disentangle the effects of the 1115 waiver's impact without acknowledging that the administrative and organizational context for the care covered by the 1115 waiver was established through the consolidated 1915(c) Children's Waiver.
Specifically, the purpose of this evaluation is to
- identify the facilitators of, and barriers to, the Children's Design implementation
- describe and delineate the baseline (i.e., pre-implementation) trends in the outcomes of interest to prepare for a summative evaluation
- assess the feasibility of identifying comparison groups and conducting difference-in- differences analyses or comparative interrupted time series analyses for the final summative evaluation.
Given the timing of this interim study, most data provided by NYS DOH for the evaluation thus far are for the period prior to the implementation of the demonstration. Therefore, this interim evaluation provides a baseline for the target population, examines stakeholder perspectives on the early implementation of the demonstration, and presents suggestions for the summative evaluation to follow.
Table 1 lists the research goals and questions that have been approved by CMS and are addressed in this interim study. There are six goals in the CMS-approved evaluation plan, which cover the demonstration implementation, care coordination, care access, and quality of care. This interim evaluation covers goals 1, 2, 3, and 5. Goals 4 and 6, as well as some research questions under goals 1, 2, 3, and 5, are outside of the scope of this interim evaluation and will be addressed in the final summative study. Among goals and research questions covered in this interim evaluation, goal 1, research question 1.1, goal 5, and research question 5.1 are related to implementation facilitators and barriers, and the remaining questions are about the baseline trends in the outcomes of interest. Research questions not included in the current study are shown in italics in Table 1.
Table 1. Summary of Research Goals, Questions, and Key Findings
| Goal | Research Question | Measure | Key Findings |
|---|---|---|---|
| 1. Improve the health outcomes for individuals under 21 receiving HCBS (HCBS Child/Youth) with access to the MMC delivery system. | 1.1. What are the consequences of targeting availability of HCBS to a more narrowly defined population than that meeting the criteria in the State Plan? | Stakeholder interviews (2020–2021): Stakeholder perspectives on implementation barriers and successes; consequences of targeting availability of HCBS to a narrowly defined population |
|
| 1.2. What are the per member, per month (PMPM) costs of HCBS for children enrollees who receive services, and how have they improved health outcomes? | To be addressed in the final summative evaluation | ||
| 1.3. To what extent are children with special needs accessing primary care providers (PCPs) who understand the children's needs? | Consumer Assessment of Healthcare Providers and Systems (CAHPS) Children with Chronic Conditions (CCC) survey (2018):
|
High satisfaction of parents of CCC with doctor's understanding of child and family life (94% and 90%, respectively) (Due to the data availability, for consumer satisfaction measures, the CCC population was considered similar to and used to approximate that of the Children's Design.) |
|
Medicaid Data Warehouse (2017–2019):
|
|
||
| 2. Improve health outcomes and increase long-term financial savings through improved access to the additional Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) benefits that address early behavioral health needs and health needs of children. | 2.1. To what extent are MMC enrollees accessing community-based specialty services in a timely manner? | CAHPS CCC Survey (2018):
|
|
| 2.2. To what extent are MMC enrollees accessing community-based health care or integrated health/behavioral health care in a manner that results in improved health care outcomes? | Medicaid Data Warehouse (2017–2019): follow-up after hospitalization for mental illness among children or adolescents ages 6 to 17 |
|
|
| Medicaid Data Warehouse (2017–2019): follow-up care for children prescribed attention-deficit/hyperactivity disorder (ADHD) medication |
|
||
| Medicaid Data Warehouse (2017–2019): metabolic monitoring for children and adolescents on antipsychotics |
|
||
| 3. Increase appropriate access to the uniform HCBS benefit package for children who meet level-of-care (LOC) criteria to achieve improved health outcomes while recognizing that children's needs, including the duration, scope, and frequency of services, change over time. | 3.1. How has enrollment in HCBS increased over the length of the demonstration? | Medicaid Data Warehouse (2017–2021): the number of children enrolled in HCBS |
|
| 3.2. What are the demographic, social, functional, and clinical characteristics of the HCBS population; and do they change over time? | To be addressed in the final summative evaluation | ||
| 4. Increase access to HCBS under the demonstration and reduce the number of children being referred and diverted to more costly institutional levels of care. | 4.1. To what extent has the demonstration improved the availability of HCBS for children? What are their health outcomes, and have they been able to remain in the community? | To be addressed in the final summative evaluation | |
| 4.2. To what extent are HCBS cost effective? What are the PMPM costs of inpatient psychiatric services, substance use disorder (SUD) ancillary withdrawal, hospital- based detox, and emergency room services for the children's HCBS population? Are these costs decreasing over time? | To be addressed in the final summative evaluation | ||
| 5. Improve access to the integrated Health Home model for all children to improve the coordination of care for children and increase access to services. | 5.1. To what extent are Health Home/HCBS enrollees accessing primary care? | Stakeholder interviews (2020–2021): stakeholders’ perspectives on care coordination |
|
Medicaid Data Warehouse (2017–2019):
|
|
||
| 5.2. (Access to Care): To the extent that there is capacity for HCBS services, to what extent are Health Home/HCBS/Fo1 enrollees accessing community-based health care or integrated health/behavioral health care? | To be addressed in the final summative evaluation | ||
| 5.3. Are Health Home/HCBS enrollees accessing necessary services, such as health monitoring and prevention services? Are chronic health and behavioral health conditions being managed appropriately? | Medicaid Data Warehouse (2018):
|
As of 2018, rates among children in HHSC and MMC were similar (90–91%). | |
CAHPS CCC Survey (2018):
|
|
||
| 6. Improve continuity of care for youth as they transition into the adult Medicaid services system, specifically to the Health and Recovery Plan from the children's Medicaid Mainstream Managed Care benefits | 6.1. Are chronic health and behavioral health conditions for young adults (e.g., ages 21–25) who transition to adult HCBS and other Medicaid services in the demonstration being managed appropriately? | To be addressed in the final summative evaluation |
Analytic Approach
A combination of qualitative and quantitative methods was used to answer the research questions to be addressed in this interim evaluation of the Children's Design.
Qualitative Methods
We collected qualitative data through semi-structured interviews with various stakeholders and used those data to explore implementation barriers, facilitators, and consequences. We conducted 12 semi-structured interviews between November 2020 and January 2021 with 26 key informants representing four types of stakeholders: children's advocates, providers, Medicaid managed care plans, and state agencies. Some interviews included multiple respondents representing different roles within the organization. Stakeholders were selected for participation from a list provided by NYS DOH to ensure adequate representation of different affected populations and types of stakeholders. Where possible, stakeholders were selected from different regions of the state, so as to ensure representation of New York City (NYC), urban areas outside of NYC, and rural areas. Data from these interviews were coded based on the goals addressed in the interim evaluation of the Children's Design.
Quantitative Methods
We conducted analyses on baseline levels of access to and quality of care using quantitative data provided by NYS DOH. We obtained population-level aggregate data derived by NYS DOH from the 2017–2019 Medicaid Data Warehouse dataset, which covers the period from April 2017 to December 2019. The majority of this time period covers the baseline period prior to the implementation of the 1115 waiver, which occurred in October 2019. Depending on specific outcome measures, analyses are based on the three subpopulations: children in Health Homes Serving Children (HHSC), FFS, and MMC. Consistent with guidance from NYS DOH in January 2021, the HHSC population is considered most similar to the target population of the Children's Design in terms of the needs for HCBS, so we treat trends in the HHSC population as a reflection of what baseline trends could have looked like for the intervention group and use the FFS and MMC populations to make benchmark comparisons. Note that FFS and MMC populations for children and youth under the age of 21 have less need for HCBS compared with HHSC or the Children's Design target population.
We used a number of quality measures derived from the Medicaid Data Warehouse (a dataset that includes Medicaid eligibility, managed care enrollment, and encounter and payment information), adapted from the Medicaid Core measures that were designated by CMS for Medicaid programs, and the Healthcare Effectiveness Data and Information Set (HEDIS) measures, including well-care visits, follow-up visits after hospitalization and medication prescription, immunization rates, metabolic monitoring for children prescribed antipsychotics, and weight and nutrition counseling. For each quality measure, we graphically charted the trends and conducted pairwise t-tests to assess the differences between groups. We also used a linear regression model to test whether the trends between the first seven time points and the last time point, which covers the first three months of implementation of the 1115 waiver demonstration, differ significantly across groups. (Each data point represents the subsequent 12 months. For example, the April 2017 data point includes the data covering April 2017–March 2018.)
In addition, for select research questions, we used data from the 2018 NYS-specific Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey of children with chronic conditions (CCC) to complement the analysis. Although both the Children's Design populations and CCC have chronic needs for care, the former has greater needs for HCBS, whereas the latter are for medical care. Due to data availability constraints, CCC was used to approximate the target population when examining consumer satisfaction.
Findings
Below we summarize highlights of our findings for each of the four goals being presented in this study. Within each goal, we briefly address the relevant research questions. Table 1 condenses this information into a matrix of key findings.
Goal 1: Improve health outcomes for individuals under 21 receiving home- and community-based services (home- and community-based services child/youth) with access to the Medicaid managed care delivery system
Research Question 1.1: What are the consequences of targeting availability of home- and community-based services to a more narrowly defined population than that meeting the criteria in the State Plan?
Based on interviews, we found that the changes under the Children's Design were perceived by the stakeholder community as a dramatic reorganization of the care system in which they were accustomed to working. They perceived the transition as challenging for providers and families as the administrative complexity increased. Stakeholders noted concerns about (1) the burden of accessing care for children's families and (2) reductions in service availability due to a decrease in the proportion of providers who participate in the Medicaid HCBS program in the context of preexisting workforce shortages. Stakeholders recognized that it is too early to fully assess the impact of the Children's Design on use of care or outcomes, in particular because of the overwhelming impact that the coronavirus disease 2019 (COVID-19) pandemic had on the entire system. The interviews with MMC plans demonstrated that under the Children's Design, there was great potential for integrating care and improving quality.
Research Question 1.3: To what extent are children with special needs accessing primary care providers who understand the children's needs?
In 2018, prior to the Children's Design implementation, a survey showed high satisfaction with primary care providers (PCPs) among parents of children with special needs. Approximately 94 percent of parents of CCC in the survey reported being satisfied with their PCP's understanding of their child's daily life, while 90 percent reported that they think that their doctors understand their family's daily life.
During 2017–2019, the percentage of HHSC receiving six or more well-child visits in the first 15 months of life was 37 percent, lower than that of those in MMC (62–65 percent) and to a lesser extent, those in FFS (46 percent). A similar pattern was observed for the percentage of children receiving at least one well-child visit in the third, fourth, fifth, and sixth year of life: MMC rates were highest (81 to 84 percent), followed by HHSC (68 to 74 percent), and FFS (43 to 44 percent). The percentage of adolescents 12 to 21 years of age receiving at least one adolescent well-care visit was higher among HHSC and MMC, ranging between 64 and 66 percent, whereas the rate was much lower among FFS, at 28 percent.
Goal 2: Improve health outcomes and increase long-term financial savings through improved access to the additional early and periodic screening, diagnostic, and treatment benefits that address early behavioral health needs and health needs of children.
Research Question 2.1: To what extent are Medicaid managed care enrollees accessing community-based specialty services in a timely manner?
In 2018, 78 percent of all parents of CCC who needed special medical equipment reported that it was usually or always easy to get it. The rates were 81 percent for special therapy and 76 percent for treatment or counseling.
Research Question 2.2: To what extent are Medicaid managed care enrollees accessing community-based health care or integrated health/behavioral health care in a manner that results in improved health care outcomes?
During 2017–2019, about 60 percent of children in MMC ages 6 to 17 who were hospitalized for treatment of selected mental illnesses had one follow-up visit within seven days of discharge; the HHSC rate was similar in 2017 but surpassed the MMC rate in later years, reaching 65 percent in 2019. The rates were lowest for children in FFS: 33 percent in 2017 and 37 percent in 2019. The rates of follow-up within 30 days of discharge were similar to the rates at seven days for all three groups.
For children ages 6 to 12 who were newly dispensed a medication for ADHD, those in HHSC had higher rates of outpatient mental health follow-up within 30 days—the initial phase—(ranging from 67–73 percent during the study period) than children in MMC (59–60 percent) and FFS (42–45 percent). Both children in HHSC and children in FFS had declines in the rate of follow-up over the study period. Children in HHSC had significantly higher follow-up rates in the continuation and maintenance phase of ADHD use—270 days after the initial phase—(67–77 percent) than those in MMC (66–68 percent) and FFS (36–46 percent). The rate of follow-up declined for HHSC and FFS over the study period, while MMC rates were mostly stable.
About 40 percent of children and adolescents in MMC and HHSC with two or more antipsychotic prescriptions had metabolic monitoring during the measurement year, and the rate remained constant over the study period, compared with a rate of 24–27 percent in FFS children.
Goal 3: Increase appropriate access to the uniform home- and community-based services benefit package for children who meet level-of-care criteria to achieve improved health outcomes while recognizing that children's needs, including the duration, scope, and frequency of services, change over time.
Research Question 3.1: How has enrollment in home- and community-based services increased over the length of the demonstration?
The number of children enrolled in HCBS was relatively stable before the consolidation of 1915(c) Children's Waiver at just above 7,100 but dropped to approximately 6,200 by 2019 upon the implementation of children's HCBS. As of February 2021, enrollment increased to just under 8,000. Please note that due to the recent implementation of the Children's Design, the data continues to be refined to ensure accuracy and alignment. Such data updates will be reflected in the final summative evaluation.
Goal 5: Improve access to the integrated Health Home model for all children to improve the coordination of care for children and increase access to services.
Research Question 5.1: To what extent are Health Home/home- and community-based services enrollees accessing primary care?
While stakeholders view care coordination under the Children's Design to be less intensive than care management under the previous system, some reported potential benefits resulting from the integration of care coordination with other health services within Medicaid managed care plans. Providers, advocates, and state officials did not perceive an impact of the Children's Design on access to primary care. Interviewees from MMCs reported that the Children's Design would enable better integration between primary care and behavioral health services, including HCBS, but they did not emphasize the improvement in primary care access as a goal.
To look at access to primary care from a quantitative perspective, we examined rates of immunizations, which were different among the three populations in the study, but relatively stable over time. The percentage of children who had the recommended immunizations by their second birthday has been consistently highest among children in MMC, at about 22 percent during 2017–2019. The rates for children in HHSC and children in FFS are very similar and stable over time, both at about 13 percent. The percentage of adolescents 13 years of age who had the recommended immunizations by their 13th birthday for MMC and HHSC was similar, ranging from 21 to 24 percent during the study period. Children in FFS had significantly lower rates than the other two groups, ranging between 8 and 10 percent over the study period.
Research Question 5.3: Are Health Homes Serving Children/home- and community-based services enrollees accessing necessary services such as health monitoring and prevention services? Are chronic health and behavioral health conditions being managed appropriately?
During 2018, approximately 91 percent of the sample of children ages 3 to 17 with an outpatient primary care or obstetrician/gynecologist (OB/GYN) visit had evidence of at least one form of weight management or nutrition counseling: BMI measurement, counseling for physical activity, or counseling for nutrition. The rate was similar among all children in MMC, at 90 percent.
In 2018, the percentage of parents of CCC who reported receiving help to get special medical equipment (79 percent), special therapy (62 percent), treatment or counseling (58 percent), and care coordination (63 percent) were comparable with those of the population that includes children with a condition that lasts for at least three months.
Conclusions
In summary, we found that families of children eligible for Children's Design, providers, advocates, and MMC representatives considered the transition from the pre-demonstration system to be challenging as the administrative complexity of accessing HCBS increased. They voiced concerns over the greater burden of accessing care and the utilization implications of the eligibility process for HCBS under the demonstration. The interviewees from MMC plans did perceive that under the demonstration, there was great potential in integrating care and improving care delivery. Quantitative analyses of baseline data show that children in HHSC, comparable with the Children's Design target population, had performance on selected measures of health care quality similar to those in MMC and higher than for children in FFS, except for access to primary care and immunizations among young children.
At the time of this writing, we are unable to draw definitive conclusions regarding the effect of the Children's Design on care coordination, care access, and quality of care due to limited data for the post-implementation period. Based on stakeholder perceptions of the enrollment process and care coordination, NYS DOH may consider additional initiatives to educate families, service providers, care coordinators, and MMC plans about the new eligibility determination, enrollment, and care coordination processes. The final summative evaluation will include data from a longer post-implementation period, which will allow quantification of broader effects, including utilization and cost, of a more mature program. In addition, the use of individual-level data and HCBS-specific outcomes measures, which are not currently available, will strengthen the rigor of the evaluation.
This research was sponsored by the New York State Department of Health and conducted by RAND Health Care.
More in this issue
