- How was the pilot program implemented during the first year, including any barriers and facilitators to implementation?
- Is the pilot program effective in reducing the number of negative incidents (e.g., assaults, use of seclusion and restraints) and improving educational outcomes compared with a similar population of youth within DYS?
- Is the pilot program effective in reducing negative incidents related to staff (e.g., injuries, criminal charges), as well as absences and turnover?
- What are the estimated costs associated with implementation of the pilot program?
- What recommendations can be made regarding the pilot program, including any conclusions regarding scalability of the program?
There is increasing recognition of the prevalence of trauma exposure among youth involved in the juvenile justice system. To better address the needs of these youth while mitigating negative downstream consequences of their behavior (e.g., youth recidivism), there have been efforts to develop trauma-informed programming. In this report, the authors conduct an evaluation of a trauma-informed pilot program run by the Colorado Division of Youth Services (DYS) in a residential youth facility.
The authors find that core elements of the pilot program are consistent with trauma-informed models. Similar models, such as the Missouri Model, have been demonstrated to be effective in reducing negative youth and staff outcomes. However, the authors found few significant effects of the pilot program on outcomes of interest during the evaluation period. That said, there were a number of implementation challenges that may have contributed to the lack of significant differences. These include potential contamination effects, as staff and youth from across units interacted during the study period, as well as obstacles related to staffing levels and larger facility-level concerns. As the program continues to be implemented, it will be important to develop a clear program model; address concerns related to training and staffing levels; and continue to monitor the implementation and outcomes of the program.
There were few significant differences in youth outcomes
- During its first year, there were few significant differences observed between youth on the pilot unit and youth on a traditional unit at the facility. However, youth in the pilot program were less likely to experience a physical response with mechanical restraints or a Level Two physical response.
Staff outcomes were similar across units
- There was no significant difference between units regarding any of the staff outcomes examined. Staff absences and turnover were high in both groups, though no staff in either group had grievances/complaints or criminal charges filed against them. Equipping staff with better tools for behavioral management of youth may be one way to reduce absences and turnover.
Implementation considerations are important to understanding the findings
- There were many factors that may have led to "contamination" effects. For example, youth from the pilot program mixed with youth from other units during school hours, and DYS has been in the process of incorporating trauma-informed principles across facilities over the past several years. Staff also described several challenges to implementation, including absences and turnover, a need for additional training, and facility-level issues.
Pilot-specific costs were relatively low
- The authors estimate the setup costs of the pilot pod to have been $337,236. Once launched, the total cost of running the pilot pod was an estimated $3,003,352. Of this total running cost, the pilot's share of facility-wide costs accounted for 85 percent ($2,567,028), with the remaining 15 percent ($436,323) being costs unique to the pilot.
- Develop a clear description of the Colorado Model approach to trauma-informed care and the pilot program model. This would provide valuable guidance to individuals at the facility implementing the program, as well as those responsible for incorporating trauma-informed principles into facilities across the state.
- Ensure all pilot program staff members are trained in the program model. Providing this type of training to all staff — including new program staff — would ensure that staff members are operating from a shared understanding of the program.
- Address challenges in staff recruitment and retention. High levels of turnover make it difficult to consistently implement the pilot program. Increasing the pool of potential applicants for positions may be one way to address these challenges.
- Explore staff concerns regarding safety and the effectiveness of the program in more depth. Staff who are concerned for their own safety may be less likely to implement trauma-informed care principles, instead falling back on behavioral management techniques that involve physical responses. Addressing the root of safety concerns may improve pilot program implementation and staff retention.
- Consider opportunities to further develop the unique features of the pilot program. This could include ways to build on the importance of the small group process or expanding the aftercare component of the program model.
- Continue monitoring the implementation and outcomes of the pilot program, and increase LMYSC's capacity for evaluation, which could be done by implementing new data collection procedures as needed. This is key to understanding the effectiveness of the program as implementation concerns are addressed.
Table of Contents
Supplemental Youth and Staff Results
Semistructured Interview Protocol
Additional Cost Analysis Tables