This report presents the findings from an implementation and outcomes evaluation of First 5 LA's Welcome Baby home visitation program. The report examines participating sites' implementation of and fidelity to the program's goals.
The Welcome Baby Program
An Implementation and Outcomes Evaluation
- To what extent are sites implementing Welcome Baby to fidelity?
- Is there variability in sites' ability to reach fidelity for Welcome Baby? If so, what factors account for this variability?
- How are sites maintaining community resource and referral networks? What, if any, gaps exist in these networks?
- What are participant perceptions of and experiences with the program and Welcome Baby service providers?
- What factors contribute to participants leaving the program early?
- To what extent do participants achieve short- and intermediate-term outcomes?
- What are the relationships between program fidelity outcomes and participant outcomes?
In 2015, First 5 LA contracted with the RAND Corporation to perform an implementation and outcomes evaluation of its Welcome Baby universal home visitation program. RAND designed and executed a mixed-methods implementation and outcomes evaluation program that examined program fidelity across 12 sites in Los Angeles County, each site's community referral and resource process, staff and participant experiences with the program, factors that may influence program attrition, short-term outcomes, and the relationship between program fidelity and outcomes. Data from multiple sources were used, including interviews with staff and focus groups with participants, quantitative data from staff and participant surveys, data collected by staff and entered into an administrative database, and document review. There was significant variation across the sites in meeting fidelity thresholds. Staff qualifications and training, reflective supervision, enrolling clients in the maternity ward, and service dosage elements were most challenging. Sites performed well in participant perceptions of their home visitor, supervisory requirements, and home visit content. Considerable variation in sites' community referral and resource process was also found. Welcome Baby participants achieved better outcomes compared to local and national benchmarks in more than half of the areas measured, including more positive parenting practices, higher levels of any breastfeeding, and safer sleep environments. Few clear patterns were evident in terms of relationships between meeting fidelity thresholds and outcomes. It is important to note challenges in evaluation, such as the lack of data to assess some of the fidelity and outcome domains.
Findings vary but are generally positive
- There was great variability in the degree to which the sites achieved fidelity to the Welcome Baby model.
- For each of the 11 fidelity domains, there were large differences in the proportion of sites that met the fidelity threshold.
- The completeness of referral directories varied widely across sites.
- Both qualitative and quantitative data indicated that Welcome Baby participants generally had a positive perception of the program.
- Adherence to Welcome Baby fidelity standards, such as covering the curriculum, was related to lower rates of participants leaving the program early
- Where regional or national benchmarks were available, Welcome Baby participants exhibited better outcomes in more than half of the outcome areas measured. Welcome Baby participants exhibited lower levels of family planning and exclusive breastfeeding compared to benchmarks.
- There was little evidence of relationships between program fidelity and participant outcomes.
- A review of staff qualifications and training requirements for each position might be helpful to see how relevant these are in the future recruitment and training of Welcome Baby program staff.
- The frequency and quality of reflective supervision may benefit from further examination.
- Site-specific targets may be appropriate, given the number of births and Welcome Baby staffing levels at the different participating hospitals.
- Detailed protocols for all high-priority referral types, including public benefits, alcohol, smoking, and drug treatment, would be helpful, as well as a standardized referral directory and memoranda of understanding with service providers to improve service access.
- Site staff that performed better in certain areas may be able to share experiences and lessons learned at sites that did not perform as well.
Table of Contents
Overview of the Welcome Baby Program
Methods and Findings by Evaluation Question
Summary, Recommendations, and Limitations