Review Program Choices for Best Fit
Review Program Choices for Best Fit
In the last few steps, you've used data from your needs and resources assessment to set program goals and develop desired outcomes. Based on these goals and outcomes, you've also examined various evidence-based home visiting programs and narrowed down your options to a few candidate home visiting programs to further consider, or you may have decided to develop your own home visiting program or implement something other than home visiting. If you've chosen to move ahead with a home visiting program, or even other types of programs that more closely address your community's needs, you can now move forward with Step 4.
Why assess fit?
No matter what types of program(s) you've selected to be on your short list, it is important to assess their fit with the population(s) that you will be serving, your organization, and the context in your local area. You might need to modify a program in order to make sure that it is a good fit, or select a different program entirely if a program isn't a good fit.
There's no single magic solution for how to make a program fit perfectly. You may have to first understand and balance competing interests, such as a program's fit with the families you intend to serve versus fit with other community agencies. However, it is important to at least consider the fit ideas presented in this chapter, while knowing they may not include all of the answers you might need. Finding fit is sometimes an evolving process.
As we briefly covered in Step 3, the first thing to consider is how well you can implement a given program with fidelity while still ensuring adequate fit. Fidelity is the faithfulness with which an evidence-based practice is implemented. This includes implementing a program without removing parts of the program that are essential to the program's effectiveness—its core components.
We will first review some of the key elements of fit, and then we will discuss how you might make responsible program adaptations if necessary. Then, you can try your hand at completing the Assessing Program Fit Tool for your identified program(s).
Elements of fit
Program fit with your target population
It is important to make good use of the data collected from the needs and resource assessments in Step 1 as you determine program fit. The first step in determining program fit is considering the cultural context and "readiness" of the target population and their communities for the proposed home visiting program(s):
- Examine the population characteristics shown to be helped by your selected program(s) and then determine the extent to which they match the characteristics of your targeted population. You can use Tip 3-2 and also the summaries of the program(s) available through the links to the programs' websites (provided in Tip 3-1) to identify the populations that have been helped by your selected programs in past work.
- Consider whether the program activities and services and methods of delivery are suitable for your targeted population. For example, if your target population consists of primarily Spanish-speaking mothers, and a particular program's home visiting curriculum is offered only in English, the program activities may not be suitable to your population without modification. Not only will the curriculum need to be translated into Spanish, the curriculum will need to be evaluated for cultural competence for a Spanish-speaking population.
Program fit with other local programs and the system of care
The next type of fit is considering how your selected program(s) fit with other local programs already offered to your target population and programs that serve the larger community. This will include services offered within the broader system of care. For each identified program, you will need to examine several issues closely:
- Are similar programs being offered for this particular population? Assess what is already going on within your particular location and for the target population you wish to serve. You may use the Resources Assessment Tool you completed in Step 1 to help you with this step.
- If similar programs exist for this population, how does your program differ? Will the new program meet certain needs of the target population that are not met by the existing program? Or will it serve people not served by the existing program because of caseload, space, or budget constraints? Together with other program providers, make sure that the new program strengthens or enhances what already exists in your area for your target population.
- Is the program you are considering consistent with the broader system of care in place in your community? Does it complement or detract from the services already being provided? Does it provide a new opportunity for cross-agency collaboration? For example, is there a program already in place serving the targeted population that can serve as a referral source to your planned home visiting program?
Program fit with your lead agency
Next, determine the level of compatibility between the program(s) you are considering and the lead agency/organization that will be responsible for implementing the home visiting program that you finally choose. It is important that the goals of your program are congruent with the lead agency/organization's philosophy and values. Staff will be much more likely to deliver the program well if they believe it fits with the organization's vision and mission. Consider the following:
- Is there a strong connection between the organizational mission and the planned home visiting program to be delivered? This will contribute to the efficient use of resources and will increase the confidence of those involved in implementing the program.
- Do staff and leadership support the chosen program? Involving staff and volunteers in the process of actually selecting or adapting a program can strengthen these connections.
Which program components can be adapted?
Evidence-based programs have a recipe of activities to address specific problems or risk factors among participants. By changing parts of the recipe, you might be jeopardizing the effectiveness of the program. Think of it like making cookies. To make cookies, you need flour, eggs, oil, baking soda, and sugar. Have you ever made cookies and forgotten the eggs? Substituted baking soda for baking powder? If you take out one of these core ingredients, you won't get cookies, you will get a hockey puck! These basic ingredients are like the core components of a program, that is, the components that are essential to maintaining program fidelity. If you take out a core component, you are not implementing the evidence-based program faithfully and it's unlikely you'll get the results you expect.
Now think about different types of cookies. There are cookies with raisins, chocolate chips, or nuts, and you can choose between these based on who will be eating them. If you are preparing cookies for someone with a nut allergy, you might choose to include raisins instead. The point is, whether or not you add these elements to the basic recipe, you'll still get a cookie. These are like the things that can be changed in an evidence-based program without substantially changing the impact of the program.
Implementing a home visiting program with fidelity, or following the core recipe in the cookie example above, will lead to better outcomes. You may be able to maintain fidelity to a home visiting program's core components while tailoring the program to better meet the needs of your population, existing programs in the community, and the culture of the lead agency. This tailoring, like adding chocolate chips in the cookie example, is called adaptation.
A program's "core components" are the program's major elements. Your chosen program might just have one component, or it could have many components. Examples of program components for home visiting programs include the following:
- Hospital screenings
- Parenting classes
- Home visits
- Well-child check-ups
- Community meetings
- A public education campaign
- Recruitment activities
- Staff training and supervision
- Program activity documentation.
Before making adaptations to a home visiting program, it is important to generally understand whether you should change any of the program's core components, and if so, how. This information will help you determine whether the potential changes you want to make to achieve fit will maintain or detract from program fidelity. If you find yourself making substantial changes to the program to improve its fit, you should probably consider selecting another program.
While there is no single standard for making decisions about adapting evidence-based programs, we offer a simple "green-yellow-red light" model for determining appropriate levels of adaptations. The model is presented in Tip 4-1. You should also take note that any modifications you consider should be reviewed by the program developers, as some programs will not allow you to use the program materials if you make substantial adaptations. Once you understand what types of changes are green-light, yellow-light, and red-light changes, you may turn to examining the selected programs in Step 3 in light of your priority populations and existing services in the community. In addition to considering whether certain adaptations are appropriate, you should also consider the cost and feasibility of these adaptations and modifications (for example, the cost of translating an entire curriculum into another language).
Tip 4-1. Evidence-Based Home Visiting Programs and Target Populations
|
Source: Adapted from Rolleri et al., 2011. |
Red-light changes: Changes that substantially compromise the core components of the program. These changes, such as reducing or eliminating elements, are highly discouraged because they compromise the integrity of the original program. For example, often home visiting programs will provide a chance for parents to practice new skills. This is a critical step in changing behavior, and these skills should be practiced for the full amount of time that the program states. The activity should not be reduced or eliminated to save time. |
|
Yellow-light changes: Changes that should be made with the help of an expert in home visiting or parent support, such as a researcher or professor. Some of these changes, such as changing the sequence of activities or adding new elements to the home visits, are more substantial and require expert assistance so that alterations don't compromise the integrity of the program. |
|
|
Green-light changes: Changes that should be made, as long as they don't change or diminish the core components, to fit the program to the participants' culture and context. This includes changing things like the wording of program material to better match the cultural background of participating families. Most programs can be improved by tailoring materials to better reflect the population you plan to serve. You should feel comfortable making these types of changes for most programs. |
Assessing program fit in your community
Tool 4-1. Assessing Program Fit
In this section, we present the Assessing Program Fit Tool, which will help you through the process of identifying whether your narrowed list of evidence-based home visiting programs will fit in your community. You can use this tool to help you examine aspects of fit for the program(s) you are considering, or, if you are already running a program, you can use the tool to see if there are ways to improve the fit you have. After the tool you will find Townville's example.
Instructions for using the Assessing Program Fit Tool:
- Make one copy of this tool for each of the candidate programs you are considering. If the list of candidate programs is longer than two or three, you may consider assessing a couple of programs first, and then revisiting your short list. This might help you to more quickly assess the remaining programs.
- Assemble the basic information about each of the programs you are considering before you start work on assessing fit, and have a copy of the Green-Light, Yellow-Light, and Red-Light Program Adaptations guide on hand to help you answer questions in the fit tool about each of the programs you are considering. You may want to contact a representative from the program model's national office if they have one. The representative may be able to provide you with additional information on the program's core components and overall design.
- Write down the relevant aspects of program design for each of the questions in the tool. For example, if your target population is all first-time mothers, you might note that "The program is designed for first-time mothers" as a relevant aspect of program design.
- Next, answer yes or no to whether the program design element fits with your needs.
- Take a look at the work you have done: Do any of the programs identified require red-light adaptations to fit the population more closely? If, like Townville, you find that a program cannot be feasible without making a change that would compromise the core of the program, then it is probably not the right program for your community.
Tool 4-1. Assessing Program Fit
View in New Window Download Microsoft Word Version
Townville Example 4-1. Assessing Program Fit for Townville
In Step 3, Townville's coalition decided to continue to investigate two programs, the ABC and XYZ programs. As they evaluated program fit, they came to the following conclusions:
- The ABC program has been shown to work with a range of different cultures. In Step 2, however, the Townville community coalition had identified a local health clinic that was already conducting a similar program. Also, the methods of service delivery were quite intense, requiring home visitors to travel to family homes very frequently. In a rural community, this would take be a significant burden for employees and would increase the costs of the program. The coalition discussed this issue with the program developer, who noted that changing the number of home visits or making them phone calls would likely compromise the core components of the program.
- The XYZ program had been shown to work well in a rural population. The coalition brainstormed and realized that the XYZ program services might complement, rather than compete with, the other programs in the system of care. In fact, they might be able to leverage existing mobile clinics to support their work! The coalition also realized, however, that the lead agency did not have expertise in maternal and child health, a core focus of the XYZ program. The lead agency was going to need to hire some additional staff with this type of expertise in order to implement the XYZ program with fidelity.
Take a look at Townville's sample Assessing Program Fit tool for the XYZ program.
|
Program Name: |
Program Design |
Does this fit your needs? (Yes/No) |
What steps can be taken to increase program fit? |
Green, yellow, or red light? |
|
|---|---|---|---|---|---|
|
Fit with Target Population |
Does the program fit with your target population's literacy and/or education level? |
The program targets new mothers and doesn't specify a literacy/education level. |
Yes |
||
|
Does the program fit with your target population's age and risk profile? |
The program targets all mothers but works best with at-risk mothers. |
Yes |
|||
|
Does the program fit with your target population's culture and language? |
The program materials are in English and Spanish; the program has been adapted for rural populations. |
Yes, but not for the Native American population |
Adapt some program materials to better address the specific circumstances of the population |
Yellow light |
|
|
Fit with Other Local Programs |
Might the program interfere with other existing local programs? |
The program does not provide the same services to the same population as any other existing programs. |
Yes |
||
|
Is the program consistent with the broader system of care in place in your community? |
Yes, it complements the system of care because the services provided are not currently provided in the community. |
Yes |
|||
|
Fit with Lead Agency |
Is the program consistent with the lead agency's culture, mission, and vision? |
The program is a maternal and child support program, but the lead agency does not specialize in maternal/child health. |
Yes |
Hire additional staff trained in maternal and child health at the lead agency. |
Green light |
|
Are staff and leadership supportive of the program design? |
Staff are very excited about the program design. |
No |
|||
Checklist 4-1. Completion of Step 4
When you finish working on this step, you should have done the following:
- Developed an understanding of what "fit" means.
- Considered the most important aspects of your program and community to make sure there is a good fit.
- Decided whether the selected program(s) fits for your participants, organization, and stakeholder community.
- Determined the right adaptations to make to improve your program(s)' fit.
- Further narrowed your choice of programs to implement.
Before moving on to Step 5
After reviewing your potential programs with fit in mind, you might have a clearer idea which programs that you selected in Step 3 are still good possibilities. If none of the potential programs have passed the "fit" test you conducted here in Step 4, you may need to go back to Step 3 and do some more investigation to find a new set of programs to consider. Knowing more about fit now may also help you more quickly zero in on potential programs if you do circle back to Step 3 for more investigation.
In Step 5, we'll show you how to examine the current capacities of the lead agency/organization to make sure it can do a good job in implementing the selected program. Step 5 will be the final reviewing step before moving onto planning and implementing the selected home visiting program as well as establishing evaluation criteria.