Improve the Program with CQI

Improve the Program with CQI

step 9 graphic

Now that you've planned, implemented, and evaluated your home visiting program, you've probably learned a lot. It is important to take time to examine what has been learned in order to know what should be modified in order to improve the program over time. To help you do this, the tasks in this step are based on a strategy called Continuous Quality Improvement (CQI).

Overview of Continuous Quality Improvement

CQI is a process of regularly considering information about planning, implementation, and desired outcomes in order to improve program quality. Step 9 involves a simple but systematic review of all your previous work to see what changes you could make to improve your program in the future. Regularly assessing program performance can also help you adapt to unforeseen changes in context that arise after you have started your program.

Conducting CQI

The CQI review is actually pretty simple. When you sit down to look over all that you've learned and accomplished in the previous eight steps, ask yourself and your staff a simple question—what can we do better?

Here are some suggestions for preparing for your CQI Review:

  • Engage your program staff in discussions about the CQI process—Let staff know ahead of time you are planning such a review. Get their ideas for how to do it, what information to use, and how to gather and incorporate their feedback.
  • Gather together all the information you want to review—With the results of your process and outcome evaluation in mind, gather information from your:
    • Needs assessment reports
    • Goals and desired outcomes
    • Fit worksheet
    • Capacity assessment tools
    • Program implementation plan
    • Any process evaluation data showing successes and challenges of delivering your program, which could include attendance or amount of services delivered, ratings of fidelity, and/or summary of satisfaction surveys from staff and participants
    • Data summary of outcome evaluation.
  • Set up a work group—It might be helpful to have a specific group tasked with using the information you've gathered to go through the CQI Tool and answer the questions.
  • Answer the questions in the second column—Complete each section of the CQI Tool by answering the questions using the information and data you have gathered from completing the earlier Steps of GTO-HV—namely your program's plan, implementation, and evaluation.
  • Think about and respond to the questions in the third column—In the third column, identify how your responses to the CQI questions might be used to improve program implementation. Any changes you make in how you address the steps will influence steps that come later. For example, if you find that there are new needs among families in your area (GTO-HV Step 1), you'll have to come up with new goals and desired outcomes targeting those needs (GTO-HV Step 2) as well as different programming (GTO-HV Step 3), fit (GTO-HV Step 4), and capacity assessments (GTO-HV Step 5); plans (GTO-HV Step 6); and evaluations (GTO-HV Steps 7 & 8). If you find that the only things you need to change the next time around are some of your implementation strategies (GTO-HV Step 6), then you probably will not need to make changes to any of the steps before planning (1–5).
  • Plan your improvements. Once you have identified actions to take to improve your program (third column), you should develop a plan to implement those improvements. For more detailed information on how to implement CQI plans, you may want to visit the GTO guide for implementing CQI (Hunter et al., 2010), which is available upon request from the manual authors. In this manual, you will find tools that help you interpret your data from the process and outcome evaluations to identify target areas to work on, plan and execute a change in your organization to improve programming, and study its impact.

To help you review your work and answer this question, we've provided a CQI Tool that helps you to reexamine the previous eight GTO steps to determine how your plans went and prompts you to think about what you might do differently next time.

Tool 9-1. Continuous Quality Improvement
View in New Window Download Microsoft Word Version

Instructions for using the CQI Tool:

  1. Insert the name of your program, the name of the person(s) completing the tool, and the date it was completed.
  2. Reconsider each of the previous eight accountability questions from GTO listed on the CQI Tool in Column 1 in light of all the information that you have gathered about your program. Summarize your conclusions in Column 2.
  3. Think about the implications of your conclusions in Column 2 for the ongoing implementation of your program. In Column 3, summarize how these conclusions will be used to improve the implementation of your program in the future.

Townville Example 9-1. Townville Conducts Continuous Quality Improvement

The Townville community coalition looked back at everything they had done and found that, in general, things had gone pretty well and little needed to be changed. However, they did realize that there were a few areas for improvement.

First, CPS had stopped offering an important intervention to at-risk families because of funding cuts. This had created a gap in the services available to at-risk families, so it was even more important for Townville to try to cover all of the eligible families. The leaders of the coalition decided to apply for funding to cover more families.

Second, home visitors were finding that they were referring families to health clinics that did not have adequate capacity to take on so many new clients. Many families thus had to wait a while before they were able to see a doctor. To remedy this issue, the lead agency suggested that it would work with clinics in a more flexible way based on their capacity for new clients.

Also, the process evaluation had revealed that many home visitors were uncomfortable with using the tablet to enter data from their home visit. The lead agency decided to implement regular refresher training for home visitors on use of the technology, and they also hired a consultant to be on-call for technical support as needed.

Finally, the outcome evaluation showed that parents were learning how to make their homes safer. However, it did not show that rates of child maltreatment were dropping. The coalition thought that this might be due to their data collection methods. Instead of looking at child maltreatment data for the entire community, perhaps they should just look at data for those families receiving home visiting services, and see whether harsh parenting techniques were declining among these families. So, they added a set of questions to the parent survey to assess how frequently parents used harsh parenting techniques.

Townville's CQI Tool is shown below.

CQI questions

Response to CQI question

How will these conclusions be used to improve program implementation?

Have the needs of the target group/resources in your local area changed?

No, the needs are still largely the same

No change needed

Have the goals/desired outcomes/target population changed?

No, the goals and desired outcomes are the same

No change needed

Are new and improved evidence-based/best practice technologies available?

Yes, the list of evidence-based home visiting programs has expanded

We will continue to use XYZ as long as the needs are the same

Does the program continue to fit with your organization (both philosophically and logistically) and your local area?

Yes

No change needed

Have the resources available to address the identified needs changed?

Yes, CPS has stopped offering their intervention program to high-risk families given funding shortfalls

We will work closely with CPS to apply for more funds to engage all identified families

How well did you plan? What suggestions do you have for improvement?

We did not adequately assess health care providers' capacity to take new clients that were referred by home visitors and some were unable to see clients promptly

We will work with clinics in a more flexible way based on their capacity for new clients

How well was the program implemented? How well did you follow the plan you created? What were the main conclusions from the process evaluation?

Program was implemented well according to process evaluation data.

However, home visitors had some trouble learning data collection using the tablet.

Regular trainings will be held to refresh home visitors on use of technology and any changes to data entry.

Technical support will be available to home visitors

How well did the program reach its outcomes? What were the main conclusions from the outcome evaluation? Have outcomes improved? Did they meet expectations?

We found an improvement in knowledge of safety procedures that was greater than expected. We didn't find changes in child maltreatment, even though we expected to see small improvements.

We will update our outcome evaluation plan to include additional measures of child maltreatment, including a parent survey to assess harsh parenting tactics

Making changes as a result of CQI analyses

If your CQI assessment suggests that you should make significant changes to your program or change the program you are delivering, it may be premature to go on to Step 10. We recommend you take some time to figure out what needs to be changed and how you'll do it. You may need to go back and re-do some of the tasks in previous steps. The authors of this manual are in the process of developing a more in-depth manual related just to CQI, Promoting Success: A Getting To Outcomes® Guide to Implementing Continuous Quality Improvement for Community Service Organizations. Feel free to contact the first author for those materials, which will lead you through a more detailed process on how to use process and outcome data to identify areas for improvement and institute improvements to enhance program outcomes.

Keep in mind that adjustments to improve the functioning of your program do not need to be major. You may find, for example, that the delivery of case management could be improved by working with one case manager to help him or her engage with clients more effectively. Such adjustments can be made while keeping other successful elements of your program moving ahead.

It's also important to keep in mind that CQI is a continuous process, and thus CQI should be integrated into how the organization functions in an ongoing way, rather than being a one-time event. These processes could take place as frequently as every 90 days to once each year.

Checklist 9-1. Completion of Step 9

When you finish working on this step, you should have done the following:

  • Completed the CQI Tool.
  • Documented successful program activities.
  • Assessed program activities that did not work well overall or for specific groups in order.
  • Identified areas for improvement.
  • Created strategies for improvement.
  • Increased buy-in within your organization by soliciting and acting on the suggestions of program staff.

Before moving on to Step 10

In each chapter, we've tried to suggest ways you could begin thinking creatively about how to implement and sustain your program at each stage of its development. In Step 10, we'll summarize these and other ideas for sustaining the successes of your program.