One of my interests is building evaluation capacity within nonprofit community service organizations to increase impact. The model RAND uses for this work is called Getting To Outcomes (GTO), a free program management tool with demonstrated success in improving program performance.
Getting To Outcomes emphasizes Continuous Quality Improvement (CQI), which we define as a planned, structured process with the goal of improving outcomes. We have adapted the Plan-Do-Study-Act (PDSA) cycle approach for use in community service settings, where data and resources to conduct analyses are often limited as compared with settings where PDSA was first developed (e.g., manufacturing) and currently used (e.g., hospitals). Plan-Do-Study-Act allows organizations to pilot small changes and see if these changes enhance service delivery and outcomes. Inherent in this approach is the use of data to analyze a program or process and understand the impact of applying the PDSA cycle. Critical to PDSA are support from leadership and the inclusion of multiple levels of the organization (i.e., a team-based strategy) to plan, design, implement, and study impact. For example, we've found that it is critical for leaders to support a dialogue among program staff about the potential strengths and weaknesses of a particular program and to support and provide resources to staff to make changes. Another important component of CQI is that it is ongoing. That means that it is not a one-time event, but should be incorporated into regular staff meetings for progress monitoring and mid-course adjustments, as well as for celebrating successes and problem solving setbacks.
Why use GTO and conduct CQI? Many organizations that we have worked with indicate that this approach has helped improve reporting and communication both within and external to their organization (e.g., with funders and other key stakeholders). For example, one CEO stated that GTO has created, "a common framework for program planning and evaluation that makes my job easier." Another executive stated, "[it] helps to create a common language when there are people from different fields coming together. By all using the same model we can be on the same page." Nonprofit staffs have reported to us that CQI has held them more accountable and motivated them to improve their programs. In addition, organizations have noted that this approach keeps track of progress towards their goals. For example, a program director from the Council of Alcoholism and Drug Abuse in Santa Barbara, CA reported, "we use it everyday to know where we stand. It provides a step-by-step plan on what needs to be done. It keeps us on track and highlights our strengths and weaknesses." Shannon Parker, manager of residential services at the Skid Row Housing Trust, a provider of permanent supportive housing for the chronically homeless in downtown Los Angeles, reported: "this method takes the mystery out of improving the quality of programming and services. The method is clear, accessible, and provides clear steps to planning, studying, acting and measuring your progress."
Getting To Outcomes has been deemed an "evidence based process" by the federal Substance Abuse and Mental Health Services Administration, and the GTO manual was awarded Best Publication by the American Evaluation Association in 2008. It has been applied to a variety of different domains and translated into Spanish, Japanese, and Hebrew. Although the manual is designed to be of the "self-help" variety, we recommend training and technical assistance to facilitate its use. For more information about Getting To Outcomes and performing CQI, you may contact me at: shunter@rand.org.
Sarah Hunter is a behavioral scientist at the RAND Corporation, a nonprofit research organization.
This commentary originally appeared on Independent Sector on March 21, 2012. Commentary gives RAND researchers a platform to convey insights based on their professional expertise and often on their peer-reviewed research and analysis.