Apr 20, 2008
Aiming to improve on existing research on after-school programs, RAND researchers evaluated the effect of the SFK program on a wide variety of behavioral outcomes. They found that SFK had positive effects on nearly every domain tested, and many of these effects persisted at 12-week follow-up. Notably, the effect sizes exceeded the average effect sizes found for other programs with like goals. This suggests that SFK is a successful program model. This study also provides the first causal link between spiritual development and resilience.
The United States invests considerable resources in after-school programs—in 2002, the federal government spent $3.6 billion. However, most evaluations of after-school programs focus only on academic benefits, lack rigor, and show mixed results, so how and how much these programs benefit children are open questions.
The RAND Corporation aimed to address the shortcomings of existing research in its evaluation of one promising after-school program, Spirituality for Kids (SFK). The SFK program began in Los Angeles five years ago and has since expanded to New York, Miami, and Las Vegas, as well as internationally to London, Mexico City, Panama, Israel, and Malawi. Despite its name, SFK is not a religious program. Its curriculum is based on established best practices in the resiliency literature that seek to build four areas of personal strength: social competence, problem solving, autonomy and self-efficacy, and sense of purpose.
The program has received enthusiastic support worldwide, but program effects have been documented only informally through testimonials. This RAND study is the first formal evaluation of the program's quantitative effects on a wide variety of behavioral and attitudinal outcomes. Researchers examined the impact of the level 1 SFK after-school program, a 10-week curriculum designed to teach children how to access inner resources and build positive connections with others.
Nineteen program sites serving 737 children in southeast Florida were randomized to treatment or wait-list control groups. The researchers surveyed teachers and children before the program began, right after it ended, and 12 weeks after it ended. They used a well-known, validated survey instrument that asked about positive behaviors (such as adaptability, communication, leadership, social skills, and study skills) and negative behaviors (such as anxiety, depression, and attention, conduct, and learning problems). They analyzed the data using quasi-experimental methods, controlling for site variation and baseline characteristics of children.
The SFK program had positive effects on virtually every domain covered by the survey. As illustrated in Figure 1, the researchers found medium to large effects (effect sizes of 0.63 to 0.80) on positive behaviors. Notably, the effect sizes exceed the average effect sizes found for other after-school programs targeting similar outcomes.
As seen in Figure 2, the researchers found small to medium effects on the incidence of behavioral problems, especially attention problems, hyperactivity, and withdrawal. These effect sizes equal or exceed the average effect sizes of other programs targeting similar outcomes. There was suggestive evidence that many program effects persisted at 12-week follow-up.
The SFK program affected school-related outcomes, even though SFK is not an academic intervention. For example, the program had a large effect on improving reported study skills and medium effects on reducing learning problems and attention problems. SFK's success in improving school-related outcomes suggests that an extremely interesting follow-up study would be to examine treatment effects on grades and subsequent standardized test scores.
Because the program continues to expand not only to new communities but to different countries and settings, the researchers recommend a follow-up study to test the replicability of the SFK model in diverse contexts and target populations. They also recommend evaluation of the level 2 and level 3 courses to test whether they can support and perhaps even build on the effects achieved after the level 1 course. Future evaluations might also seek to extend the follow-up period beyond 12 weeks.
This evaluation not only provides SFK with the data it needs to strategize, plan, and improve, but it makes a significant contribution to the research literature. The evaluation sets standards for rigorous evaluation of after-school programs and highlights a successful program model. Finally, because of its rigorous research design, it offers the first causal link between spiritual development and resilience.