Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Abstract

Aiming to improve on existing research on after-school programs, RAND researchers evaluated the effect of the SFK program on a wide variety of behavioral outcomes. They found that SFK had positive effects on nearly every domain tested, and many of these effects persisted at 12-week follow-up. Notably, the effect sizes exceeded the average effect sizes found for other programs with like goals. This suggests that SFK is a successful program model. This study also provides the first causal link between spiritual development and resilience.

The United States invests considerable resources in after-school programs—in 2002, the federal government spent $3.6 billion. However, most evaluations of after-school programs focus only on academic benefits, lack rigor, and show mixed results, so how and how much these programs benefit children are open questions.

The RAND Corporation aimed to address the shortcomings of existing research in its evaluation of one promising after-school program, Spirituality for Kids (SFK). The SFK program began in Los Angeles five years ago and has since expanded to New York, Miami, and Las Vegas, as well as internationally to London, Mexico City, Panama, Israel, and Malawi. Despite its name, SFK is not a religious program. Its curriculum is based on established best practices in the resiliency literature that seek to build four areas of personal strength: social competence, problem solving, autonomy and self-efficacy, and sense of purpose.

The program has received enthusiastic support worldwide, but program effects have been documented only informally through testimonials. This RAND study is the first formal evaluation of the program's quantitative effects on a wide variety of behavioral and attitudinal outcomes. Researchers examined the impact of the level 1 SFK after-school program, a 10-week curriculum designed to teach children how to access inner resources and build positive connections with others.

The Study Was Rigorous; Assessed Behavior Change Over Time

Nineteen program sites serving 737 children in southeast Florida were randomized to treatment or wait-list control groups. The researchers surveyed teachers and children before the program began, right after it ended, and 12 weeks after it ended. They used a well-known, validated survey instrument that asked about positive behaviors (such as adaptability, communication, leadership, social skills, and study skills) and negative behaviors (such as anxiety, depression, and attention, conduct, and learning problems). They analyzed the data using quasi-experimental methods, controlling for site variation and baseline characteristics of children.

SFK Had Significant Positive Effects

The SFK program had positive effects on virtually every domain covered by the survey. As illustrated in Figure 1, the researchers found medium to large effects (effect sizes of 0.63 to 0.80)[1] on positive behaviors. Notably, the effect sizes exceed the average effect sizes found for other after-school programs targeting similar outcomes.

As seen in Figure 2, the researchers found small to medium effects on the incidence of behavioral problems, especially attention problems, hyperactivity, and withdrawal. These effect sizes equal or exceed the average effect sizes of other programs targeting similar outcomes. There was suggestive evidence that many program effects persisted at 12-week follow-up.

Figure 1. SFK Effects on Improving Positive Behaviors Greatly Exceed Effects of Other Programs

Figure 2. SFK Effects on Reducing Negative Behaviors Equal or Exceed Effects of Other Programs

SFK May Also Effect School-Related Outcomes

The SFK program affected school-related outcomes, even though SFK is not an academic intervention. For example, the program had a large effect on improving reported study skills and medium effects on reducing learning problems and attention problems. SFK's success in improving school-related outcomes suggests that an extremely interesting follow-up study would be to examine treatment effects on grades and subsequent standardized test scores.

SFK's Success Points to More Follow-Up Studies

Because the program continues to expand not only to new communities but to different countries and settings, the researchers recommend a follow-up study to test the replicability of the SFK model in diverse contexts and target populations. They also recommend evaluation of the level 2 and level 3 courses to test whether they can support and perhaps even build on the effects achieved after the level 1 course. Future evaluations might also seek to extend the follow-up period beyond 12 weeks.

Study Contributes to Research Literature

This evaluation not only provides SFK with the data it needs to strategize, plan, and improve, but it makes a significant contribution to the research literature. The evaluation sets standards for rigorous evaluation of after-school programs and highlights a successful program model. Finally, because of its rigorous research design, it offers the first causal link between spiritual development and resilience.

Notes

  • [1] Effect sizes are calculated by dividing the estimated treatment effect by the pretest standard deviation for each scale. A positive sign is assigned to an effect size whenever the treatment group did better than the control group, and a negative sign is used whenever the control group did better. Thus, a positive effect size for a negative behavior means that the treatment group experienced a greater reduction in the behavior than did the control group.

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.