In this resource we provide an example of a streamlined Getting To Outcomes (GTO) process applied to a sexual assault prevention activity (SAPA), Green Dot, being planned by a sexual assault prevention GTO implementation team (GTO team) operating out of a fictional setting, Joint Base (JB) Hensonburg. JB Hensonburg is a large base in the United States that trains young men and women and is in a rural setting. Entertainment in the adjacent community is largely limited to bars and other drinking establishments. Groups of service members under the legal drinking age sometimes rent hotel rooms on the weekend to host parties and drink alcohol in private, away from the barracks, where alcohol use is prohibited.
In recent years, installation Sexual Assault Response Coordinators (SARCs) have noticed that many of the reports of sexual assaults against young service members have occurred at off-site, private residences and hotels. In addition, a recent RAND report showed that JB Hensonburg had more sexual assault reports in the past year than other similarly sized sites. The GTO team at JB Hensonburg has been working to learn more about the problems and identify new prevention strategies to better educate and prepare their population to intervene as bystanders and prevent further incidents.
The GTO team includes the site SARC, the Headquarters (HQ) Sexual Assault Prevention and Response (SAPR) research analyst, and an experienced facilitator who delivers the annual SAPR training. The examples of the GTO process in this document follow the GTO team’s work of using the GTO Handbook to consider and then select, plan, and evaluate Green Dot, a bystander training program developed by Dorothy Edwards and her colleagues. Green Dot was selected for implementation at JB Hensonburg after assessment of other options, using the GTO process, concluded that they were not appropriate.
Green Dot is a bystander intervention program that seeks to prevent sexual assaults by combating social norms that condone violence and by increasing the capacity of individuals to recognize high-risk situations that could lead to sexual assault and intervene safely to prevent the situation from escalating. The program consists of a four-hour interactive bystander training for socially influential service members, a 90-minute bystander workshop for leaders, and a 60-minute bystander workshop for all other service members and civilians on an installation, all led by trained facilitators. Additionally, Green Dot implementation includes a targeted social marketing strategy, skill reinforcement and strengthening activities (including booster sessions), and community mobilizing initiatives throughout the year. For more information, visit Green Dot.
Although the implementation team and JB Hensonburg are fictional, the remaining details are as realistic as possible. That is, the information about existing data sources, risk and protective factors, SAPA, and the evidence supporting them are correct, to the best of our knowledge.
See Getting To Outcomes® Handbook for Strengthening Sexual Assault Prevention Activities in the Military for summary information about using GTO to plan, implement, and evaluate a SAPA in the military.
GTO Step 1 Process Example
The JB Hensonburg GTO team begins at GTO Step 1: a problems and resources assessment of their community and target population to identify relevant problems, resources, and gaps. First, they gather information on the problems and needs in their community, using some of the sources listed in Prevalence of Sexual Assault in the Military: Risk and Protective Factors, Data Sources, and Data Uses. O-4 Gribble, the JB Hensonburg Sexual Assault Response Coordinator (SARC) and a GTO team member, volunteers to review the data and readily identifies several sources of Department of Defense (DoD) data from Prevalence of Sexual Assault in the Military: Risk and Protective Factors, Data Sources, and Data Uses that can help the GTO team understand the scope of the problem, including the Defense Equal Opportunity Management Institute Organizational Climate Survey, which has site-level data.
O-4 Gribble documents that, DoD-wide, 6.2 percent of servicewomen and 0.7 percent of servicemen had experiences that met the DoD definition of sexual assault in the past year (2018 Workplace and Gender Relations Survey of Active Duty Members [WGRA]; Breslin et al., 2019). She notes that the risk of sexual assault is not spread equally across the force. Junior enlisted service members (E-1–E-4) are at higher risk than the general population; 9.1 percent of junior enlisted women and 0.9 percent of junior enlisted men were sexually assaulted in the past year (2018 WGRA; Breslin et al., 2019). The GTO team pays close attention to these data. JB Hensonburg hosts occupational training for a number of military occupations, and the team notes that these early-career, high-risk service members could benefit from additional prevention activities.
She also consults the RAND Sexual Assault and Sexual Harassment in the U.S. Military report to learn the specific risk for her base (Morral et al., 2018a; Morral et al., 2018b). According to the analyses presented in the report, the risk of sexual assault at JB Hensonburg was higher in 2014 than the sexual assault risk at some other bases. From the analysis, she learns that the additional risk was explained entirely by measured demographics and service history characteristics of the service members stationed there. That is, even though JB Hensonburg looks like it has a high rate of sexual assault, it probably has to do with the high proportion of junior enlisted service members and the gender imbalance in some of the occupations stationed at JB Hensonburg, rather than that anything that is "wrong" with JB Hensonburg’s leadership or culture relative to other sites. She makes a note that this will be important to clarify for leadership as well when she briefs them on the GTO team’s progress.
She also notices an interesting pattern in the 2016 report from the WGRA. When service members were asked whether their military peers and leaders corrected incidents of sexual harassment and rejected sexual assault, their perceptions varied based on the pay grade they were rating. Most service members (72–85 percent) thought that senior enlisted personnel and senior officers would "recognize and correct incidents of sexual harassment," but only 54–67 percent thought that junior enlisted personnel would do so (Peebles, Grifka, and Davis, 2017). Similarly, although most service members (87–93 percent) believed that senior enlisted personnel and senior officers made "it clear that sexual assault has no place in the military," only 61–70 percent believed that junior enlisted personnel did so (Peebles, Grifka, and Davis, 2017). Finally, even though most service members intervened when they saw a high-risk situation for sexual assault (88 percent), few had observed any high-risk situations in the past year (29 percent).
The GTO team wonders whether younger service members need a higher "dose" of the SAPA. By late career, it appears that most service members have embraced the military rejection of sexual harassment and sexual assault, but it also appears that this value has not yet been fully accepted by younger service members. Can the GTO team speed this process up for younger service members?
Next, the JB Hensonburg GTO team reviews existing prevention activities that address sexual assault. Through formal and informal sources, members of the team identify three practices currently implemented on the base that target sexual assault. They document the practices: (1) annual sexual assault prevention training delivered by the site SAPR office, (2) response services delivered by the site SAPR office, and (3) an awareness-raising "color run" 5K race sponsored by the training command.
Based on the activities reviewed, the JB Hensonburg GTO team believes that, although steps are being taken to address sexual assault at JB Hensonburg, gaps remain. All service members currently receive basic education about sexual assault and have access to response services if they are victimized. The color run 5K appears to be a well-liked awareness raising activity, but the GTO team suspects that watching or participating in a race will not prevent any future sexual assaults from occurring. The team feels confident that a prevention activity to target early-career enlisted service members for an additional and higher dose of training would not be duplicative of other efforts on the base and should be the priority of their prevention activity. Sexual assault risk is higher for junior enlisted service members, who make up most of the JB Hensonburg population, and junior enlisted service members have a limited ability to recognize situations that increase risk for sexual assault and intervene to reduce risk.
See Step 1GTO Step 2 Process Example
As reviewed in Step 1, the JB Hensonburg GTO team has decided to focus on a prevention activity for junior enlisted service members who
- are high risk for committing sexual assault
- might not recognize or intervene in situations that increase risk for sexual assault.
As part of the problem assessment, the GTO team reviewed existing surveys (see Prevalence of Sexual Assault in the Military: Risk and Protective Factors, Data Sources, and Data Uses) for examples of data sources on sexual assault prevalence and risk and protective factors), that assess sexual assault and bystander behaviors. From the problems they identified, they first agreed on three goals for their initiative (see GTO Step 2). O-4 Kittur, a GTO team member, agrees to lead the effort to select measurable outcomes based on the existing surveys and identify their pros and cons.
Based on O-4 Kittur's review of data sources, the GTO team selects three questions that have been used in the WGRA and Workplace and Gender Relations Survey of Reserve Component Members and three bystander scales that will help them to assess progress toward their goals.
By formalizing their goals, the team is better equipped to begin outreach to site leadership to share the team's goals and desired outcomes. O-5 Lomen, the GTO team chair, schedules short meetings with key site leaders to share the team's goals to improve sexual assault prevention among junior enlisted service members by increasing their ability to recognize high-risk situations, maintaining the high likelihood of intervening once a situation has been recognized as high risk, and shifting the social norms toward expecting bystander intervention even among junior enlisted service members. These meetings will be used to obtain feedback about the goals and direction of the initiative, solicit buy-in for the team's plans, and learn more about any similar prevention activities with which the team will want to coordinate. The next step will be to move forward with selection of the best prevention activity to achieve their goals.
See Step 2GTO Step 3 Process Example
In GTO Step 3, the GTO team at JB Hensonburg plans to consider (1) the best available research evidence and (2) practitioner expertise and other available resources to help them identify the best candidates among the possible options. O-3 Rate volunteers to lead the effort and begins by searching for a list of evidence-based activities for sexual assault prevention maintained by registries. Types of Sexual Assault Prevention Activities in the Military: Finding and Assessing Effective Prevention Activities lists sources of information on several types of evidence-based activities.
Three of the clearinghouses that have reviewed SAPA prove particularly useful. Not only do these sites provide useful information about the level of evidence supporting the prevention activity, but they also summarize each prevention activity's content, goals, and evaluation studies. The reference list for each prevention activity also provides next steps to learn more about a selected prevention activity. O-3 rate records the evidence-base categorizations from (1) the Penn State Clearinghouse for Military Family Readiness, (2) the National Institute of Justice's Crime Solutions clearinghouse, and (3) the Culture of Respect summary of sexual assault prevention for universities. Because of small differences in the criteria for inclusion, the recommendations across clearinghouses are not identical. However, O-3 Rate notices that all three clearinghouses recommend two bystander interventions: Bringing in the Bystander and Green Dot.
She identifies evidence in scientific articles and other reports by searching for review articles in Google Scholar using the search terms "review" AND "prevention" AND ("sexual assault" OR "sexual violence" OR "sexual aggression" OR "rape") searching for evaluation studies in Google Scholar using the search terms ("evaluation" or "intervention" or "program") AND "prevention" AND ("sexual assault" OR "sexual violence" OR "sexual aggression" OR "rape") searching the gray literature (literature published outside of a commercial publisher) using internet search tools to find government and nonprofit reports.
O-3 Rate finds many evaluations of SAPA. She notes that most of them are evaluated with college students or high school students and finds few evaluations with military members. Although the match is not perfect, she appreciates that college students share many similarities with the junior enlisted service members with whom the GTO team plans to work. Both groups are young adults who were successful in high school and typically are away from home and parental monitoring for the first time.
O-3 Rate records what she finds. A bystander approach seems like the most promising evidence-based match to the implementation team's goals, and both Bringing in the Bystander and Green Dot seem like strong prevention activities. After working through GTO Step 3, the GTO team moves on to the next step in the GTO process—to assess the fit of these two options for JB Hensonburg.
See Step 3GTO Step 4 Process Example
During GTO Step 4, the GTO team reviews materials for Bringing in the Bystander and Green Dot to assess their fit. It will be important that the selected prevention activity be suitable for young adults who are just beginning their military careers, living away from family for the first time (aside from basic training), and mostly unmarried.
The implementation team’s process of evaluating the fit of the two prevention activities selected in Step 3, enables a careful comparison of both. They decide that Green Dot is a better fit with the military culture of JB Hensonburg and requires less adaptation to be compatible with its culture and norms. Bringing in the Bystander would require the creation of significant new content for the military target population. This exercise has increased the team's confidence that Green Dot will be a good fit for JB Hensonburg.
The Air Force adaptation of Green Dot has already made many of the cultural changes necessary to better match a military setting (for example, intervention options must take into account pay grade and chain of command). The GTO team notes that Air Force–specific material could prove distracting to service members at JB Hensonburg. Based on their previous experience with training curricula, prevention activity materials are sometimes rejected as irrelevant for such reasons as outdated uniforms in images and failure to use service-specific language for ranks. Because JB Hensonburg is a joint base, the GTO team knows that they will need to develop two sets of materials, an adaptation for each of the two service branches that make up the base. The JB Hensonburg GTO team is now ready to move forward to Step 5 of the GTO process. In this step, they will examine current prevention activity readiness to assess whether they have the resources they need to implement Green Dot well.
See Step 4GTO Step 5 Process Example
In GTO Step 5, the JB Hensonburg GTO team considers whether they have the capacity necessary to deliver Green Dot as it was intended. To make this determination, they systematically consider the five types of capacity:
Staff Capacities
With regard to staff capacities, the GTO team plans to staff the pilot with two of its own members. The team noted that O-4 Simmons is a licensed social worker who has delivered group trainings to service members for the past ten years. Given his background, no additional communication or sensitivity training will be necessary, and he is certainly well versed in military culture. Through his participation in the implementation team, he is familiar with Green Dot, but he has not been formally trained to facilitate the group sessions. He will reach out to the developers of Green Dot to obtain the necessary training. The GTO team has agreed to support travel for two trainers. O-3 Rate has an undergraduate degree in psychology. She took several courses on research methods and statistics and volunteered in a professor’s research lab while she was a student. In addition to taking the Green Dot training and serving as the second facilitator, O-3 Rate will be responsible for obtaining, entering, and analyzing the baseline surveys and the three-month follow-up surveys assessing bystander readiness.
The GTO team is confident that O-4 Simmons and O-3 Rate will have the capacity to deliver and assess a small pilot of Green Dot at JB Hensonburg. GTO team members have all volunteered to serve as backup during this critical testing period. However, after the pilot is completed, if the team decides to roll Green Dot out across JB Hensonburg, the team plans to revisit staffing. The developers of Green Dot recommend larger implementation teams when the prevention activity is rolled out with the saturation necessary to influence the climate of large group.
The GTO team plans to conduct the pilot test using a volunteer occupational training group and feels confident that they will find a training group to participate and obtain needed permission from its leadership.
Leadership Capacities
In considering leadership capacities, the team has identified a leadership vacuum for the second phase, in which all training groups will be encouraged to participate. To scale up Green Dot for all of JB Hensonburg, it will be necessary to obtain the support and permission of mission and occupational training group leadership. If members of an occupational training group are tasked to attend Green Dot but receive inconsistent messaging from direct leaders who might not support or take the effort seriously, the impact of Green Dot will be stunted. The implementation team understands that Green Dot was intended for use across an organization with targeted training for leadership, and, therefore, they will need to socialize and obtain support prior to any further rollout. They hope that preliminary results from their pilot will help them to "sell" the prevention activity to site leadership.
Given the critical role that leadership will play, the GTO team begins to plan for phase 2. They will schedule meetings and briefings with leaders with the goals of socializing Green Dot and identifying a well-regarded champion with influence across multiple levels of leadership who has a strong interest in positive approaches to reducing sexual assault among junior enlisted service members. Once the champion is identified, the team and the champion will work together to create a plan to rally both personnel and leaders to prioritize the initiative.
The team plans to coordinate with the military Green Dot implementation team, the JB Hensonburg SAPR office, and commanders for all occupational training groups to ensure situational awareness of sexual assault prevention efforts and to avoid duplication. The annual sexual assault prevention training delivered by SAPR contains some similar material. The GTO team plans to discuss a possible opt-out for enlisted service members who complete Green Dot in a given fiscal year (with the plan that they would attend the SAPR annual training the next year and in subsequent years).
Technical Capacities
In considering technical capacities, the team documents that the necessary technical capacities for the Green Dot trainings are modest. Meeting rooms for groups of 20–30 service members can be reserved in a building near each workgroup’s usual work location. For meeting rooms without a computer and projector, the GTO team will plan to procure a portable setup.
Financial and Resource Capacities
The out-of-pocket expenses needed for the GTO team to pilot Green Dot are modest. Travel for two facilitators to obtain Green Dot training will be supported by the GTO team, and minimal material costs are needed for copies and office supplies.
The cost of Green Dot will be driven largely by personnel time, which the GTO team is not responsible for covering during the pilot phase. Site leadership has already agreed to task two facilitators to deliver the Green Dot pilot. Both Green Dot facilitators will invest 50 percent of their time in the pilot for four months. In addition, each occupational training group that attends Green Dot will be investing labor hours in sexual assault prevention rather than in day-to-day tasks or other training. The GTO team will rely on identifying a volunteer occupational training group whose leadership sees the value of preventing sexual assault and is therefore willing to invest personnel time in the effort.
Collaboration and Partnership Capacities
Collaboration and partnership will be important, and outreach is planned with SAPR and Air Force site personnel who have Green Dot experience, as well as leadership at multiple levels.
After completing GTO Step 5, the GTO team feels even more confident in their capacity to conduct a pilot of Green Dot sessions at JB Hensonburg. The process of assessing each capacity systematically provided a structure that has allowed them to set aside issues that have been addressed and focus their efforts on the few remaining capacity gaps.
Over the next months, they will invest their time in
- identifying a prevention activity champion and increasing leadership support for Green Dot
- obtaining Green Dot facilitator training for O-4 Simmons and O-3 Rate
- gathering necessary Green Dot materials and evaluation tools
- identifying workgroups who are willing to participate in the pilot.
Moving into GTO Step 6, they will develop an implementation and evaluation plan.
See Step 5GTO Step 6 Process Example
Given that Green Dot was developed for a similar demographic and was more recently adapted for the Air Force, the GTO team is pleased with the out-of-the-box fit for junior enlisted service members assigned to JB Hensonburg. Green Dot was created for young adults, newly separated from their nuclear families, who are unmarried and might be socially active and dating, which is a good match to the planned target population at JB Hensonburg. The plan is to deliver the prevention activity to enlisted service members who are just beginning their military careers. In other words, many prevention activity participants will still be completing occupational training and subject to the dense schedules dictated by the training command. The GTO team understands that they will need to immediately reach out to training leaders and work closely with them to identify dates and times when trainees could attend Green Dot sessions.
Having completed GTO Steps 1–5, the GTO team is now ready to develop the operations of their selected prevention activity. In GTO Step 6, they make a Prevention Activity Work Plan Tool (see Figure 1) to create a detailed plan for running the prevention activity.
Using this tool in GTO Step 6 left the team feeling assured that no key Green Dot tasks had been left out. The process of identifying tasks and then assigning a responsible team member improved the efficiency with which the team completed preparation tasks. O-5 Lomen, the GTO team chair, particularly appreciated the opportunity to match the skills, expertise, and interests of group members to implementation tasks. As shown in the Prevention Activity Work Plan Tool, planning for specific components of implementation was largely overseen by the group member who would ultimately be responsible for the task when Green Dot rolls out.
The GTO team also outlines their expected costs. Most Green Dot costs are tied up in personnel time, either the time of the facilitators or in the hours that service members spend to attend Green Dot. Currently, the GTO team is not obligated to produce these dollars from their own budget.
Having already briefed leadership early in the GTO process on the value of sexual assault prevention and its relevance to mission readiness, the GTO team received leadership support for their decision to implement a robust SAPA. The GTO team has also sought leadership feedback and made adjustments as necessary throughout the decisionmaking process. They also know that their recommendation to implement a bystander intervention will not come as a surprise to leadership because leadership has been aware of the problems and goals from the beginning. The GTO team also has a better understanding of leadership’s concerns and will use that knowledge to better tailor their briefing to their audience. Therefore, they expect that commanders will permit the effort as mission relevant and cover it with the general budget.
Figure 1: Example Prevention Activity Work Plan Tool
| Example Prevention Activity Work Plan Tool | |||
| Completed by GTO team. May 23rd 2018. Prevention activity: Green Dot | |||
| Tasks: Administrative | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Prepare budget (see Prevention Activity Budget Tool | June 2019 | MR. Stubbe | 15 May 2018 |
| Acquire curriculum and materials, including evaluation materials | June 2019 | O-3 Rate | |
| Set preferred implementation dates | June 2019 | O-5 Lomen | |
| Tasks: Leadership Engagement | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Senior leadership (O-6 to O-8) breifings, e.g.: Briefing 1 (prior to implementation), Briefing 2 (evaluation results) | O-5 Lomen | ||
| Following receipt of long-term outcomes | O-5 Lomen | ||
| Other leadership (O-4 to O-6) breifings, e.g.: Briefing 1 (problems and goals), Briefing 2 (implementation plan), Briefing 3 (evaluation results) | June 2018 | O-5 Lomen | |
| June 2019 | O-5 Lomen | ||
| Following receipt of long-term outcomes | O-5 Lomen | ||
| Tasks: Policies and Procedures | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Obtain required permissions and draft necessary taskers | July 2019 | O-5 Lomen | |
| Tasks: Preparation | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Identify a local Green Dot champion | June 2019 | O-5 Lomen | |
| Conduct outreach to dvelop site community support | July-August 2019 | O-5 Lomen and Mr. Stubbe | |
| Meet with SAPR office to ensure situational awareness | August 2019 | O-5 Lomen | |
| Obtain facilitator training | July 2019 | O-4 Simmons and O-3 Rate | |
| Conduct a "dry run" for facilitator and assistant practice | August 2019 | Implementation team | |
| Reserve training facilities | July 2019 | O-3 Rate | |
| Prepare facilitator packets for Green Dot sessions | August 2019 | O-3 Rate | |
| Prepare participant materials (for example, worksheets) for Green Dot sessions | August 2019 | O-3 Rate | |
| Test computer and projectors at the reserved training facilities | 1 week before scheduled session | O-3 Rate | |
| Purchase refreshments for Green Dot sessions | Week of scheduled session | O-3 Rate | |
| Tasks: Recruitment (and Retention) | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Develop and test participant recruitment (and retention) plan and materials | July 2019 | O-4 Simmons and O-3 Rate | |
| Notify eligible population and solicit volunteer training group | Early august 2019 | O-5 Lomen | |
| Confirm dates, time, and space and send reminders to workgroup leader | One week prior to session | O-4 Simmons | |
| Send thank-you email to session participants and request informal feedback | Within 3 days of last program session | O-4 Simmons | |
| Tasks: Implementation | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Build detailed schedule for implementing Green Dot (where and when each part of the program will be conducted - for examplee, when and how each component of a media campaign will be rolled out) | August 2019 | Implementation team | |
| Conduct Green Dot implementation leader training | September 3, 2019 | O-4 Simmons and O-3 Rate | |
| Conduct Green Dot Intensive Bystander Training, influencer group | September 5, 2019 | O-4 Simmons and O-3 Rate | |
| Hold Green Dot Overview Talk | September 7, 2019 | O-4 Simmons and O-3 Rate | |
| Tasks: Evaluation | When Will It Be Done? (Time Frame) | Who Is Responsible? | Date Done |
| Design evaluation and complete GTO process and outcome evaluation planner tools | May/June 2019 | O-4 Kittur | 9/21/18 |
| Recruit evaluator | June 2019 | O-4 Kittur | |
| Collect data | Each session (in person) and three months after each session (by email) | O-3 Rate | |
| Enter data | Within one week of each session | O-3 Rate | |
| Analyze data | 1 month following all pilot sessions and within 1 month of receipt of all 3-month follow-up data | Evaluator | |
| Review process evaluation data from relevant data collection tools and complete GTO Step 7 Summary Tool | 1 month following receipt of pilot study post-test data | Evaluator | |
GTO team members are also sensitive that prevention activities need to be worth their costs. Ultimately, they will be expected to provide a recommendation to leadership about whether all Green Dot costs, including the largely invisible personnel costs, are offset by the benefits produced by the trainings. In an effort to consider and prepare themselves for this analysis, they prepare process and outcome evaluation plans. For assistance with evaluation planning, they refer to Using Getting To Outcomes to Plan Evaluation of Sexual Assault Prevention Activities and Process and Outcome Evaluation Measures for Sexual Assault Prevention Activities in the Military.
They summarize their evaluation measures along with their priority problems and their goals and specific desired outcomes in a SAPA Overview, like the one shown in Figure 2.
For their planned pilot study, the team decides to use attitude and confidence outcomes that can be assessed in the short follow-up period they have available. Before Green Dot sessions, they will ask attendees to complete Intent to Help Strangers and Friends and Bystander Efficacy scales (Banyard et al., 2014; Banyard, 2008). These measures, conducted before the first Green Dot session, will provide an assessment of trainees' current willingness to engage in bystander helping behaviors and their confidence that they would be able to do so. Then, after the final Green Dot session is complete, participants will fill out the Intent to Help Strangers and Friends and Bystander Efficacy scales again. By comparing the first and second surveys, the GTO team will be able to see whether there were any immediate changes in confidence and willingness to engage in bystander helping behaviors after completion of Green Dot.
Figure 2: Example of a Sexual Assault Prevention Activity Overview
| Example of a Sexual Assualt Prevention activity Overview | |||||
| What priority problem(s), challenge(s), or gaps do you want to address? (From GTO Step 1) | What are the goals you intend to reach by addressing this problem, challenge, or gap? (From GTO Step 2) | What are your specific desired outcomes that you will be able to evaluate for each goal? (From GTO Step 2; update after prevention activity selection) | What prevention activity are you using to acheive these desired outcomes? (Finalized by GTO Step 6) | How will you assess the quality of your implementation? (Process evaluation measures and methods from GTO Step 6) | How will you assess the outcomes of your prevention activity? (Outcome evaluation mesaures from GTO Step 6) |
| 1. Few service members (29 percent) recognized one or more high-risk situations for sexual assualt in the past 12 months. | Increase the number of bystanders with the skill to identify risky situations. | After participating in Green Dot, at last 75% of junior enlisted service members will rate themselves as more likely to engage in bystander helping behaviors. | Green Dot | Attendance, facilitator ratings, participant satisfaction survey | Intent to Help Strangers and Friends Scales |
| 2. Among service members who did recognize a high-risk situation for sexual assualt, most took action to reduce the risk (89 percent). | Maintain the high percentage of service members who are willing to take action once they categorize a situation as risky. | After participating in Green Dot, at least 75% junior enlisted service members will feel more confident in their ability to engage in bystander helping behaviors. | Green Dot | Attendance, facilitator ratings, participant satisfaction survey | Bystanders Efficacy Scale |
GTO Steps 7–9 Process Example
By summer 2019, the JB Hensonburg implementation team had completed the planning process for implementing and evaluating Green Dot. Over the next six months, they were focused on promoting Green Dot in the community, obtaining facilitator training, recruiting an occupational training group to participate, and, finally, implementing Green Dot. In mid-January 2020, they were ready to see and interpret the results of their process evaluation using the data they collected from each Green Dot group, using the sign-in sheets and satisfaction surveys and debriefing sessions with the implementation team.
They also analyzed the outcome data from the pre- and post-Green Dot surveys with each group and were disappointed to learn that Green Dot participation did not seem to increase attendees' intentions to help (either friends or strangers), but attendees' confidence to help dramatically improved. Even if a similar number of service members plan to intervene in high-risk situations, if the confidence of those who are willing to intervene has improved, it is possible that the prevention activity will still have a positive impact on preventing sexual assaults.
Using the Evaluation Summary and CQI Review Worksheet (Figure 3) from GTO Steps 7-9, they summarized the results and considered and decided on improvements needed to improve the implementation and outcomes. As a resource for these activities they consulted Using Evaluation Results to Improve and Sustain Sexual Assault Prevention Activities in the Military.
The results of the process evaluation helped the implementation team to make sense of the failure to achieve all the desired outcomes. Attendance was high, so the team rules out poor attendance as the source of the problem. Fidelity to the Green Dot prevention activity elements was also high; the team feels confident that service members received Green Dot as intended. However, they noticed that only half of attendees thought the prevention activity was important and only about half thought it would help to prevent sexual assaults. O-4 Simmons had informal conversations with some of the service members who had attended the training, and they expressed frustration with “all the sexual assault prevention classes” they had to take. This seemed an odd sentiment for service members who had served for less than a year, and therefore could not have participated in more than two sexual assault prevention trainings total. O-4 Simmons thought the negative attitudes might have been passed down by attendees' direct leaders. Indeed, the implementation team had noticed negative attitudes on the JB with indirect and direct negative comments passed along to them when they mentioned that they worked with the SAPR office; this also corresponded with the few feedback emails they received as part of their process evaluation.
Because of this hypothesis about the source of the problem, the implementation team reconsidered its approach. The senior mission commanders had been so positive about the Green Dot approach that they had mistakenly believed that leadership support was in place. Now they realized that, without mid-level leadership buy-in, they might never be able to effectively reach the junior enlisted service members they were trying to reach.
They decide to run a second pilot study. They will use the information they've learned from this evaluation to refine their approach. First, they will coordinate with multiple leadership levels one month before the sessions for junior enlisted service members. They will prioritize leaders who have taken on their first command role within the past two years and will implement the leadership engagement training phase of Green Dot that they had skipped during the first pilot. Second, they decide to add short feedback interviews with ten attendees to collect feedback more formally. They were grateful for the information received through O-4 Simmons' informal conversations and decide that they want to standardize the approach to make sure they get this helpful feedback again.
The process of implementing Green Dot with junior enlisted service members had gone very smoothly during the first pilot, so few changes to the actual prevention activity sessions are expected. The facilitator delivered the model with high fidelity and increased his comfort with the material, and the attendees were engaged in the process. The implementation team hopes that this experience will translate to a smooth second phase.
Although disappointing, the implementation team believes they have learned a lot from implementing Green Dot and from the results of their process and outcome evaluations. They feel confident that they have the information they need to move forward with the prevention activity and evaluation improvements.
Figure 3: Example of an Evaluation Summary and CQI Review Worksheet
| GTO Steps 7-9: Evaluation Summary and CQI Review Worksheet | |
| Completed by: (blank), Date: (blank), SAPA: (blank) | |
| Evaluation review | Response |
| How effectively did the SAPA help us reach our desired outcomes? (GTO Step 8) | |
| 1. Which if any desired outcomes were not met or not completely met? Were any of unmet outcomes critically important (i.e., must be met to justify continuing the SAPA) | Self-reported intentions to help strangers and friends did not appear to change in a meaningful way (Intent to Help Scale). |
| 2. Which desired outcomes were reached or exceeded? | There was a large shift in confidence. After finishing the Green Dot prevention activity, participants were much more confident that they would be able to intervene as a bystander than they were prior to the prevention activity (Bystander Efficacy Scale). |
| 3. Was there progress towards our long-term goals? | Improvement on one out of two desired outcomes. |
| How well did we implement this SAPA? (GTO Step 7) | |
4. How did implementation of the SAPA go in terms of the following aspects?
|
|
| 5. Which, if any, of 4a through 4f above were likely to have impacted your desired outcomes? How and why? These will be critical components to improve (or sustain) the next time you implement the P4 | Implementation team believes that improving 4f is critical for success |
| Determination based on evaluation results | |
| 6. Are our process and outcome evaluation results overall good enough to continue implementing the SAPA? If yes, what adjustments, if any, will be critical to its improvement? If no, how and why was this decision made? Which missed outcomes were most important to this decision? | Yes. But must improve leadership support among direct supervisors of the target population. |
| CQI Review | Response |
| 7. Was the problem the right one to be addressing with our SAPA? (GTO Step 1) Was there any improvement in the overall trend of the priority problem? Review the original problem data from Step 1 and compare with any newer updates to the data, if available. Have the needs changed or remained? Are there new priorities now that should be addressed instead? | Problem remains. Too soon to expect improvement in the overall trend. No major change or shift in priorities. |
| 8. Do we need to change goals and desired outcomes or potential participants? (GTO Step 2) Target different conditions or behaviors? Reset benchmarks up or down? | No need to change goals and desired outcomes. Will repeat the pilot with the same benchmarks and renewed outreach to gain mid-level leadership support. |
| 9. Should we consider a different SAPA? (GTO Step 3) Or are there other improvements we need to make? | No. Confidence to intervene improved and even if the same number as pre Green Dot intend to intervene, improved efficacy could reduce sexual assaults. |
| 10. Do we need to improve the SAPA's philosophical and logistical fit to our site, community, and participants? (GTO Step 4) If not, why not? What adaptations could be made? Were any adaptions made? How did that go? | Yes. Engage multiple levels of leadership and, most importantly, the leaders with whom junior enlisted service members have the most day-to-day contact. |
GTO Step 10 Process Example
In May 2020, the JB Hensonburg implementation team meets to review their efforts to date and begin planning for the next year. Because of the outcome evaluation showing increased confidence to intervene but no improvement in intentions to engage in positive bystander behaviors, they agree that it is not appropriate to expand Green Dot to the whole installation. However, as noted in Step 9, they have decided to test Green Dot a second time. For their second pilot study of Green Dot, they will prioritize leadership training for those leaders who have the most day-to-day contact with junior enlisted service members. By implementing the leadership engagement training phase of Green Dot, the implementation team hopes to increase positive attitudes toward Green Dot and positive role models for bystander behaviors that will then trickle down to junior enlisted service members.
The iterative process built into GTO makes it straightforward to cycle back to Step 1 and begin the process anew. During this second cycle, the time investment will be lower because many decisions can remain in place. For example, the prevention activity selection, process evaluation, and outcome evaluation will remain similar.
The team records their decisionmaking process in the Evaluation Summary and CQI Review Worksheet. O-4 Gribble, O-4 Kittur, and O-3 Rate volunteer to review and update the work they completed for GTO Steps 1–3, and the implementation team schedules a follow-up meeting to review their changes. Although they had hoped that Green Dot would be effective on the first try, they are glad that they followed a careful GTO process. This allowed them to quickly pinpoint a hypothesis for why the first pilot was not successful, design a possible solution, and get to work piloting that solution. The team is thankful that resources were not invested in blindly scaling up a Green Dot prevention activity, and they remain committed to refining their chosen prevention activity so that it will successfully reduce sexual assault at JB Hensonburg.
See Step 10References
- Banyard, V. L., "Measurement and Correlates of Pro-Social Bystander Behavior: The Case of Interpersonal Violence," Violence and Victims, Vol. 23, 2008, pp. 83–97.
- Banyard, Victoria L., Mary M. Moynihan, Alison C. Cares, and Rebecca Warner, "How Do We Know If It Works? Measuring Outcomes in Bystander-Focused Abuse Prevention on Campuses," Psychology of Violence, Vol. 4, No. 1, 2014, pp. 101–115.
- Breslin, Rachel A., Lisa Davis, Kimberly Hylton, Ariel Hill, William Klauberg, Mark Petusky, and Ashlea Klahr, 2018 Workplace and Gender Relations of Active Duty Members: Overview Report, Alexandria, Va.: Office of People Analytics, U.S. Department of Defense, OPA Report No. 2019-027, May 2019.
- Morral, Andrew R., Terry L. Schell, Matthew Cefalu, Jessica Hwang, and Andrew Gelman, Sexual Assault and Sexual Harassment in the U.S. Military: Annex to Volume 5, Tabular Results from the 2014 RAND Military Workplace Study for Installation- and Command-Level Risk of Sexual Assault and Sexual Harassment, Santa Monica, Calif.: RAND Corporation, RR-870/8-OSD, 2018a. As of October 26, 2021: https://www.rand.org/pubs/research_reports/RR870z8.html
- Morral, Andrew R., Terry L. Schell, Matthew Cefalu, Jessica Hwang, and Andrew Gelman, Sexual Assault and Sexual Harassment in the U.S. Military: Volume 5, Estimates for Installation- and Command-Level Risk of Sexual Assault and Sexual Harassment from the 2014 RAND Military Workplace Study, Santa Monica, Calif.: RAND Corporation, RR-870/7-OSD, 2018b. As of October 26, 2021: https://www.rand.org/pubs/research_reports/RR870z7.html
- Peebles, H., A. Grifka, and L. Davis, "Military Workplace Climate," in Lisa Davis, Amanda Grifka, Kristin Williams, and Margaret Coffey, eds., Military Workplace Climate: 2016 Workplace and Gender Relations Survey of Active Duty Members Overview Report, Alexandria, Va.: U.S. Department of Defense Office of People Analytics, 2017, pp. 231–284. As of October 23, 2021: https://www.sapr.mil/public/docs/reports/FY16_Annual/Annex_1_2016_WGRA_Report.pdf