Evaluating Innovative Leader Development in the U.S. Army

Susan G. Straus, Michael G. Shanley, Carra S. Sims, Bryan W. Hallmark, Anna Rosefsky Saavedra, Stoney Trent, Sean Duggan

Research SummaryPublished Nov 19, 2014

Key Findings

  1. Students were extremely satisfied with the course structure, content, and delivery, attributing learning largely to course content (adaptability) and the training methods.
  2. There was substantial improvement in self-efficacy and interest and an increase in the perceived need for adaptability and related leader behaviors in the students' current jobs.
  3. Students showed a substantial increase in knowledge of AWALP concepts.
  4. AWALP graduates reported substantial application of adaptive performance principles on the job, especially in coaching, training, delegating to subordinates, and seeking subordinate input.
  5. AWALP, supported by systematic course evaluation, provides a promising approach for training as the Army seeks to further develop adaptable leaders and teams.

The U.S. Army's shift from a doctrine of "command and control" to "mission command" calls for adaptable soldiers and leaders — individuals who can rapidly recognize changes in the environment, identify critical elements in unfamiliar situations with less-than-perfect information, and facilitate timely action to meet new requirements, all while under considerable stress. The principles of mission command also emphasize leaders who value a bottom-up rather than a top-down approach and who develop teams that can anticipate and manage transition.

In this context, the Army's Asymmetric Warfare Group (AWG) designed and implemented the Asymmetric Warfare Adaptive Leader Program (AWALP) — a ten-day course to enhance adaptability in leaders and promote innovative solutions in training for and conducting unified land operations. The course design is based on a theory of adaptive performance that posits eight dimensions of such performance, such as solving problems creatively, dealing with changing or ambiguous situations, interpersonal adaptability, cultural adaptability, and making decisions under stress. Although much Army training focuses on standardized procedures for accomplishing tasks, AWALP places more emphasis on an outcomes approach, focusing on the results the commander intends to achieve. This approach encourages trainees to take the initiative to adjust actions to adapt to the situation, which, in turn, requires independent thinking and problem solving. Thus, AWALP exemplifies mission command principles for both course content and how the course is taught.

RAND Arroyo Center evaluated AWALP (using data from 104 students enrolled in three AWALP courses in 2013), addressing multiple aspects of individual and team adaptive performance and identifying potential areas for improvement in AWALP curriculum and delivery. The RAND Arroyo Center team also provided a set of instruments and protocols to foster ongoing assessment and improvement in AWALP and in other courses or events that include adaptability training.

How Effective Is AWALP?

The evaluation used multiple measures of adaptive performance — including piloting new ones — to assess a range of training outcomes. The evaluation assessed reactions to the course — satisfaction with course content, design, and delivery. It also assessed changes in attitudes and learning using pretraining and posttraining surveys and tests, respectively. Researchers also developed a measure to assess adaptability at the team level for practical exercises, using both the students and guides as raters of team adaptive performance. The measure required raters to assess whether the dimensions of adaptive performance were required in each of three exercises and to rate the team's performance on the dimensions. Finally, the evaluation assessed perceived transfer performance — how training results in payoffs to the organization — by conducting telephone interviews with AWALP graduates and their supervisors to assess how graduates apply AWALP principles on the job and the longer-term impact of the course on adaptability behavior and attitudes.

Overall, the results of this evaluation provide evidence of AWALP success across a range of measures. Table 1 highlights key findings by outcomes.

Table 1. Key Findings from AWALP Evaluation

Outcomes Key Findings
Reactions to AWALP
  • Students were extremely satisfied with the course structure, content, and delivery, attributing learning largely to course content (adaptive performance) and the training methods, which differ substantially from typical Army training. All students reported that they would recommend the course to others.
  • 30 percent of students anticipated that leader buy-in and 20 percent anticipated that command climate would pose challenges to applying adaptability concepts on the job.
Attitudes Toward Adaptability
  • There was substantial improvement in self-efficacy for, and interest in, being adaptable, even after accounting for students' individual characteristics associated with adaptive performance.
  • Results suggest an increase in the perceived need for adaptive performance in students' current jobs. Thus, students not only sensed a change in their own capabilities and interests about adaptability but appeared to see their work contexts in a different light.
Knowledge About Course Concepts
  • Students showed increased knowledge of AWALP concepts, with average scores on a multiple-choice test of 60 percent correct pretraining and 76 percent correct posttraining.
  • AWALP was successful at fostering knowledge gain for a wide range of students.
Team Adaptability
  • Students' and guides' ratings of requirements for adaptive performance generally were similar to each other across the three exercises. Evidence for improved student accuracy in ratings of requirements over time was mixed.
  • Both students and guides gave favorable ratings of team performance, but students gave higher ratings and seemed to become more confident about the level of team adaptability as the course progressed.
Perceived Transfer of Training
  • Graduates reported substantial application of AWALP principles, especially in coaching, training, delegating to subordinates, and seeking subordinate input.
  • While AWALP principles were successfully disseminated to graduates' subordinates, dissemination was somewhat less successful to peers, commanders, or throughout the unit.
  • Graduates remained positive about AWALP training but found command climate and entrenched leadership the biggest obstacles to applying AWALP principles.
  • Graduates' supervisors were positive about the effect of AWALP on graduates, but interpreting results requires caution because the sample size was small.

What Are Some Recommendations and Lessons Going Forward?

Students had few recommended improvements to AWALP, but other evaluation results point to ways to improve AWALP and the future evaluation of the course. Table 2 summarizes key recommendations.

Table 2. Key Recommendations

Areas Key Recommendations
Improvements to AWALP
  • Put more emphasis in the curriculum on responding to potential challenges in implementing adaptive performance principles on the job.
  • Provide additional feedback to students about individual and team adaptive performance.
  • Integrate additional concepts about team adaptability into the curriculum.
Ongoing Evaluation of AWALP
  • Continue to administer the knowledge test, questions about attitudes toward adaptability and related leader behaviors, reactions to the course, and ratings of teams but make some changes to reduce the burden on respondents.
  • Assess additional aspects of team performance associated with adaptive performance.
  • Continue to assess training transfer through interviews of course graduates to assess how they are using AWALP principles and to identify lessons learned to improve the course.
Future Evaluation of AWALP
  • Conduct systematic behavioral observations during the course to further evaluate individual and team adaptive performance.
  • Assess transfer of training by measuring the association of individual performance in training with subsequent job performance, where graduates' supervisors would provide independent, quantifiable ratings of job performance.
  • Modify recruiting processes for supervisors to improve success in obtaining their feedback.

For related efforts by the U.S. Army's Training and Doctrine Command (TRADOC) to train adaptability, the RAND Arroyo Center team proposed two options for expanding the AWALP approach. One option is to increase the number of students receiving training, either by increasing throughput in the current course or by continuing AWG's efforts to stand up local versions of AWALP by training local trainers through mobile training teams. Another way to disseminate AWALP more broadly is to incorporate adaptability principles into existing professional military education courses, such as advanced leader courses for noncommissioned officers. And while AWALP provides a starting point for training soldiers to work in and lead teams, TRADOC can support mission command principles further by creating a follow-on course that expands instruction at the team level, addressing such topics as shared mental models, transactive memory systems, team trust, and team facilitation.

Conclusion

The shift in Army doctrine from command and control to mission command calls for profound changes in leader and team conduct, with a concomitant transformation in training. AWG's successful development and implementation of AWALP exemplifies mission command principles in terms of both the content of the course and how it is taught. AWALP, supported by systematic course evaluation, provides a promising approach for the Army as it seeks to further develop adaptable leaders and teams.

Available for Download

Topics

Document Details

Citation

RAND Style Manual
Straus, Susan G., Michael G. Shanley, Carra S. Sims, Bryan W. Hallmark, Anna Rosefsky Saavedra, Stoney Trent, and Sean Duggan, Evaluating Innovative Leader Development in the U.S. Army, RAND Corporation, RB-9797-A, 2014. As of September 20, 2024: https://www.rand.org/pubs/research_briefs/RB9797.html
Chicago Manual of Style
Straus, Susan G., Michael G. Shanley, Carra S. Sims, Bryan W. Hallmark, Anna Rosefsky Saavedra, Stoney Trent, and Sean Duggan, Evaluating Innovative Leader Development in the U.S. Army. Santa Monica, CA: RAND Corporation, 2014. https://www.rand.org/pubs/research_briefs/RB9797.html.
BibTeX RIS

Research conducted by

This publication is part of the RAND research brief series. Research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.