The authors evaluated the effectiveness of the Common Core — the first phase of the U.S. Army's system for developing critical thinking skills in its officer corps — and the extent of differences among course venues: distributed, resident, and mixed delivery. Future evaluation should focus on whether the best possible outcomes within venues, rather than equivalent outcomes across venues, are achieved.

Developing Army Leaders

Lessons for Teaching Critical Thinking in Distributed, Resident, and Mixed-Delivery Venues

Susan G. Straus, Michael G. Shanley, James C. Crowley, Douglas Yeung, Sarah H. Bana, Kristin J. Leuschner

ResearchPublished Mar 3, 2014

Cover: Developing Army Leaders
Download PDF
Additional Downloads

Includes Summary; Research Synopsis

The U.S. Army uses the Command and General Staff Officer Course (CGSOC) as a key component of its system for developing critical thinking skills and abilities in its officer corps. The Common Core is the first phase of CGSOC. The Common Core is taught in three venues: a resident course taught at Fort Leavenworth and at satellite campuses; Advanced Distributed Learning (ADL), a web-based, self-paced course that uses interactive multimedia instruction; and The Army School System (TASS), primarily for Reserve Component officers, which combines resident and interactive multimedia instruction and is taught by the U.S. Army Reserve Command's 97th Brigade and its three subordinate battalions. CGSOC consists of nine blocks of instruction taught as stand-alone modules in the resident course (14–16 weeks long) and organized into three phases in TASS and ADL (designed to be taken over a period of up to 18 months). In response to the interests of Army leadership, this study sought to answer the following questions about the Common Core, focusing on the 2009–2010 academic year: Based on current methods of evaluation, how effective is the Common Core, and to what extent are there differences among distributed, resident, and mixed-delivery venues? Based on current measures, how can course delivery be improved? How well do current methods of evaluation gauge course success and point to needed improvements? To answer these questions, the authors analyzed available data from Command and General Staff School, including responses to student surveys, grades on assignments, and student characteristics. In addition, the authors conducted a quasi-experimental study to assess consistency in grading among faculty members.

Key Takeaways

Students in All Venues Were Generally Satisfied with the Common Core — The First Phase of the U.S. Army's System for Developing Critical Thinking Skills in Its Officer Corps

  • Students were generally satisfied with most aspects of the course in all venues.
  • However, students engaged in online instruction were less satisfied with opportunities for performance feedback and wanted more peer interaction.
  • Students reported technical and administrative problems with some aspects of online instructional delivery.

Student Grades Were High Across All Venues; Grading Needs Improvement

  • There were no meaningful differences in grades across venues.
  • The average scores on assignments suggest that there was leniency in grading.
  • The authors conducted an exploratory, quasi-experimental study to assess the consistency of grading among faculty on four assignments. Although the number of faculty who participated in the study was small, responses of those who did participate showed that reliability across these graders was generally quite low.

Future Evaluation Should Focus on Whether the Best Possible Outcomes Within Venues, Rather Than Equivalent Outcomes Across Venues, Are Achieved

  • The authors found few meaningful differences in students' self-assessed learning and course grades among the three venues.
  • These results should not be interpreted to mean that the venues are equally effective or that differences do not exist.
  • Although the learning goals are the same, the venues are designed to support different student environments and needs.

Recommendations

The Command and General Staff School (CGSS) should continue to examine grading practices and, if needed, provide professional development to ensure that faculty grade to a common standard.

CGSS should investigate the use of blended learning strategies, such as those used in their Advanced Operations Course, to provide Advanced Distributed Learning students with opportunities for instructor and peer interaction.

CGSS should address technical issues with access to and functions of online courseware. The demands of Phase II on The Army School System students also warrant further investigation, along with consideration of mitigation strategies such as changes in course structure.

Surveys used in the Common Core should be modified to obtain more diagnostic information:

  • Assess self-efficacy questions before and after training in combination with individual-level explanatory variables to assess training effectiveness.
  • Move from five- to six-point response options for close-ended questions to increase variability in responses.
  • Analyze responses to open-ended questions to better understand students' experiences in the course and to improve future surveys.

Use web analytics, such as time spent on online lessons, to assess student behavior in the course and its association with performance in training.

CGSS should survey course graduates and their supervisors to assess the impact of training on subsequent job performance.

Topics

Document Details

Citation

Chicago Manual of Style

Straus, Susan G., Michael G. Shanley, James C. Crowley, Douglas Yeung, Sarah H. Bana, and Kristin J. Leuschner, Developing Army Leaders: Lessons for Teaching Critical Thinking in Distributed, Resident, and Mixed-Delivery Venues. Santa Monica, CA: RAND Corporation, 2014. https://www.rand.org/pubs/research_reports/RR321.html.
BibTeX RIS

Research conducted by

The research described in this report was sponsored by the United States Army and conducted by the RAND Arroyo Center.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.