An Assessment of the Ability of the U.S. Department of Defense and the Services to Measure and Track Language and Culture Training and Capabilities Among General Purpose Forces
Oct 17, 2012
RAND Corporation and MITRE Corporation researchers examined how U.S. Department of Defense (DoD) language, regional expertise, and culture (LREC) training and capabilities relate to overall unit readiness and mission accomplishment, the existing means of tracking LREC skills among general purpose forces, the extent to which current tracking reflects LREC unit readiness, and ways DoD can improve tracking and measurement for this purpose. They found that the current LREC tracking has limited ability to support LREC decisionmaking and provided short- and long-term recommendations.
Despite an increased recognition of the importance of language, regional expertise, and culture (LREC) capabilities and recent efforts to improve training in those skills, the U.S. Department of Defense (DoD) faces challenges in tracking the expertise of warfighters and the training they have received. The Defense Language Office (DLO) in the Office of the Under Secretary of Defense for Personnel and Readiness thus tasked the RAND Corporation and the MITRE Corporation to jointly answer the following questions:
To address these questions, the research team conducted interviews with LREC practitioners and policymakers, reviewed existing policies and the research literature, and analyzed survey data. They found that the available data are not sufficient to establish a direct link between LREC training and skills and mission effectiveness. They also found that existing tracking mechanisms are not sufficient to support LREC decisionmaking. They then provided a set of short- and long-term recommendations to fill these gaps.
The review of documented policy, interview responses, and survey data suggests a commonly held belief that LREC training and skills contribute positively to unit readiness and mission effectiveness. Although there is substantial anecdotal evidence to support this belief, the link has not been established through formal, rigorous analysis. Several initiatives are under way to begin collecting data that could be used to assess such a relationship. For example, the Director's Action Group of the Joint Chiefs of Staff Manpower and Personnel Directorate (J1) is working to standardize assessments of the LREC skills necessary for geographic combatant command missions.
Most LREC practitioners and policymakers interviewed for the study expressed the views that LREC skills are critical to the readiness of some but not all units, that the skills are important for those performing specific missions, and that the needs vary among units and individuals. Because many of the required LREC skills differ by mission and task, the interviewees suggested that LREC mission readiness should be defined and measured at the mission and task levels. For example, units that staff vehicle checkpoints would benefit from local language and culture training specific to that task, while medical personnel who interact with local nationals would require a substantially different vocabulary. Pilots might not need any language or culture training because they do not interact with members of the local population.
LRI is a combined database for tracking the language qualifications of individuals across all services. It provides systematic tracking of Defense Language Proficiency Test (DLPT) scores and self-reporting on language capabilities. In 2012, information on regional and cultural expertise will be added. However, people interviewed for this study expressed concern that determining LREC readiness—particularly for GPF who may have very low levels of proficiency—may require additional sources of information. For example, not everyone with foreign-language skills takes the DLPT or takes the tests for very low proficiency. Furthermore, not everyone who completes a self-reporting questionnaire provides all the requested information, so LREC skills may be under-reported.
Interviewees indicated that the current tracking data for LREC skills are not adequate to determine unit readiness. Several interviewees emphasized the need for a DoD-wide recommendation stating what constitutes mission readiness with respect to LREC skills for GPF, similar to definitions of readiness for weapon training. Currently, DoD does not have enough information to clearly state what constitutes readiness in this area.
Informed by policy, directives, academic literature, and interviews, the research team provided recommendations, including those outlined here.
Standardize LREC-related terminology. There is confusion concerning such terms as LREC and culture. It would be helpful to establish standard terms and definitions and to disseminate this terminology through a website that would be accessible to LREC stakeholders and researchers.
Develop measures of mission effectiveness. There is no clear definition of being mission effective or achieving mission success. Defining these terms and the corresponding metrics will be essential to determining the linkage between those concepts and LREC training and skills. Measuring mission effectiveness will require more than a single metric. It may include objective factors, such as number of casualties or amount of anti-U.S. graffiti, and subjective factors, such as public sentiment reported by the local media.
Develop standardized LREC after-action reports (AARs) and surveys to assess the link between LREC training and skills and mission success. Commanders would complete the standardized LREC AARs to provide data—such as mission type, date, location, LREC training received, self-reported skills, and metrics of mission success—that could be used to assess the association between LREC training and skills and mission success (as defined following the recommendation above). Individual GPF members would complete surveys providing similar data.
Develop an infrastructure for LREC data, research, and decisionmaking. This infrastructure should be designed to collect LREC data, improve the accuracy of the data collected, and make the data accessible to decisionmakers and researchers. It should also establish standard terminology, metrics, and improved guidelines for data reporting. Finally, it should facilitate research and dissemination of information related to how LREC training and skills affect mission success.
Develop a causal model linking LREC to mission success. The purpose of establishing a causal model would be to link LREC training and skills to mission readiness and effectiveness, to test assumptions, and to define quantitatively what it means to be LREC-ready. The short-term recommendations above would contribute to the development of this model.
These recommendations are designed to enable DLO to develop a set of LREC readiness metrics at the general level for GPF and at the mission-specific level at which specialized LREC skills may be required. If LREC training is designed to bring GPF to a low but operationally effective skill level, there must be a way to define LREC readiness and effectiveness and a system in place to determine how LREC training and skills relate to the resulting operational success of a unit.