Civilian Education and Training in the Department of Defense

The Department of Defense (DoD) prides itself on providing "world class" training and continuing education to its military employees and would like to expand that reputation to its civilian employees. But just what does "world class" mean in the civilian context and how can the DoD best measure the quality of its efforts? A recently released RAND study, Ensuring the Quality and Productivity of Education and Professional Development Activities: A Review of Approaches and Lessons for DoD, points out that the DoD's education and training efforts are highly decentralized and suggests that defense policymakers can gauge the caliber of those efforts by choosing one of four assessment approaches.

Background

With some 700,000 civilian employees, the Department of Defense is the single largest employer of civil service workers in the U.S. government. Over the years, many of these civilian employees have taken advantage of a myriad of education and professional development courses that the DoD provides. Today, the department offers more than 100 of these courses to civilian employees. Many of these are sponsored by the Office of the Secretary of Defense, which directs some 20 institutions (such as the Naval Postgraduate School) and 36 programs (such as the Defense Acquisition Career Development Program). Many more are provided by other DoD offices and agencies. And still more civilian education and professional development takes place within each military service.

To get a handle on these various efforts, the DoD in 1998 established the Office of the Chancellor for Education and Professional Development, chartering it to serve as the "principal advocate for the academic quality and cost-effectiveness of all DoD civilian education and professional development activities." The Chancellor's office is responsible for ensuring that all such education and professional development activities achieve appropriate standards of quality and productivity.

Identifying Assessment Alternatives

Shortly after its creation, the Chancellor's office asked RAND's National Defense Research Institute (NDRI) to help identify promising approaches DoD might use to assess its educational programs. RAND approached this task by reviewing the relevant literature and analyzing the methods that corporations, state governments, and universities use to assess education and professional development programs. In addition, it interviewed assessment experts and visited organizations that conduct such evaluations.

RAND found that the Chancellor's office might structure such evaluations in several ways, which are outlined in the table on the back page. It can allow individual DoD institutions or programs to review the quality and productivity of their efforts, with no outside involvement (Approach 1). A related approach is for such a review process to be monitored by the Chancellor's office or a third party (Approach 2). Alternatively, the Chancellor's office or a third party can take on the entire review process itself (Approach 3). Or the office can focus not on the institutions or programs but on their outcomes and attempt to measure and certify student achievement (Approach 4). Each approach has strengths and weaknesses; each works well in a different setting.

The Academic Audit Approach: An Intriguing Alternative

Given that the DoD system of education and professional development is highly complex and decentralized, and that the Chancellor's office has little formal authority over the organizations providing courses, NDRI found that Approach 2—involving an intermediary—appears to be the most promising. An intermediary is an organization other than the customer or provider of the education and could be the Chancellor's office, other DoD entities, or non-DoD organizations. Such intermediaries would be responsible for evaluating the processes that individual departments and schools employ to assess the quality and productivity of their educational efforts. This practice would be similar to quality improvement efforts that have been used in the business world for the last 25 years and that were adopted by the International Organization for Standardization in the 1980s to certify that manufacturing companies worldwide adhere to certain quality standards. This practice also would be similar to academic audits that increasingly are being used in other education settings. Such audits typically are conducted by intermediary organizations and focus on assuring that providers of education have effective processes in place to measure their own quality and engage in ongoing self-improvement.

The study also urged that the DoD look beyond assessments of existing education and development efforts. Following the lead of many corporations and educational institutions, a clear link between education and professional development on the one hand and the basic mission of the DoD on the other is needed. The Chancellor's office should advocate for the development of a central learning organization within the DoD that would be modeled after a corporate learning organization or state higher education coordinating board. Such a move would be challenging: It would require high-level DoD support and substantial collaboration among a range of stakeholders, including other organizations within the Department of Defense responsible for workforce planning and personnel policies.