The U.S. Department of Defense has struggled to assess the progress and effectiveness of its efforts to inform, influence, and persuade audiences in support of key national security objectives. This handbook compiles strong assessment practices and examples from a range of sectors in an easy-to-navigate, quick-reference format for defense assessment practitioners.

Assessing and Evaluating Department of Defense Efforts to Inform, Influence, and Persuade

Handbook for Practitioners

Christopher Paul, Jessica Yeats, Colin P. Clarke, Miriam Matthews, Lauren Skrabala

ResearchPublished Apr 17, 2015

Cover: Assessing and Evaluating Department of Defense Efforts to Inform, Influence, and Persuade
Download PDF
Additional Downloads

Includes Metaevaluation Checklist

To achieve key national security objectives, the U.S. government and the U.S. Department of Defense (DoD) must communicate effectively and credibly with a broad range of foreign audiences. DoD spends more than $250 million per year on inform, influence, and persuade (IIP) efforts, but how effective (and cost-effective) are they? How well do they support military objectives? Could some of them be improved? If so, how? DoD has struggled with assessing the progress and effectiveness of its IIP efforts and in presenting the results of these assessments to stakeholders and decisionmakers. To address these challenges, a RAND study compiled examples of strong assessment practices across sectors, including defense, marketing, public relations, and academia, distilling and synthesizing insights and advice for the assessment of DoD IIP efforts and programs. This handbook was designed to be an easy-to-navigate, quick-reference guide to planning and conducting assessments of DoD IIP efforts, analyzing the data generated, and presenting the results. It also offers some background on current assessment practices in DoD and the typical users and uses of DoD IIP assessment results. A companion volume, Assessing and Evaluating Department of Defense Efforts to Inform, Influence, and Persuade: Desk Reference, offers a more detailed exploration and additional examples of assessment in practice.

Key Takeaways

Across Sectors, Best Practices for Assessing Efforts to Inform, Influence, and Persuade Efforts Adhere to a Handful of Common Principles

  • Effective assessment requires clear, realistic, and measurable goals.
  • Effective assessment starts in planning.
  • Effective assessment requires a theory of change or explicit logic of the effort that connects activities to objectives.
  • Change cannot be measured without a baseline.
  • Assessment over time requires continuity and consistency.
  • Assessment is iterative, not something planned and executed once.
  • Assessment requires resources, but any assessment that reduces the uncertainty is valuable.

DoD Has Historically Struggled to Assess the Progress and Effectiveness of Its IIP Efforts

  • There is a lack of shared understanding about how IIP efforts function, which broadens the scope of the assessment questions asked. Good accountability assessments would show not only show that these efforts support broader military campaign and national security goals but also how they do so.
  • In complex operating environments, IIP efforts often face constraints, disruptors, and unintended consequences. Good assessment can help predict these challenges and overcome them when they do arise.
  • Good assessment can support learning from both success and failure. Well-designed, early assessment can help identify problems and get a struggling IIP effort on a path to success.
  • Organizations that do assessment well have cultures that value assessment. Organizing for assessment involves dedicating the necessary resources to the assessment process (5 percent is a common benchmark); ensuring leadership buy-in, advocacy, and willingness to learn from assessment results; training assessment personnel; and implementing a system of continuous assessment, data collection, and program change in response to assessment results.

Recommendations

  • Practitioners require and should demand specific, measurable, achievable, relevant, and time-bound (SMART) campaign objectives for the purposes of assessment. If program and activity managers cannot provide objectives that meet these standards, assessment practitioners should infer or create their own.
  • Practitioners should be explicit about the theory of change or the underlying logic of the effort that connects DoD IIP activities to objectives. If commanders or program designers have not outlined an explicit logic for an activity, assessment practitioners should elicit or develop one in support of assessment.
  • Practitioners require resources for assessment. Assessment is not free, and if its benefits are to be realized, DoD must provide the necessary resources.
  • Practitioners must take care to match the design, rigor, and presentation of assessment results to the intended uses and users. Assessment supports decisionmaking, and providing the best decision support possible should remain at the forefront of practitioners' minds.
  • When complex environments or human dynamics inject uncertainty into IIP efforts, fail fast. That is, field an experimental effort with rapid assessment with the intention of learning from initial shortcomings and making corrections until the effort is effective.

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Web Only
  • Year: 2015
  • Pages: 128
  • Document Number: RR-809/2-OSD

Citation

Chicago Manual of Style

Paul, Christopher, Jessica Yeats, Colin P. Clarke, Miriam Matthews, and Lauren Skrabala, Assessing and Evaluating Department of Defense Efforts to Inform, Influence, and Persuade: Handbook for Practitioners. Santa Monica, CA: RAND Corporation, 2015. https://www.rand.org/pubs/research_reports/RR809z2.html.
BibTeX RIS

Research conducted by

This research was conducted within the International Security and Defense Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.