Research Brief
A group of soldiers and men dressed in Middle Eastern clothing sit in a circle.

Locally focused stability operations are designed to help host nation citizens better protect and govern themselves. How can practitioners tell if such efforts are working?

Photo by Cpl. Artur Shvartsberg/U.S. Marine Corps

Locally focused stability operations (LFSO) are a "bottom-up" approach to fostering security, development, and governance, working with and through communities experiencing conflict. LFSO are often facilitated by U.S. military advisors but ideally are a whole-of-government effort led by the host nation (HN). In recent years, U.S. and NATO forces practiced LFSO to help villages in Afghanistan create their own defensively oriented local police units, bring in appropriate development projects, and create links with their district governments. This approach continues to foster security in those areas.

A group of soldiers create a barrier by filling square containers with dirt.

Photo by Spc. Tia Sokimson/U.S. Army

The complexity of the operational environments in which LFSO take place makes it difficult for LFSO practitioners to determine whether their efforts are successful. Every LFSO is distinctive, and, as such, there is no established doctrine for measuring LFSO progress and outcomes. Many practitioners have developed their own set of assessment tools, with varying degrees of success. Effective assessment is important, however, as it can help answer such key questions as:

  • Are the mission objectives being met?
  • Do LFSO personnel have what they need to achieve mission objectives?
  • Do new developments require adjustments in objectives or methods?

In addition, assessments can do more than support a specific LFSO effort. If done right, they can suggest best practices to inform evolving doctrine for stability operations.

To help with the planning and conduct of LFSO assessment, RAND researchers (1) identified challenges related to LFSO assessment, (2) surveyed relevant practices, and (3) created concrete guidance for assessment planning, based on reviews of available tools, findings of prior research, and input from assessment experts and LFSO practitioners. This document compiles best practices for designing LFSO assessments and offers a notional scenario that illustrates a five-step approach to LFSO assessment.

Best Practices for LFSO Assessment

Assessments should be based on a clear "Theory of Change."

Assessments should reflect how and why the commander believes the tasks that have been laid out will result in the desired end-state. This involves documenting the Theory of Change. As illustrated below, the Theory of Change is the underlying structure that connects activities, desired outcomes (effects), and superordinate objectives through a "chain of consequences" and illustrates how outputs at a given stage may become inputs at the next stage. A clearly articulated Theory of Change will enable the assessment team to identify appropriate inputs, outputs, and outcomes to measure.

Theory of Change: A Chain of Consequences Connecting Activities and Goals

Theory of Change: A "Chain of Consequences" Connecting Activities and Goals

Assessments should be commander-centric.

Assessment process should start during campaign or mission planning and should be reviewed and assessed regularly to determine whether changes are needed.

Assessments should "triangulate the truth."

Assessment teams should fully exploit the data and methods available, leveraging the strengths of a given source against the weaknesses of others to obtain valid estimates for the ground truth.

Challenges and Solutions for LFSO Assessment

The study identified a number of challenges that commanders and assessment experts face when conducting LFSO assessments. The table below summarizes these challenges and offers solutions to address them.

LSFO Assessment Challenges Recommended Solutions
Assessing the impact of stability operations in a complex environment is not easy. Identify the challenges, take a deep breath, and forge ahead.

Doctrine and training fail to adequately address the complexities of assessment and create appropriate skill sets.

Prioritize assessment-related doctrine/training.

Institutionalize the assessor role.

Assign individuals with the "right" personality traits.

Have subject-matter experts help fill in contextual gaps.

Allow analysts to deploy to gain operational grounding.

The United States, coalition partners, HNs, nongovernmental organizations, and other stakeholders may have competing visions of stability.

Establish an interagency/international working group to identify a set of variables to track across all lines of effort (security, governance, and development).

Develop an off-the-shelf assessment capability that uses a standard framework and is accepted by the broader stakeholder community.

There is a strategic- vs. tactical-level challenge: Too much aggregation of data obfuscates nuance, too little can overwhelm consumers. Present results in a way that efficiently and clearly summarizes but can support more detailed exploration of data should the need arise.
Assessments sometimes rely on invalid or untested assumptions about causes and effects.

Avoid drawing hasty conclusions by identifying/documenting and testing/validating assumptions.

Adjust the Theory of Change (see next page) accordingly.

Bias, conflicts of interest, and other external factors can create perverse incentives. Employ diverse sources and methods to validate data, using observable indicators, devil's advocacy, ratios, and other tools.
Redundant reporting requirements and knowledge-management challenges impede the assessment process.

Ask for data less frequently but require more in-depth responses or ask for data more often but use less onerous questions.

Provide direct benefits (e.g., tailored products) to those who provide data to validate and motivate.

Maintaining the continuity of the assessment process can be difficult across deployment cycles. Plan for personnel turnover (training, documentation, knowledge management).
Assessment planning often ignores HN perspectives.

Invite HN participation in assessment planning and execution.

Carefully consider hidden agendas.

LFSO Assessment Planning Example

Here we show how commanders and an LFSO assessment team might put our recommendations into action through an LFSO scenario based in a notional West African country in 2015. The operation order below describes the operational environment and presents the concept of operations. The Theory of Change, and how assessment should proceed in this particular case, is presented further below.

Operation Order 123 — Host Nation LFSO Training and Assessment

  1. Situation: the Host Nation (HN), a large West African country, is conducting a multi-year LFSO campaign in its rural northern border provinces to counter a growing insurgency and increasing insurgent-related violence aimed at government forces and local citizens.
    1. Enemy Forces:
      1. Enemy capabilities: small, loosely coordinated insurgent cells equipped with small arms and IEDs, civilian vehicles, cell phone–based communications, supported by several IED makers and cross-border arms smugglers
      2. Probable enemy course of action: attacks on HN forces, government representatives, local civilians, and NGO staff
    2. Friendly Forces: HN LFSO units consist of army and police augmented by villager guard forces and civilian government staff, supported by US military and interagency advisors
  2. U.S. Mission: Support HN government LFSO units through training and assessment
  3. Concept of operations:
    1. HN LFSO personnel are trained by US interagency team
    2. No direct US involvement at the village level
    3. US interagency team conducts assessment of LFSO based on input from HN LFSO personnel, intelligence sources, and villager opinion polling (performed by contracted NGO)
  4. (…)

Assessment Metric Choice Should Be Guided by LFSO Theory of Change

The commander's LFSO Theory of Change should guide all aspects of LFSO assessment, including which metrics to use. The metrics are organized by line of effort (security, development, governance, stability), metric type (quantitative, narrative, or mixed data), collection frequency, and the parties charged with information collection (the assessment team, LFSO teams in the villages, HN staff, or others). The table below presents a sample of local security-related metrics that the notional assessment team in this scenario might consider, based on the Theory of Change outlined on the next page. Of course, there are many other kinds of metrics available for LFSO assessment. A more comprehensive list, along with a fuller description of the scenario, is provided in the associated RAND report.

Security-Related Metrics

Metric Rationale Type Collection Frequency Collector
Number of members of guard force that meet proficiency and equipping standards   Quantitative Biweekly LFSO team
Number of guard force patrols per week Contributes to feeling of security by villagers, deters insurgents, helps gather intelligence, and demonstrates guard force capability and capacity Quantitative Biweekly LFSO team
Level of insurgent attacks in area of operations   Mixed Biweekly LFSO team
Insurgent (INS) influence/propaganda/ intimidation efforts in area of operations   Mixed Biweekly LFSO team, intelligence team
Number of calls to tip line from area of operations Indicates level of intimidation, and pro-HN-government attitude of villagers Quantitative Biweekly LFSO team
Villager perceptions of security   Narrative Quarterly (survey team), biweekly (LFSO team) Survey team, LFSO team
Children moving around unaccompanied Validates survey data about villager perceptions of security Narrative Biweekly LFSO team

Five Steps to LFSO Assessment

  1. Identify the challenges specific to the scenario.

    Specific challenges to the reliable and valid assessment of the LFSO need to be identified and documented at the beginning of the planning process. Assessment in this scenario is difficult, mainly because of the complex mission design and the number of organizations involved in the effort. The assessment team must also identify and mitigate the potential for biased data stemming from organizational self-interest.

  2. Establish the Theory of Change.

    The Theory of Change behind the operation will inform the kinds of metrics to use and the data to be collected, and focuses the assessment by clarifying its purpose, identifying relevant influences to consider, and helping the assessment team choose appropriate metrics for measuring impact. For example, an LFSO Theory of Change would assert that sustainable security in a village requires a whole-government approach integrating increased security, economic development, and improved governance. (See figure below.)

  3. Determine metrics and how/when to collect them.

    When determining appropriate metrics for measuring impact, the following should be specified: Metric type, such as Quantitative data — numbers, ratios (fractions); ordinal metrics (discrete, ranked categories, such as "high," medium," and "low"), nominal metrics (discrete, non-ranked categories, such as "nationality"); Qualitative data — narrative data, descriptive text, etc.; and Mixed data — text containing quantitative information; Collection frequency, such as biweekly, monthly, quarterly, etc.; and Responsibility, such as who is responsible for collecting the information, other actions, etc. See the previous page for a sample of metrics that might be used in this scenario.

  4. Set up processes for data analysis (including aggregation) and communication of results.

    Assessment planning must include determining how best to communicate results to decisionmakers. Practitioners should remember that too much aggregation of data obfuscates nuance, but too little aggregation can overwhelm analysis. Results should be presented in a way that efficiently and clearly summarizes results but can support a more detailed exploration of data should the need arise.

  5. Brief leadership and stakeholders on the assessment plan.

    Once the assessment team has gone through the planning process outlined above, the results should be discussed with the leaders involved in the LFSO effort, in order to obtain additional guidance and, after potentially revising the assessment plan to address any concerns, approval and support from the organizations involved.

Properly Assessing Locally Focused Stability Operations (LFSO) Is Critical for Success

LFSO are a powerful way for the United States and allied forces to help destabilized areas create safety, foster economic growth, and establish accountable governance. However, LFSO practitioners must make sure that their hard work is actually helping to achieve those strategic goals. LFSO assessment thus is a vital activity, that, if done properly, gives LFSO practitioners and their commanders the necessary perspective of their actions. The following principles should guide LFSO assessment efforts:

  • Don't wait. Effective assessment planning is done concurrently with campaign or mission planning.
  • Have direction. The commander's Theory of Change should not only guide overall mission planning, but the assessment planning as well. Effective assessment always requires clearly defined objectives and concepts of operations that are connected to this Theory of Change — "you need to know what you are measuring before you can measure it."
  • Employ Expertise. The most useful assessments are done by trained assessment staff in support of assessment-aware commanders, as part of a process that includes robust input from subordinate commands and staff.
  • Embrace reality. Assessments should favor real and potentially messy data over clean but inaccurate data.

Related Products

Research conducted by

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.