This report presents the results of a study investigating how evaluations of counterterrorism (CT) and preventing and countering violent extremism (PCVE) policies were designed and conducted over the last five years and what practical lessons can be drawn from these. The study encompassed a set of interconnected research tasks, including the production of an inventory of CT and PCVE evaluations and of an ad-hoc analytical framework.

Cover: Counterterrorism evaluation
Read Online

Counterterrorism evaluation

Taking stock and looking ahead

by Jacopo Bellasio, Joanna Hofman, Antonia Ward, Fook Nederveen, Anna Knack, Arya Sofia Meranto, Stijn Hoorens

Download eBook for Free

FormatFile SizeNotes
PDF file 1.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What is meant by counterterrorism policy in the Netherlands and other countries? And what different types of counterterrorism and preventing and countering violent extremism measures, policies and interventions can be distinguished in the Netherlands and abroad?
  2. What evaluations of counterterrorism and preventing and countering violent extremism policies have been conducted over the last five years in the Netherlands and abroad?
  3. What can be said about counterterrorism and preventing and countering violent extremism evaluations characteristics?
  4. What are the differences and similarities between the identified evaluations of counterterrorism and preventing and countering violent extremism policies?
  5. What does the evaluation literature say about quality criteria for evaluations?
  6. To what extent do the identified counterterrorism and preventing and countering violent extremism evaluations meet these quality criteria?

In 2010, a study commissioned by the Research and Documentation Centre (Wetenschappelijk Onderzoek- en Documentatiecentrum, WODC) in the Netherlands first aimed to assess evaluation practice and culture in the fields of counterterrorism (CT) and preventing and countering violent extremism (PCVE). Seven years on from this initial study, in light of the continuous developments and investments occurring in the fields of CT and PCVE, there was a growing necessity for a renewed analysis of how strategies, policies and initiatives in these fields are evaluated and what advances could be made.

Mindful of this, in December 2017 the WODC commissioned RAND Europe to conduct a study aimed at investigating how evaluations of CT and PCVE policies in the Netherlands and abroad have been designed and conducted over the last five years, and what practical lessons can be drawn regarding such evaluations.

To achieve this goal, the study entailed the undertaking of three interconnected research tasks. First, the study opened with the production of an inventory of evaluations of CT and PCVE strategies, policies and interventions conducted since 1 January 2013. In parallel to this, the study team developed an analytical framework to be used for assessing evaluations collected. Lastly, the study team conducted a structured analysis and review of the evaluations inventory.

This report discusses the activities, results and findings of this study and presents recommendations for future work in this area. It is aimed at a specialist audience of academics, practitioners and policy-makers with an intimate understanding of evaluation, particularly in the context of CT and PCVE.

Key Findings

  • The scope, purpose and activities characterising CT have evolved in recent years in parallel with the growth of the PCVE field. This is both a response to changes in the threat landscape and a result of a growing understanding of terrorism and violent extremism. Dutch authorities have designed and implemented a wide array of CT and PCVE legislation and measures, in recognition of the need for an approach that goes beyond traditional CT measures.
  • The study produced an inventory comprising of 48 CT and PCVE evaluations manuscripts (38 in English, 6 in Dutch and 4 in German). These were analysed through an analytical framework to identify trends, patterns and characteristics of CT and PCVE evaluation.
  • When compared with results from previous reviews, results from this study suggest that a growing volume of CT and PCVE evaluations is being undertaken and that the majority of these rely on primary data from multiple sources, perspectives and methods. Nonetheless, there appear to be limits to the extent to which evaluation practice has advanced and grown evenly across all areas of CT and PCVE work as significant gaps and shortcomings continue to mar a number of evaluations (e.g. evaluations characterised by designs that undermine their ability to draw robust conclusions about an initiative's impact).
  • A number of issues and challenges continue to mar evaluations in the fields of CT and PCVE, not all of them are exclusive to these fields. Lessons and reflections identified from evaluations reviewed pertain to: (i) inherent complexities of the fields of CT and PCVE; (ii) challenges associated with measuring real-world phenomena; (iii) challenges associated with existing evaluation designs; (iv) practical difficulties of conducting evaluations; and (v) drawbacks and benefits of specific evaluation methods.

Recommendations

  • Continue to invest in the evaluation of planned and existing initiatives in the CT and PCVE policy areas, setting minimum quality and robustness requirements for future evaluations.
  • Develop new evaluation designs, frameworks and approaches for conducting evaluation in the CT and PCVE policy areas, encouraging the adoption wherever possible of (quasi-)experimental designs.
  • Encourage further research on the dynamics, drivers and factors governing the phenomena of radicalisation, violent extremism and terrorism.
  • Conduct regularly mapping and stocktaking exercises akin to the present study, providing resources and means required to gain access to CT and PCVE initiatives' evaluators and beneficiaries.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Methodology

  • Chapter Three

    Understanding the study context and its key definitions

  • Chapter Four

    Building an analytical framework

  • Chapter Five

    Analysing CT and PCVE evaluations

  • Chapter Six

    Identifying issues and learning lessons from CT and PCVE evaluations

  • Chapter Seven

    Overall conclusions and recommendations

  • Annex A

    CT and PCVE evaluations inventory

  • Annex B

    Methodology

  • Annex C

    Stakeholder elicitation details and tools

  • Annex D

    Analytical framework

Research conducted by

The research described in this report was commissioned by the Research and Documentation Centre (Wetenschappelijk Onderzoek- en Documentatiecentrum, WODC) and conducted by RAND Europe.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.