Rigorously Interpreted Quotation Analysis for Evaluating Causal Loop Diagrams in Late-Stage Conceptualization

Published in: System Dynamics Review, Volume 38, Issue 1, pages 41–80 (January/March 2022). doi: 10.1002/sdr.1701

Posted on RAND.org on December 01, 2022

by Andrada Tomoaia-Cotisel, Samuel D. Allen, Hyunjung Kim, David Andersen, Zaid Chalabi

As a field, system dynamics has made more progress in developing formal model evaluation methods that use quantitative data than ones that use qualitative data. This article describes a formal method for evaluating a causal loop diagram (CLD) in late-stage conceptualization—referred to as rigorously interpreted quotation analysis—and illustrates its application in a case study. The method uses a systematic and explicit interpretive process to confirm or disconfirm all diagram elements in a CLD by comparing it to stakeholders' verbatim descriptions of their experiences in a complex dynamic situation. In so doing, this method enables the resolution of discrepancies between a CLD and qualitative data, building confidence in the structural aspects of a dynamic hypothesis. It does so via a process that is approachable for experts and stakeholders alike.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.