The U.S. Coast Guard needs to measure its operational-level performance effectively. To assist this effort, the authors created logic models detailing elements of the Coast Guard's 11 missions. After identifying and assessing the validity, reliability, and feasibility of relevant existing metrics, the authors go on to propose potential metrics that could improve on or complement existing metrics, as well as a framework for applying the metrics.
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 1.2 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback200 pages | $28.50 | $22.80 20% Web Discount |
Research Questions
- What aspects of operational performance should the Coast Guard aim to measure?
- How well does the Coast Guard measure operational performance?
- How could the Coast Guard better measure operational performance?
The U.S. Coast Guard needs to measure its operational-level performance effectively to make informed decisions about resource allocation. To assist the Coast Guard in this effort, the authors worked with official documentation and subject-matter experts to develop logic models describing each of the 11 statutory missions of the Coast Guard and using the descriptions to ascertain what aspects of these missions should be measured. The authors examined existing metrics in the light of these logic models, evaluating the metrics in terms of their validity (how well they measured elements of the logic models), reliability (how consistently measurements can be made), and feasibility (how readily measurements can be made). They also analyzed the extent to which existing metrics measure elements of the logic models. They then developed and evaluated sets of potential metrics that could improve on or complement the existing metrics, together with a framework for applying metrics in decisionmaking. This report describes the logic models, existing metrics, and potential metrics, including their relationships with one another and their derivation from Coast Guard sources. Note that, as of this writing, the Coast Guard is considering these findings and has not adopted them as doctrine.
Key Findings
This Research Produced a Series of Logic Models Describing Coast Guard Missions
- Using Coast Guard documentation and discussions with subject-matter experts, the authors developed logic models that describe each of the service's 11 statutory missions in detail. These logic models include the outcomes, accomplishments, and activities (collectively termed "elements") associated with each mission, as well as relationships among them.
- The elements of these logic models suggest suitable areas for performance measurement.
Existing Metrics Do Not Capture All the Elements of These Models
- Existing metrics capture only 29 percent of the elements of the logic models.
- Of the existing metrics, 48 percent have high validity, 74 percent have high reliability, and 82 percent have high feasibility; only 34 percent rate highly on all three criteria.
The Authors Developed a Menu of Potential Additional Performance Measures Addressing All Elements of the Models
- The potential measures either cover unaddressed elements or offer higher levels of validity, reliability, and/or feasibility than existing metrics.
- These measures are grouped by mission and aligned with specific elements of the logic models.
- The Coast Guard can select the potential measures that address its most critical operational issues.
The Authors Developed a Framework for Applying Metrics to Make Decisions
- By using a structured framework, the Coast Guard can use metrics more effectively to aid operational-level decisions.
Recommendations
- The logic models offer a means for the Coast Guard to assess what it needs to measure with respect to each mission.
- The service can select from among existing and proposed metrics to find those that are most suitable for addressing operational questions and making decisions.
- The metrics will need to be evaluated to determine appropriate values for individual metrics and relationships among metrics.
- The Coast Guard can also coordinate with other components of the Department of Homeland Security, as well as with the Department of Defense, National Oceanic and Atmospheric Administration, Environmental Protection Agency, Drug Enforcement Agency, and other agencies, to align operational metrics in a way that facilitates communication and decisionmaking.
- The Coast Guard can use a structured framework to facilitate the use of metrics in making operational-level decisions.
Table of Contents
Chapter One
Introduction
Chapter Two
Methodology
Chapter Three
Safety Missions
Chapter Four
Security Missions
Chapter Five
Stewardship Missions
Chapter Six
Discussion of Overall Findings
Chapter Seven
Using Metrics to Support Decisionmaking
Appendix A
Search and Rescue
Appendix B
Marine Safety
Appendix C
Ports, Waterways, and Coastal Security
Appendix D
Drug Interdiction
Appendix E
Migrant Interdiction
Appendix F
Defense Readiness
Appendix G
Ice Operations
Appendix H
Aids to Navigation and Waterways Management
Appendix I
Marine Environmental Protection
Appendix J
Living Marine Resources
Appendix K
Other Law Enforcement
Research conducted by
This research was sponsored by the U.S. Coast Guard Atlantic Area. It was conducted within the International Security and Defense Policy Center and the RAND Homeland Security and Defense Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.