This report reviews the literature on open source intelligence and describes the evolution of open source intelligence over the past 50-plus years. It defines open source information and the open source intelligence cycle and draws parallels with other intelligence disciplines. It examines the methods used by open source tools and the challenges of using off-the-shelf technology for open source analysis, and it suggests areas for further study.
Defining Second Generation Open Source Intelligence (OSINT) for the Defense Enterprise
- How has open source intelligence evolved over the past 50-plus years?
- How should second-generation OSINT and the intelligence cycle be defined?
- How does OSINT compare with the other intelligence disciplines?
- What methods are used by open source tools?
- What are the challenges of using off-the-shelf technology for OSINT?
- What areas of further study should be pursued?
This report presents a framework for understanding the modern practice of open source intelligence. It reviews the literature on open source intelligence and reexamines definitions used in other areas by the U.S. Intelligence Community in the context of modern open source information. The report describes the evolution of open source intelligence over the past 50-plus years, defines open source information and the open source intelligence cycle, and draws parallels between open source as an intelligence discipline and other intelligence disciplines. It also examines the methods used by open source tools and the challenges of using off-the-shelf technology for open source analysis. It concludes by suggesting areas for further study.
Second-Generation Open Source Intelligence (OSINT) and the OSINT Cycle
- OSINT has been revolutionized over the past two decades.
- OSINT is often underutilized by the Intelligence Community because of the difficultly in understanding emerging OSINT sources and methods, particularly social media platforms.
- A more robust definition of OSINT is needed, as open source data can come in many forms.
- Commercial off-the-shelf tools can serve the interests of the Intelligence Community, but they are rarely a perfect match, and many tools have extremely limited utility for intelligence analysis because they are not designed for its purposes.
- Rapid advances in machine learning and natural language processing are changing the efficiency of these methods for sorting, translating, and analyzing data for intelligence purposes.
- There are many opportunities for further research into OSINT methodology, commercial off-the-shelf tools for open source analysis, and open source analytic methods to bring greater intelligence value to the IC and to enable more-efficient operations.
- A rigorous evaluation of the difficulty of the OSINT methodological cycle for different OSINT subtypes could provide for a better division of resources and effort among various OSINT sources.
- The Intelligence Community could benefit greatly from additional work on analytic methods and existing analytic tools.
- A living database of the various platforms could assist the Intelligence Community offices that have an OSINT function, enabling them to spend resources wisely and avoid redundancies.
- It would be productive to evaluate the allocation of Intelligence Community resources between the types of open source information and to explore areas where data science and analytic tools could enhance the Intelligence Community's use of all types of open source information.
Table of Contents
Rethinking OSINT as an Intelligence Discipline
OSINT Tools and Methods