The National Computer Security Survey (NCSS): Final Methodology
Sep 16, 2008
Corporate Approaches and Challenges to Decisionmaking
|PDF file||0.1 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
The emerging field of cybersecurity economics could benefit from better data, better understanding, and better methods for using resources wisely, not only to protect critical products and services but also to provide assurances that software will work as expected. This research brief presents findings that address these key cybersecurity concerns, perceptions of the importance of cybersecurity, and considerations for cybersecurity investment decisions. In particular, it suggests that companies, the government, and other organizations can help improve our understanding of cybersecurity economics by monitoring cybersecurity incidents and responses, soliciting and using standard terminology and measures, and sharing data whenever possible.
Cybersecurity economics is an emerging field. There is a significant need for better data, better understanding, and better methods for using resources wisely, not only to protect critical products and services but also to provide assurances that software will work as expected. In two articles, RAND senior scientist Shari Lawrence Pfleeger and her colleagues addressed these key cybersecurity concerns and identified how different types of companies or organizations perceive the importance of cybersecurity and make cybersecurity investment decisions.
To understand these security approaches, the RAND team considered a business framework that could help explain the interview results and also identify which one of three market disciplines companies embrace to compete in the marketplace: operational excellence, product leadership, or customer intimacy. This framework has been useful in other software-engineering contexts in which it has assisted technology adoption within the context of corporate culture. In addition, the authors believe that the framework can be used not only to analyze existing security attitudes but also to predict likely future cybersecurity actions and attitudes.
An operationally excellent company strives to provide both high-quality customer service and the lowest prices for its goods and services. It emphasizes efficiency and dedication to quality control along with a carefully managed supply chain. Because security is a facet of quality, an operationally excellent company takes security very seriously. By applying standards, controlling processes, and encouraging certification, operationally excellent companies consider security to be central to their trusted brand.
By contrast, a product leader focuses on features and functionality, prizing innovation as it experiments with new offerings. Whereas operationally excellent companies take few risks, product leaders encourage new ventures and a steady stream of new products. Although they take security seriously, good-enough security is a guiding principle; innovation—not process—is the key to avoiding or preventing security problems. As a result, security takes a back seat to performance, is less centralized, and is not the key determinant of a product's success.
The third market discipline, customer intimacy, emphasizes customer needs and requests and excels at meeting them. Security is important for customer-intimate companies when customers express security needs. Thus, the security organizations of customer-intimate companies are less top-down than those of operationally excellent companies, and their centralized procedures involve significant customer interaction. As a result, security is built into products and services only when the customer demands security.
No matter what an organization's cybersecurity posture, it needs data on which to base its security decisions. However, lack of data and uncertainty about the data frequently inhibit sound corporate decisionmaking.
One significant problem is the lack of standards in defining, tracking, and reporting security incidents and attacks. Different surveys ask vastly different questions about "electronic attacks", "virus encounters", "virus disasters", "data intrusions", and "security incidents", among many other terms. Thus, much of the reported evidence is categorized differently from one study to another, and the answers are based on respondents' perceptions, not on consistent capture and analysis of solid empirical data. Moreover, the lack of careful sampling often obscures which population the reported data describe. This hodgepodge of definitions, concepts, and survey types makes it difficult for software managers to know what cybersecurity data to collect and how to compare them with survey results.
Understanding the source and effects of attacks is similarly problematic. Several surveys note that the sources of attacks are unknown in a significant percentage of cases. In addition to the number and types of attacks, significant variations exist in terms of effect, particularly the cost of an attack. Software managers need this cause-and-effect information, not only to design more secure systems but also to estimate resource needs for preventing, mitigating, and recovering from attacks, particularly attacks against the development platforms on which new software is created.
A more significant problem is the difficulty in detecting and measuring both the direct and indirect costs of security breaches. There are neither accepted definitions of loss nor standard, reliable methods to measure it. For example, one survey notes that respondents historically underestimate costs by a factor of seven to 10.
Survey results also highlight another gap concerning security investments: how much organizations have invested in security protection, prevention, and mitigation. Little is known about how companies make investment decisions or how effective their security investments are. Inputs required for such decisionmaking—such as the rate and severity of attacks, cost of enterprise-wide damage and recovery, and actual cost of all types of security measures—are not known with any accuracy. Simple questions, such as how much more security an extra dollar buys, go unanswered.
Faced with these challenges, a RAND study by Davis et al. implemented a national computer security survey on behalf of the Bureau of Justice Statistics and the U.S. Department of Homeland Security. This first large-scale, carefully sampled survey of the state of U.S. cybersecurity was intended to improve the nature and quality of data available to U.S. decisionmakers. By asking broad questions of 36,000 businesses representing all sectors of the economy, the survey results will be similar to the FBI's annual crime statistics, providing a baseline from which cybersecurity trends can be derived. This computer security survey has demonstrated the significant barriers to information sharing that must be overcome before industry surveys are likely to provide a good picture of industry's exposure to cybercrime and the costs and actions necessary to mitigate it.
Software project managers need better data to support their decisionmaking about security. Ideally, a data source should provide information to support the following tasks:
To better understand the cybersecurity challenges, multidisciplinary research is needed within and across the boundaries of engineering, business, and arts and science. Although there is a paucity of empirical analysis and a lack of agreement on findings, researchers are working on five key issues: software quality, market interventions, evaluations, corporate decisionmaking, and cybersecurity modeling.
Companies, the government, and other organizations can be active players in improving our understanding of cybersecurity economics by monitoring cybersecurity incidents and responses, soliciting and using standard terminology and measures, and sharing data whenever possible. They can participate in surveys and studies to better understand the nature and extent of such incidents. By sharing information with researchers and colleagues, they can enable business sectors to take a coordinated approach to preventing and mitigating attacks, as well as inform government policies that affect cybersecurity. And finally, they can apply appropriate business measures so security investment decisions can eventually harmonize with other corporate investment decisions.
This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.