Appendix E. Methodology Used to Develop the AI Security Guide

This guide was developed as a practical resource for a broad community of users. Its scope deliberately integrates both technical safeguards and governance considerations to provide a comprehensive foundation for organizations seeking to reduce the security risks inherent to AI systems.

Research Design

We developed the AI Security Guide using a structured, multistep research design intended to balance research depth with practical usability. This multistep process included the following:

  • Comprehensive source review. We conducted an extensive review of leading frameworks, standards, and both academic and industry publications relating to AI security. We included sources that emphasized technical safeguards, risk management considerations, and governance frameworks to ensure that the guide was rooted in authoritative guidance across the technical and policy communities.
  • Threat-agnostic orientation. We deliberately designed the guide to be threat-agnostic and not dependent on any single organization’s security posture. Instead of anchoring controls to specific adversaries or scenarios, we grounded our analysis in well-documented AI model vulnerabilities and their mitigations. This approach allowed us to identify security controls with the broadest inherent risk-reduction potential, ensuring that the guide remains applicable across diverse environments and use cases.
  • Control prioritization and structured scoring. We approached the identification and prioritization of security controls using a three-step process:
    1. AI-assisted threat mapping. We used an internal large language model (LLM), RAND’s instance of GPT-4o, to map each security control to relevant Open Web Application Security Project (OWASP) and Berryville Institute of Machine Learning (BIML) threats. For each control, we provided the control name, the AI pipeline elements, and subcategories to which the control applied. We instructed the model to assign all applicable risks (without limitation) and to generate a concise explanation of how the control mitigates each threat.
    2. Expert validation and weighting. RAND subject-matter experts then manually reviewed and validated the LLM’s output for accuracy and relevance. Using the validated mapping, we scored each control according to an inverse weighting of OWASP criticality rankings (highest-ranked OWASP risks = 10 points; lowest = 1 point). This ensured that greater priority was given to controls addressing the most-critical threats and buying down the most risk.
    3. Composite scoring across OWASP and BIML. To generate a final priority score, we summed the points for all threats that a single control mitigates. A higher composite score indicates greater overall risk buydown. Because BIML threats are not ranked, we aligned each BIML risk with its closest OWASP category and applied the corresponding criticality ranking. This produced a consistent, transparent scoring framework in which each control’s value is directly proportional to the severity and number of threats it mitigates.
  • Lifecycle alignment. We organized controls according to the phases of the AI lifecycle—design, develop, deploy, and operate. This framing ensured end-to-end coverage of the vulnerabilities and mitigations as they manifest across the AI model pipeline.
  • Iterative validation and subject-matter expert engagement. We shared draft outputs of the individual sections and the guide as a whole with multiple internal and external reviewers. We sought both general comments and structured feedback to help refine our control selection and prioritization. We also sought feedback on applicability (Is the guide appropriate for the intended audience?), usability (e.g., Is the guide easy to navigate?), content (Is the information accurate?), and utility (Is the guide useful?). End-user testing provided further validation and refinement in an operational context.

This approach helped produce a guide that balances empirical grounding with practical, easy-to-navigate guidance across the AI lifecycle. The outcome is both evidence-based and usable by a variety of practitioners across the technical, compliance, and policy communities.