Design
Table B.1. Security Controls Related to Data
| Subcategory | Identifier | Action or Control | OWASP and BIML Threats Mitigated | Why the Control Is Relevant |
|---|---|---|---|---|
| Access control and authorization | D.ACA-1 | Restrict access to sensitive data based on security clearance levels. |
|
Ensures that only authorized personnel with appropriate security clearance can access sensitive data, preventing unauthorized access and potential misuse |
| D.ACA-2 | Enable multifactor authentication (MFA) for all cloud data access. |
|
Ensures that only authorized users can access cloud data, reducing the risk of compromised authentication and unauthorized access to models and data | |
| Continuous security monitoring | D.CSM-1 | Continuously monitor systems for unauthorized access or anomalies involving sensitive data. |
|
Ensures detection and response to unauthorized access or anomalies, protecting against unauthorized data access, sensitive information disclosure, and model manipulation |
| Cryptographic protection and encryption | D.CPE-1 | Implement homomorphic encryption for secure data analysis and privacy-preserving machine learning (ML). |
|
Ensures that data remain confidential during processing, preventing unauthorized access and extraction of sensitive information |
| D.CPE-2 | Encrypt all sensitive data used or generated by AI-enabled applications. |
|
Prevents unauthorized access and disclosure, ensuring data confidentiality and integrity and mitigating risks of sensitive information being extracted or inferred from AI models | |
| Data backup and recovery | D.DBR-1 | Perform regular backups and establish recovery plans for infrastructure data. |
|
Ensures data integrity and availability by allowing recovery from disruptions, which prevents denial of service and mitigates supply chain vulnerabilities |
| Data classification and labeling | D.DCL-1 | Classify information by sensitivity and apply corresponding security measures. |
|
Ensures that sensitive data are identified and protected with appropriate security measures, preventing unauthorized access and disclosure |
| Data collection and minimization | D.DCM-1 | Collect only the data that are strictly necessary. |
|
Reduces the attack surface for data poisoning, sensitive information disclosure, membership inference, and model skewing by minimizing the amount of data that can be manipulated or extracted |
| D.DCM-2 | Implement checks to ensure that only valid and authorized data are collected. |
|
Ensures that only authorized and valid data are collected, preventing unauthorized data access and manipulation and protecting against data poisoning attacks | |
| Data integrity validation and monitoring | D.DIVM-1 | Implement an immutable backup storage system to ensure that all objects, especially log data, are unchangeable. |
|
Ensures that data, especially log data, remain unchanged and tamper-proof, preventing unauthorized modifications that could poison training data or skew model behavior |
| Data isolation and segmentation | D.DIS-1 | Store training data separately from production data to prevent compromise. |
|
Reduces the risk of unauthorized access and manipulation, thereby preventing poisoning and supply chain attacks |
| D.DIS-2 | Separate and label untrusted content to limit its influence on user prompts. |
|
Prevents malicious inputs from influencing model behavior and causing unintended actions | |
| Data processing and governance constraints | D.DPGC-1 | Create clear data policies and allow users to opt out of data training processes. |
|
Ensures that data used in training are properly managed and that users can opt out, reducing the risk of poisoned data, sensitive information disclosure, and unauthorized data extraction |
| D.DPGC-2 | Establish and enforce rules for secure data processing. |
|
Helps prevent excessive resource usage, unauthorized access, and manipulation of training data, ensuring data integrity and proper model behavior | |
| Data quality and integrity assurance | D.DQIA-1 | Use OWASP CycloneDX or Machine Learning Bill of Materials (ML-BOM) to track data origins and transformations; verify data legitimacy. |
|
Ensures the integrity and legitimacy of data, preventing malicious alterations and unauthorized data extraction during model development |
| D.DQIA-2 | Validate and verify training data with checks and multiple data labelers before model training. |
|
Ensures the integrity and accuracy of training data, preventing malicious alterations that could degrade model performance or introduce vulnerabilities | |
| D.DQIA-3 | Use digital signatures and checksums to verify and reject inauthentic feedback data. |
|
Ensures the authenticity and integrity of feedback data, preventing malicious alterations that could degrade model performance or introduce vulnerabilities | |
| Data sanitization and lifecycle management | D.DSLM-1 | Ensure that data are fully erased from all storage locations. |
|
Ensures that sensitive data are not recoverable, preventing unauthorized access and extraction of private information used in training or inference |
| D.DSLM-2 | Implement data sanitization to scrub or mask sensitive content before used in training models. |
|
Prevents sensitive information from being included in training datasets, reducing the risk of poisoning, disclosure, and skewing | |
| Data storage security and access control | D.DSSAC-1 | Encrypt stored data and implement access controls to protect information at rest. |
|
Prevents unauthorized access and extraction of sensitive information, ensuring data integrity and confidentiality |
| Data transmission security | D.DTS-1 | Encrypt data in transit between AI models, application programming interfaces (APIs), and applications. |
|
Ensures that sensitive information is protected from interception and unauthorized access, preventing disclosure and manipulation |
| D.DTS-2 | Use secure protocols for data collection over networks, and monitor all network traffic. |
|
Ensures proper configuration, prevents unsafe data handling, protects sensitive information, and mitigates risks of data extraction and supply chain attacks | |
| Network security and isolation | D.NSI-1 | Segment networks to isolate and protect sensitive data. |
|
Helps prevent unauthorized access to sensitive data and limits the exposure of critical information, thereby mitigating risks of data extraction, disclosure, and manipulation |
| Privacy-preserving techniques | D.PPT-1 | Implement data protection measures during the design phase of system development. |
|
Ensures that data protection principles are integrated from the start, preventing unauthorized access, manipulation, and extraction of training data |
| Robust data training and processing | D.RDTP-1 | Regularly monitor and update training datasets to prevent transfer of malicious knowledge. |
|
Helps prevent the introduction and persistence of malicious or biased data, ensuring the integrity and reliability of the model |
| D.RDTP-2 | Use secure and trusted datasets to prevent malicious knowledge transfer. |
|
Ensures that training data are free of malicious alterations, preventing adversaries from introducing vulnerabilities or biases into the model | |
| D.RDTP-3 | Train ML models using randomized or shuffled data to enhance security. |
|
Helps prevent attackers from inferring membership of specific data points and reduces the risk of poisoning attacks by making it harder to introduce malicious patterns | |
| Third-party security | D.TPS-1 | Rigorously vet data vendors and validate model outputs against trusted sources for poisoning. |
|
Ensures that data vendors are trustworthy and model outputs are validated to prevent and detect poisoning and supply chain attacks |
Table B.2. Security Controls Related to Human Talent
| Subcategory | Identifier | Action or Control | OWASP and BIML Threats Mitigated | Why the Control Is Relevant |
|---|---|---|---|---|
| Access control and authorization | HT.ACA-1 | Grant access to sensitive data only as needed, based on least privilege. |
|
Ensures that access to sensitive data is restricted to those who need it, preventing unauthorized access and potential data leaks |
| HT.ACA-2 | Implement attribute-based access control (ABAC) for dynamic access policies. |
|
Ensures that access to resources is dynamically controlled based on attributes, preventing unauthorized access to objects, properties, and functions and securing plugin interactions | |
| HT.ACA-3 | Implement MFA to prevent misuse of legitimate credentials to access systems. |
|
Ensures that even if credentials are compromised, additional authentication factors are required, reducing the risk of unauthorized access and misuse | |
| Cloud and infrastructure security | HT.CIS-1 | Grant cloud access only to necessary users with minimum required permissions. |
|
Ensures that only authorized users have access to cloud resources, reducing the risk of unauthorized access and potential misuse or theft of AI models and data |
| Compliance and regulatory training | HT.CRT-1 | Implement access regulation, security awareness training, and role-based security training per applicable security standards. |
|
Ensures proper access control, authorization, and security configurations through training and compliance with established security standards (CMMC 2.0, National Institute of Standards and Technology (NIST) SP 800-171, TS//SCI) |
| Data handling | HT.DH-1 | Train employees on best practices for data handling. |
|
Ensures that employees are aware of proper data handling techniques to prevent malicious data manipulation and supply chain vulnerabilities |
| HT.DH-2 | Allow only authorized personnel to collect data. |
|
Helps prevent unauthorized individuals from introducing malicious data or components into the training dataset, thereby mitigating risks of data poisoning and supply chain attacks | |
| Human oversight and approval controls | HT.HOAC-1 | Ensure that privileged users receive training on their roles and responsibilities. |
|
Ensures that privileged users understand access control policies, reducing the risk of unauthorized access and misuse of sensitive functions and data |
| HT.HOAC-2 | Implement human approval for all privileged operations to prevent unauthorized actions. |
|
Ensures that privileged operations are subject to human oversight, preventing unauthorized actions and excessive autonomy of AI systems | |
| Incident response and accountability | HT.IRA-1 | Implement a disciplinary process for handling security breaches. |
|
Ensures accountability and corrective actions for security breaches, reducing the risk of repeated misconfigurations and compromised components in the AI pipeline |
| Information technology (IT) security management and governance | HT.ISMG-1 | Implement and maintain IT security management policies and procedures. |
|
Ensures that employees are trained on security policies and access controls, reducing the risk of authentication flaws, authorization issues, misconfigurations, and supply chain vulnerabilities |
| Personnel screening and background verification | HT.PSBV-1 | Conduct screening and background checks for all employees and contractors. |
|
Ensures that individuals with access to the AI development pipeline are trustworthy, reducing the risk of compromised components, model theft, and malicious actions |
| Physical security and access management | HT.PSAM-1 | Restrict access to sensitive data based on security clearance levels. |
|
Ensures that only authorized personnel with the appropriate security clearance can access sensitive data, preventing unauthorized access and potential misuse |
| HT.PSAM-2 | Restrict physical access to authorized personnel only. |
|
Helps prevent unauthorized access to sensitive components, reducing the risk of model theft, model extraction, supply chain attacks, and manipulation of training processes | |
| Role-based access | HT.RBA-1 | Implement role-based access control (RBAC) to restrict model training initiation and modification to authorized users. |
|
Ensures that only authorized personnel can initiate or modify model training, preventing unauthorized access and potential malicious modifications |
| HT.RBA-2 | Define roles, tasks, and knowledge areas for the cybersecurity workforce. |
|
Ensures that employees have clear roles and responsibilities, reducing the risk of unauthorized access and misuse of AI systems | |
| HT.RBA-3 | Assign distinct roles for AI model development, deployment, and maintenance. |
|
Ensures that distinct roles and responsibilities prevent unauthorized access and excessive permissions, reducing the risk of unauthorized actions and supply chain vulnerabilities | |
| Security training and awareness programs | HT.STAP-1 | Train employees to identify and respond to social engineering attacks. |
|
Helps prevent manipulation of inputs and unauthorized access to sensitive information, thereby protecting against prompt injection, input manipulation, and data extraction attacks |
| HT.STAP-2 | Conduct regular security awareness, education, and training sessions for all employees. |
|
Ensures that employees are aware of security best practices, reducing the risk of misconfigurations, supply chain vulnerabilities, overreliance on AI systems, and inadvertent disclosure of sensitive information | |
| HT.STAP-3 | Train users to avoid inputting sensitive information and to follow best practices for secure large language model (LLM) interaction. |
|
Helps prevent prompt injection, sensitive information disclosure, and input manipulation attacks by ensuring that users do not inadvertently provide inputs that could be exploited | |
| HT.STAP-4 | Educate developers on AI supply chain risks and verify packages before installation. |
|
Ensures that developers are aware of and can mitigate risks associated with compromised components in the AI development pipeline by verifying packages before installation | |
| Threat detection and monitoring | HT.TDM-1 | Implement measures to detect and prevent unauthorized access and credential manipulation. |
|
Ensures that unauthorized access and credential manipulation are detected and prevented, thereby protecting against improper authentication, unauthorized function access, insecure handling of outputs, and adversarial inputs |
Table B.3. Security Controls Related to Compute (on Premises)
| Subcategory | Identifier | Action or Control | OWASP and BIML Threats Mitigated | Why the Control Is Relevant |
|---|---|---|---|---|
| Access control and authorization | COP.ACA-1 | Harden training systems and environments to reduce attack surfaces. |
|
Reduces vulnerabilities that could be exploited to poison training data, compromise supply chain components, or embed backdoors |
| COP.ACA-2 | Implement RBAC to restrict model training initiation and modification to authorized users. |
|
Ensures that only authorized users can initiate or modify model training, preventing unauthorized access and potential malicious modifications | |
| COP.ACA-3 | Limit entry to areas with sensitive data to authorized personnel only. |
|
Prevents unauthorized personnel from directly accessing sensitive data, models, and systems, thereby mitigating risks of theft, tampering, and extraction | |
| COP.ACA-4 | Implement badge systems, guards, and surveillance to control physical access. |
|
Prevents unauthorized access to hardware and systems, protecting against theft, tampering, and unauthorized extraction of models and sensitive data | |
| Change management | COP.CM-1 | Record, test, and approve all changes. |
|
Ensures that all changes are properly documented, tested, and approved, reducing the risk of misconfigurations and vulnerabilities introduced through unverified changes |
| Data integrity and backup systems | COP.DIBS-1 | Implement an immutable backup storage system to ensure that all objects, especially log data, are unchangeable. |
|
Ensures that log data and other objects remain unchanged, preventing unauthorized modifications that could lead to data poisoning, model skewing, or misconfigurations |
| COP.DIBS-2 | Implement unidirectional data diodes to ensure that logs are non-editable at the hardware level. |
|
Ensures that logs are non-editable and securely transferred, preventing tampering and unauthorized access to sensitive information | |
| Data protection and confidential computing | COP.DPCC-1 | Implement confidential computing for all applicable network services and information transfers. |
|
Ensures that data and computations are protected from unauthorized access and tampering, mitigating risks of sensitive information disclosure, unauthorized data extraction, and manipulation during processing |
| COP.DPCC-2 | Store sensitive data in secure environments with controlled access. |
|
Ensures that sensitive data are protected from unauthorized access and extraction, reducing the risk of information disclosure and manipulation | |
| Infrastructure resilience and availability | COP.IRA-1 | Ensure that data center operations are available and resilient. |
|
Helps prevent resource exhaustion and service degradation caused by excessive or malicious resource consumption |
| COP.IRA-2 | Install uninterruptible power supply (UPS) and backup generators to ensure continuous power during emergencies. |
|
Ensures a continuous power supply to prevent service disruption and maintain system availability during power outages, mitigating risks of denial of service and resource exhaustion | |
| Infrastructure security management | COP.ISM-1 | Implement and maintain security measures for infrastructure, including data protection and monitoring. |
|
Ensures proper configuration and monitoring of infrastructure to prevent misconfigurations, maintain accurate inventory, and secure supply chain components |
| Network security and isolation | COP.NSI-1 | Deploy a firewall to scan and validate all requests and responses for attack signatures. |
|
Scans and validates requests and responses to detect and block attack signatures, preventing unauthorized access, resource abuse, and malicious inputs |
| COP.NSI-2 | Isolate the network by strictly limiting external connections. |
|
Helps prevent unauthorized access and data exfiltration, reducing the risk of SSRF, supply chain vulnerabilities, and model theft | |
| Continuous security monitoring | COP.CSM-1 | Implement continuous security monitoring and protection for all systems and networks. |
|
Helps detect and respond to misconfigurations, manage inventory, prevent denial-of-service attacks, and identify compromised components in the AI supply chain |
| Vulnerability management and security testing | COP.VMST-1 | Perform comprehensive security testing to identify vulnerabilities and weaknesses in the entire system. |
|
Helps identify and mitigate such vulnerabilities as broken authentication, authorization flaws, security misconfigurations, and various injection and poisoning attacks by thoroughly evaluating the system’s source code and configurations |
Table B.4. Security Controls Related to Compute (Cloud)
| Subcategory | Identifier | Action or Control | OWASP and BIML Threats Mitigated | Why the Control is Relevant |
|---|---|---|---|---|
| Access control and authorization | CC.ACA-1 | Implement RBAC to restrict data access based on user roles. |
|
Ensures that only authorized users can access specific data and functionalities based on their roles, preventing unauthorized access and misuse |
| CC.ACA-2 | Enable MFA for all cloud data access. |
|
Ensures that even if authentication credentials are compromised, unauthorized access to cloud data is prevented, thereby protecting against broken authentication and supply chain vulnerabilities | |
| Change management | CC.CM-1 | Record, test, and approve all changes to cloud infrastructure. |
|
Ensures that all changes to cloud infrastructure are properly documented, tested, and approved, reducing the risk of misconfigurations and vulnerabilities introduced through unverified changes |
| Cloud-specific security implementation | CC.SI-1 | Implement and enforce cloud-specific security measures. |
|
Ensures proper configuration and management of cloud resources, preventing misconfigurations and securing the supply chain against compromised components |
| Data protection and confidential computing | CC.DPCC-1 | Implement confidential computing for all applicable network services and information transfers. |
|
Ensures that data and computations are protected from unauthorized access and tampering, mitigating risks of sensitive information disclosure, model extraction, and various inference attacks |
| CC.DPCC-2 | Store sensitive data in secure cloud environments with controlled access. |
|
Prevents unauthorized access and manipulation, ensuring that sensitive information is protected and reducing the risk of data extraction and model skewing | |
| Infrastructure resilience and operations | CC.IRO-1 | Ensure that data center operations are available and resilient. |
|
Ensures that cloud data center operations can handle high loads and resource demands without service degradation or interruption |
| Infrastructure security management | CC.ISM-1 | Implement and maintain security measures for cloud infrastructure, including data protection and monitoring. |
|
Ensures proper configuration and monitoring of cloud infrastructure to prevent misconfigurations, maintain accurate inventory, and secure supply chain components |
| Network security and isolation | CC.NSI-1 | Implement firewalls, intrusion detection system and intrusion protection system (IDS/IPS), and network segmentation in cloud-based training environments. |
|
Helps prevent unauthorized access, monitor and block malicious traffic, and ensure proper configuration to protect against resource exhaustion, sensitive flow access, and supply chain attacks |
| Physical security controls | CC.PSC-1 | Implement physical access controls and monitoring for cloud infrastructure facilities. |
|
Ensures that unauthorized physical access to cloud infrastructure is prevented, thereby protecting against theft or tampering with AI models and the supply chain |
| Continuous security monitoring | CC.CSM-1 | Implement continuous security monitoring and protection for all cloud systems and networks. |
|
Helps detect and respond to misconfigurations, manage inventory, prevent denial-of-service attacks, and identify compromised components in the AI supply chain |
| Training environment security | CC.TES-1 | Harden training systems and environments to reduce attack surfaces. |
|
Reduces the risk of unauthorized access and manipulation, ensuring the integrity and security of training data and components |
| Vulnerability management | CC.VM-1 | Perform comprehensive security testing to identify vulnerabilities and weaknesses in cloud systems. |
|
Helps identify and mitigate vulnerabilities (such as broken authentication, authorization flaws, security misconfigurations, and various injection and poisoning attacks) by thoroughly evaluating cloud systems |
Table B.5. Security Controls Related to Algorithms
| Subcategory | Identifier | Action or Control | OWASP and BIML Threats Mitigated | Why the Control Is Relevant |
|---|---|---|---|---|
| Adversarial robustness testing | A.ART-1 | Perform regular penetration tests and breach simulations to evaluate trust boundaries and access controls. |
|
Helps identify and mitigate vulnerabilities in trust boundaries and access controls, ensuring that adversarial inputs and unauthorized access attempts are effectively managed |
| A.ART-2 | Implement measures to detect and prevent unauthorized access and credential manipulation. |
|
Ensures that algorithms are trained to recognize and resist unauthorized access attempts and credential manipulation, thereby preventing adversarial inputs and unauthorized actions | |
| Change management | A.CM-1 | Authorize, test, and document all changes to AI models and environments. |
|
Ensures that changes are properly authorized, tested, and documented to prevent misconfigurations, maintain accurate inventory, and secure the supply chain from compromised components |
| Comprehensive security testing | A.CST-1 | Perform comprehensive security testing to identify vulnerabilities and weaknesses in the entire system. |
|
Helps identify and mitigate vulnerabilities (such as broken authentication, authorization flaws, security misconfigurations, and various injection and poisoning attacks) by thoroughly evaluating the system’s source code and configurations |
| Continuous security monitoring | A.CSM-1 | Implement continuous security monitoring and protection for all systems and networks. |
|
Helps detect and respond to misconfigurations, manage inventory, prevent denial-of-service attacks, and identify compromised components in the AI supply chain |
| Cryptographic protection and confidential computing | A.CPCC-1 | Use cryptographic keys to create a secure enclave for data during processing. |
|
Ensures that data and model processing occur in a secure enclave, preventing unauthorized access and extraction of sensitive information |
| A.CPCC-2 | Implement a Federal Information Processing Standard (FIPS) 140-2–compliant hardware security module for safeguarding and managing cryptographic keys and processing. |
|
Ensures secure storage and management of cryptographic keys, preventing unauthorized access and tampering, thereby protecting against authentication compromise, model theft, and supply chain vulnerabilities | |
| A.CPCC-3 | Use secret key managers and encryption to protect API keys from unauthorized access. |
|
Ensures that authentication mechanisms are not compromised, prevents misconfigurations that could expose keys, and secures the supply chain by safeguarding critical components | |
| Data integrity and verification | A.DIV-1 | Verify integrity of training datasets and model code using hashing, checksums, and digital signatures. |
|
Ensures that the training datasets and model code have not been tampered with, preventing unauthorized modifications and ensuring the integrity of the AI pipeline |
| A.DIV-2 | Implement digital signatures and hash verification to detect and prevent tampering. |
|
Ensures the integrity of algorithms by verifying that they have not been tampered with, preventing unauthorized modifications and ensuring the authenticity of the model components | |
| Data storage and access protection | A.DSAP-1 | Encrypt data and implement access controls to prevent unauthorized access and theft. |
|
Ensures that only authorized entities can access sensitive data and models, preventing unauthorized access, theft, and manipulation |
| A.DSAP-2 | Implement autonomous, irretrievable post-process deletion of such components as models and cryptographic keys. |
|
Ensures that sensitive components, such as models and cryptographic keys, are completely removed post-process, preventing unauthorized access, theft, or manipulation | |
| Infrastructure security and configuration management | A.ISCM-1 | Establish and enforce secure baselines and hardening guidelines for all systems. |
|
Ensures that systems are configured securely by default, reducing the risk of misconfigurations and vulnerabilities in the supply chain |
| Model security and anti-theft | A.MSAT-1 | Perform regular backups of model code, training data, and sensitive information for recovery. |
|
Ensures that model code, training data, and sensitive information can be restored if they are stolen or compromised, mitigating the impact of theft and supply chain attacks |
| A.MSAT-2 | Obfuscate model code to prevent reverse engineering and theft. |
|
Makes it significantly harder for attackers to understand, replicate, or steal the model, thereby protecting against unauthorized access and replication | |
| Model validation and security testing | A.MVST-1 | Conduct assessments to validate security controls before model integration. |
|
Helps prevent malicious data or inputs from compromising the model’s integrity and performance |
| A.MVST-2 | Create and use lists of known glitch tokens to scan output before model context inclusion. |
|
Helps prevent crafted inputs from causing unintended actions or bypassing safety measures, ensuring that outputs are validated before use | |
| Network security and isolation | A.NSI-1 | Segment networks to isolate sensitive infrastructure. |
|
Helps prevent unauthorized access to sensitive infrastructure, reducing the risk of sensitive information disclosure, model inversion, membership inference, and model skewing by isolating critical components |
| Package security | A.PS-1 | Verify digital signatures of packages before installation to ensure integrity. |
|
Ensures that packages have not been tampered with, preventing compromised components from being introduced into the AI development pipeline |
| A.PS-2 | Use secure repositories, such as Anaconda, that enforce strict security measures and vet packages. |
|
Ensures that only vetted and secure packages are used, preventing compromised components from entering the AI development pipeline | |
| Privacy by design implementation | A.PDI-1 | Implement data protection measures during the design phase of system development. |
|
Ensures that data protection principles are integrated from the start, reducing risks of data poisoning, sensitive information disclosure, and model inversion or skewing |
| Privacy-preserving techniques | A.PPT-1 | Implement differential privacy to protect individual records and prevent malicious knowledge transfer. |
|
Ensures that individual records in the training dataset remain confidential and resistant to extraction or manipulation, thereby preventing attackers from inferring sensitive information or injecting malicious data |
| Vulnerability management | A.VM-1 | Regularly scan AI systems for vulnerabilities and apply patches promptly. |
|
Ensures that vulnerabilities in AI systems are identified and mitigated promptly, preventing exploitation due to misconfigurations, compromised components, or skewed models |