Operate

Table B.9. Security Controls Related to Integration into Internal Products

Subcategory	Identifier	Action or Control	OWASP and BIML Threats Mitigated	Why the Control Is Relevant
Access control and authorization	II.ACA-1	Restrict AI model interface access to authorized users only.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization LLM08: Excessive Agency ML05: Model Stealing Attack	Prevents unauthorized access to and manipulation of AI models and APIs, ensuring that proper authentication and authorization mechanisms are in place
Access control and authorization	II.ACA-2	Implement access controls and encryption to secure API keys, credentials, and certificates.	API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization API8:2023—Security Misconfiguration LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack	Ensures that sensitive data (such as API keys, credentials, and certificates) are securely handled, preventing unauthorized access and misuse
Anomaly detection	II.AD-1	Implement automated anomaly detection and block potentially malicious inbound and outbound traffic.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM04: Model Denial of Service ML01: Input Manipulation Attack ML08: Model Skewing	Helps identify and block unusual patterns in traffic that may indicate resource abuse, unauthorized access, or crafted inputs designed to degrade model performance or manipulate outputs
Anomaly detection	II.AD-2	Continuously monitor for unusual inputs or outputs indicating security issues during model operation.	LLM01: Prompt Injection LLM04: Model Denial of Service ML01: Input Manipulation Attack ML08: Model Skewing	Helps detect and respond to crafted inputs designed to manipulate model behavior or degrade performance
Data protection and privacy controls	II.DPPC-1	Encrypt data in transit between AI models, APIs, and applications.	API2:2023—Broken Authentication API6:2023—Unrestricted Access to Sensitive Business Flows LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML04: Membership Inference Attack ML09: Output Integrity Attack BIML: Input Extraction BIML: (training) Data Extraction	Ensures that sensitive information is protected from interception and unauthorized access, preventing disclosure and manipulation
	II.DPPC-2	Implement tokenization and pattern matching to detect and redact sensitive information before processing.	API6:2023—Unrestricted Access to Sensitive Business Flows LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML04: Membership Inference Attack	Ensures that sensitive information is detected and redacted before processing, preventing unauthorized access to and disclosure of confidential data
	II.DPPC-3	Verify model outputs to ensure that no sensitive information is leaked.	LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML04: Membership Inference Attack BIML: Input Extraction	Ensures that model outputs are checked to prevent the leakage of sensitive information, protecting against attacks that aim to extract or infer private data from the model’s responses
Incident response and security operations	II.IRSO-1	Monitor AI applications for unauthorized access and misuse; respond to security events.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization LLM08: Excessive Agency ML05: Model Stealing Attack ML10: Model Poisoning	Ensures detection of and response to unauthorized access or misuse, preventing unauthorized actions and protecting against unauthorized model extraction or manipulation
Incident response and security operations	II.IRSO-2	Provide clear reporting paths for harmful outputs and offer a bug bounty for jailbreaks.	LLM01: Prompt Injection LLM02: Insecure Output Handling LLM06: Sensitive Information Disclosure ML01: Input Manipulation Attack ML09: Output Integrity Attack	Ensures that harmful outputs and vulnerabilities are promptly reported and addressed, reducing the risk of prompt injection, insecure output handling, sensitive information disclosure, input manipulation, and output integrity attacks
Infrastructure security and configuration management	II.ISCM-1	Apply hardening standards to servers, APIs, and deployment environments for integrated AI systems.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows API8:2023—Security Misconfiguration LLM05: Supply Chain Vulnerabilities LLM07: Insecure Plugin Design	Ensures secure configurations and resource limitations to prevent misconfigurations, excessive resource usage, unauthorized access, insecure plugins, and compromised components
Infrastructure security and configuration management	II.ISCM-2	Use tools to check container configurations, dependencies, and runtime behaviors for vulnerabilities.	API8:2023—Security Misconfiguration LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that container configurations, dependencies, and runtime behaviors are free from vulnerabilities, preventing misconfigurations and compromised components in the AI development pipeline
Input validation and sanitization	II.IVS-1	Implement strict input validation to filter harmful or sensitive data inputs.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API3:2023—Broken Object Property Level Authorization API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM01: Prompt Injection LLM02: Insecure Output Handling LLM06: Sensitive Information Disclosure ML01: Input Manipulation Attack ML08: Model Skewing BIML: Input Manipulation	Ensures that harmful or sensitive data inputs are detected and filtered out, preventing unauthorized access, resource abuse, and manipulation of model outputs
Input validation and sanitization	II.IVS-2	Apply semantic filters and string-checking to identify and handle sensitive content.	LLM01: Prompt Injection LLM06: Sensitive Information Disclosure ML01: Input Manipulation Attack ML09: Output Integrity Attack	Ensures that sensitive content is identified and handled appropriately by applying semantic filters and string-checking, which helps prevent malicious inputs and outputs, and protects against the disclosure of sensitive information
Network security and segmentation	II.NSS-1	Protect APIs and services with firewalls, gateways, and network segmentation.	API1:2023—Broken Object Level Authorization API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery API8:2023—Security Misconfiguration LLM04: Model Denial of Service LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Helps prevent unauthorized access, resource abuse, and potential misconfigurations by isolating network traffic and enforcing strict access controls, thereby protecting against various attacks and ensuring secure integration into internal operations
Network security and segmentation	II.NSS-2	Restrict LLMs’ access to network resources, internal services, and APIs to mitigate insider threats.	API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery LLM07: Insecure Plugin Design LLM08: Excessive Agency	Limits the LLMs’ ability to autonomously interact with sensitive resources, prevents insecure plugin interactions, controls access to sensitive business flows, and mitigates SSRF risks
Operational security and change management	II.OSCM-1	Document and review all changes to training datasets, model code, and configurations.	LLM03: Training Data Poisoning LLM05: Supply Chain Vulnerabilities LLM10: Model Theft ML02: Data Poisoning Attack ML05: Model Stealing Attack ML06: AI Supply Chain Attack	Ensures that all changes to training datasets, model code, and configurations are documented and reviewed to prevent unauthorized or malicious modifications, thereby protecting against data poisoning, supply chain vulnerabilities, and model theft
Operational security and change management	II.OSCM-2	Maintain a centralized inventory for production ML models with governance and access control.	API9:2023—Improper Inventory Management LLM10: Model Theft ML05: Model Stealing Attack ML06: AI Supply Chain Attack	Ensures proper documentation, governance, and access control to prevent unauthorized access to, replication of, or compromise of ML models
Resource management and rate limiting	II.RMRL-1	Restrict the number of requests per source entity within a specified period.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM04: Model Denial of Service	Prevents excessive resource usage and potential denial-of-service attacks by limiting the number of requests from any one source entity
Resource management and rate limiting	II.RMRL-2	Dynamically monitor and manage resource allocation to prevent excessive consumption by any user or request.	API4:2023—Unrestricted Resource Consumption LLM04: Model Denial of Service	Prevents excessive resource consumption by dynamically managing and monitoring resource allocation, ensuring that no single user or request can degrade service performance
Security assessment and vulnerability management	II.SAVM-1	Conduct regular security assessments of AI models after deployment.	LLM10: Model Theft ML01: Input Manipulation Attack ML03: Model Inversion Attack ML04: Membership Inference Attack ML05: Model Stealing Attack ML09: Output Integrity Attack	Helps identify and mitigate vulnerabilities, such as unauthorized access, model extraction, and manipulation of model outputs, ensuring the integrity and confidentiality of the AI models
Security assessment and vulnerability management	II.SAVM-2	Run vulnerability scans on applications at the end of the continuous integration/continuous delivery (CI/CD) pipeline.	API8:2023—Security Misconfiguration API9:2023—Improper Inventory Management LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that any security misconfigurations, undocumented endpoints, or compromised components are identified and addressed before deployment
Security monitoring and logging	II.SML-1	Conduct regular monitoring and auditing of model-interface interactions to detect suspicious activities.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API3:2023—Broken Object Property Level Authorization API4:2023—Unrestricted Resource Consumption API5:2023—Broken Function Level Authorization API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery API8:2023—Security Misconfiguration API9:2023—Improper Inventory Management API10:2023—Unsafe Consumption of APIs LLM01: Prompt Injection LLM02: Insecure Output Handling LLM04: Model Denial of Service LLM06: Sensitive Information Disclosure LLM07: Insecure Plugin Design LLM08: Excessive Agency ML01: Input Manipulation Attack ML03: Model Inversion Attack ML09: Output Integrity Attack	Helps detect and respond to unauthorized access, suspicious activities, and potential misconfigurations, ensuring the integrity and security of model-interface interactions
Security monitoring and logging	II.SML-2	Implement tamper-evident logs for all input and output interactions to detect integrity attacks.	API8:2023—Security Misconfiguration LLM02: Insecure Output Handling ML09: Output Integrity Attack	Ensures that any unauthorized changes to input and output interactions are detected, helping maintain the integrity of the system and identify potential security misconfigurations

Table B.10. Security Controls Related to Integration and External Products

Subcategory	Identifier	Action or Control	OWASP and BIML Threats Mitigated	Why the Control Is Relevant
Access control and authorization	IE.ACA-1	Implement access controls and encryption to secure API keys, credentials, and certificates.	API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization API6:2023—Unrestricted Access to Sensitive Business Flows API8:2023—Security Misconfiguration LLM06: Sensitive Information Disclosure ML06: AI Supply Chain Attack	Ensures that sensitive data (such as API keys, credentials, and certificates) are securely handled, preventing unauthorized access and misuse
Access control and authorization	IE.ACA-2	Assign unique API tokens to applications and restrict model access to minimum necessary privileges.	API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization API6:2023—Unrestricted Access to Sensitive Business Flows LLM08: Excessive Agency ML05: Model Stealing Attack	Ensures that applications have unique tokens for authentication and restricts model access to necessary privileges, preventing unauthorized access and excessive permissions
Data protection and privacy controls	IE.DPPC-1	Implement tokenization and pattern matching to detect and redact sensitive information before processing.	API6:2023—Unrestricted Access to Sensitive Business Flows LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML04: Membership Inference Attack	Helps prevent sensitive information from being exposed or misused by detecting and redacting confidential content before it is processed
Data protection and privacy controls	IE.DPPC-2	Verify model outputs to ensure that no sensitive information is leaked.	LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML04: Membership Inference Attack BIML: Input Extraction	Ensures that the model outputs are checked to prevent the leakage of sensitive information, protecting against attacks that aim to extract or infer private data from the model’s responses
Incident response and security operations	IE.IRSO-1	Provide clear reporting paths for harmful outputs and offer a bug bounty for jailbreaks.	LLM01: Prompt Injection LLM02: Insecure Output Handling LLM06: Sensitive Information Disclosure LLM08: Excessive Agency ML01: Input Manipulation Attack ML09: Output Integrity Attack	Ensures that users can report harmful outputs and jailbreaks, allowing prompt identification and mitigation of issues related to prompt injection, insecure output handling, sensitive information disclosure, excessive agency, input manipulation, and output integrity attacks
Incident response and security operations	IE.IRSO-2	Monitor AI applications for unauthorized access and misuse; respond to security events.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API5:2023—Broken Function Level Authorization LLM08: Excessive Agency ML05: Model Stealing Attack	Ensures detection of and response to unauthorized access or misuse, preventing unauthorized actions and protecting against unauthorized model extraction or excessive permissions
Infrastructure security and configuration management	IE.ISCM-1	Apply hardening standards to servers, APIs, and deployment environments for integrated AI systems.	API4:2023—Unrestricted Resource Consumption API8:2023—Security Misconfiguration LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that servers, APIs, and deployment environments are configured securely to prevent misconfigurations, resource abuse, and vulnerabilities in the supply chain
Infrastructure security and configuration management	IE.ISCM-2	Use tools to check container configurations, dependencies, and runtime behaviors for vulnerabilities.	API8:2023—Security Misconfiguration LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that container configurations, dependencies, and runtime behaviors are secure and compliant with standards, preventing misconfigurations and vulnerabilities in the supply chain
Input validation	IE.IV-1	Implement strict input validation to filter harmful or sensitive data inputs.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API3:2023—Broken Object Property Level Authorization API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery API8:2023—Security Misconfiguration LLM01: Prompt Injection LLM02: Insecure Output Handling ML01: Input Manipulation Attack ML08: Model Skewing	Ensures that harmful or sensitive data inputs are detected and filtered out, preventing unauthorized access, resource abuse, injection attacks, and manipulation of model outputs
Input validation	IE.IV-2	Apply semantic filters and string-checking to identify and handle sensitive content.	LLM01: Prompt Injection LLM02: Insecure Output Handling LLM06: Sensitive Information Disclosure ML01: Input Manipulation Attack ML09: Output Integrity Attack	Ensures that sensitive content is identified and handled appropriately, preventing malicious inputs and outputs from causing unintended actions or revealing confidential data
Model lifecycle and change management	IE.MLCM-1	Document and review all changes to training datasets, model code, and configurations.	LLM03: Training Data Poisoning LLM05: Supply Chain Vulnerabilities ML02: Data Poisoning Attack ML06: AI Supply Chain Attack ML10: Model Poisoning BIML: Data Manipulation	Ensures that all changes to datasets, model code, and configurations are tracked and reviewed to prevent unauthorized or malicious modifications
Model lifecycle and change management	IE.MLCM-2	Maintain a centralized inventory for production ML models with governance and access control.	API9:2023—Improper Inventory Management LLM10: Model Theft ML05: Model Stealing Attack ML06: AI Supply Chain Attack	Ensures proper documentation, governance, and access control over production models, preventing unauthorized access or theft and ensuring integrity of the supply chain
Model protection and anti-theft	IE.MPAT-1	Implement a software interface for weight interaction to prevent illegitimate copying.	LLM10: Model Theft ML05: Model Stealing Attack BIML: Model Extraction	Ensures that interactions with model weights are mediated through a secure software interface, preventing unauthorized access to and copying of the model
Model protection and anti-theft	IE.MPAT-2	Replace deployed model with one below the capability threshold.	LLM10: Model Theft ML05: Model Stealing Attack ML10: Model Poisoning	Helps prevent unauthorized access to or theft of high-capability models, reduces the risk of model stealing attacks, and mitigates the impact of any potential model skewing by ensuring that the deployed model is less valuable or impactful
Network security and API protection	IE.NSAP-1	Protect APIs and services with firewalls, gateways, and network segmentation.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery API8:2023—Security Misconfiguration API10:2023—Unsafe Consumption of APIs LLM04: Model Denial of Service LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Helps prevent unauthorized access, resource abuse, and misconfigurations by controlling and monitoring traffic between APIs and services
Network security and API protection	IE.NSAP-2	Restrict LLMs’ access to network resources, internal services, and APIs to mitigate insider threats.	API6:2023—Unrestricted Access to Sensitive Business Flows API7:2023—Server Side Request Forgery LLM07: Insecure Plugin Design LLM08: Excessive Agency ML06: AI Supply Chain Attack	Limits LLMs’ ability to interact with sensitive business flows, prevents unauthorized remote resource fetching, reduces excessive permissions, ensures that plugins cannot access unauthorized resources, and mitigates supply chain vulnerabilities
Output validation and constraints	IE.OVC-1	Define model roles, capabilities, and limitations; enforce strict context adherence and ignore core instruction modifications.	LLM01: Prompt Injection LLM08: Excessive Agency ML01: Input Manipulation Attack	Ensures that the model adheres strictly to predefined roles and limitations, preventing it from executing unintended actions or responding to malicious inputs that attempt to alter its core instructions
Output validation and constraints	IE.OVC-2	Specify output formats, request detailed reasoning and citations, and validate with deterministic code.	API10:2023—Unsafe Consumption of APIs LLM02: Insecure Output Handling ML09: Output Integrity Attack	Ensures that outputs are validated and formatted correctly, preventing injection attacks, unsafe data handling from third-party APIs, and manipulation of model outputs
Resource management and rate limiting	IE.RMRL-1	Restrict the number of requests per source entity within a specified period.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM04: Model Denial of Service	Helps prevent resource exhaustion and service degradation by limiting the number of requests from any one source entity
Resource management and rate limiting	IE.RMRL-2	Dynamically monitor and manage resource allocation to prevent excessive consumption by any user or request.	API4:2023—Unrestricted Resource Consumption LLM04: Model Denial of Service	Prevents excessive resource consumption by dynamically managing and monitoring resource allocation, ensuring that no single user or request can degrade service performance
Security assessment and vulnerability management	IE.SAVM-1	Conduct regular security assessments of AI models after deployment.	LLM10: Model Theft ML01: Input Manipulation Attack ML05: Model Stealing Attack ML08: Model Skewing ML09: Output Integrity Attack	Helps identify and mitigate vulnerabilities that could lead to model theft, manipulation, or incorrect outputs
Security assessment and vulnerability management	IE.SAVM-2	Run vulnerability scans on applications at the end of the CI/CD pipeline.	API8:2023—Security Misconfiguration API9:2023—Improper Inventory Management LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that applications are checked for vulnerabilities and misconfigurations before deployment, preventing security flaws from being introduced into the production environment
Security monitoring and logging	IE.SML-1	Conduct regular monitoring and auditing of model-interface interactions to detect suspicious activities.	API1:2023—Broken Object Level Authorization API2:2023—Broken Authentication API3:2023—Broken Object Property Level Authorization API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM01: Prompt Injection LLM04: Model Denial of Service ML01: Input Manipulation Attack ML09: Output Integrity Attack	Helps detect unauthorized access, suspicious input patterns, and resource misuse, ensuring timely responses to potential security breaches
Security monitoring and logging	IE.SML-2	Implement tamper-evident logs for all input and output interactions to detect integrity attacks.	API8:2023—Security Misconfiguration LLM02: Insecure Output Handling ML09: Output Integrity Attack	Ensures that any unauthorized changes to input or output interactions are detected, helping maintain the integrity of the system and identify misconfigurations
Third-party security	IE.TPS-1	Conduct security audits quarterly to identify and mitigate transfer of learning vulnerabilities.	LLM05: Supply Chain Vulnerabilities ML05: Model Stealing Attack ML06: AI Supply Chain Attack ML10: Model Poisoning	Helps identify and address vulnerabilities in the integration and external product phase, preventing supply chain attacks, model theft, and skewing by ensuring the integrity and security of transferred models
Third-party security	IE.TPS-2	Update and review secure packages before using them for development.	API10:2023—Unsafe Consumption of APIs LLM05: Supply Chain Vulnerabilities ML06: AI Supply Chain Attack	Ensures that only secure and verified packages are used, preventing the introduction of vulnerabilities through compromised or malicious components
Threat detection	IE.TD-1	Continuously monitor for unusual inputs or outputs indicating security issues during model operation.	API6:2023—Unrestricted Access to Sensitive Business Flows LLM01: Prompt Injection LLM04: Model Denial of Service ML01: Input Manipulation Attack ML08: Model Skewing	Helps detect and mitigate prompt injections, denial-of-service attempts, input manipulations, adversarial examples, and unauthorized access to sensitive flows by identifying anomalies that deviate from expected behavior
Threat detection	IE.TD-2	Implement automated anomaly detection and block potentially malicious inbound and outbound traffic.	API4:2023—Unrestricted Resource Consumption API6:2023—Unrestricted Access to Sensitive Business Flows LLM04: Model Denial of Service ML01: Input Manipulation Attack ML08: Model Skewing	Helps identify and block unusual patterns that could indicate resource abuse, unauthorized access, or crafted inputs designed to disrupt or manipulate the system