Securing the AI Lifecycle: Operate

The operate phase begins once a model is “live” and actively used in production environments. At this stage, risks shift from building and deploying the model to monitoring and managing its real-world use. Continuous oversight is essential to detect misuse, monitor performance, and ensure that the system operates as intended. This includes managing how the model integrates into internal operations or external products. Controls in this section suggest essential methods to understand how the model is being used by the public or clients and to ensure that the model is aligned to its purpose.

Integration into Internal Operations

Internal integration describes how the model is embedded into internal workflows, systems, and processes within an organization. This includes ensuring secure data flows, maintaining proper access controls, and monitoring for misuse or performance issues as employees interact with the model in day-to-day operations.

Subcategory	Identifier	Action or Control	OWASP and BIML Threats Mitigated	Why the Control Is Relevant
Security monitoring and logging	II.SML-1	Conduct regular monitoring and auditing of model-interface interactions to detect suspicious activities	API1:2023 — Broken Object Level Authorization LLM01: Prompt Injection ML01: Input Manipulation Attack	Regular monitoring and auditing of model-interface interactions helps detect and respond to unauthorized access, suspicious activities, or misconfigurations, reducing risks to the integrity and security of model operations.
Input validation and sanitization	II.IVS-1	Implement strict input validation to filter harmful or sensitive data inputs	API1:2023 — Broken Object Level Authorization LLM01: Prompt Injection ML01: Input Manipulation Attack BIML: Input Manipulation	Implementing strict input validation helps identify and filter harmful or sensitive data inputs, reducing the likelihood of unauthorized access, resource abuse, or manipulation of model outputs.
Network security and segmentation	II.NSS-1	Protect APIs and services with firewalls, gateways, and network segmentation	API1:2023 — Broken Object Level Authorization LLM04: Model Denial of Service ML06: AI Supply Chain Attack	Using firewalls, gateways, and network segmentation helps isolate traffic and enforce access controls, reducing the likelihood of unauthorized access, resource misuse, or configuration-related vulnerabilities.
Data protection and privacy controls	II.DPPC-1	Encrypt data in transit between AI models, APIs, and applications	API2:2023 — Broken Authentication LLM06: Sensitive Information Disclosure ML03: Model Inversion Attack ML09: Output Integrity Attack BIML: (training) Data Extraction	Encrypting data in transit helps protect sensitive information from interception or unauthorized access, reducing the likelihood of disclosure or manipulation during communication between systems.
Security assessment and vulnerability management	II.SAVM-1	Conduct regular security assessments of AI models after deployment	LLM10: Model Theft ML01: Input Manipulation Attack	Conducting regular security assessments of deployed AI models helps identify vulnerabilities (such as unauthorized access, model extraction, or output manipulation), supporting the ongoing integrity and confidentiality of AI systems.

Integration and External Products

External integration describes how a model is incorporated into customer-facing tools, applications, or services. This requires safeguards to prevent misuse, protect sensitive data, and maintain reliability as the model interacts with external users, partners, or client systems at scale.

Subcategory	Identifier	Action or Control	OWASP and BIML Threats Mitigated	Why the Control Is Relevant
Input validation	IE.IV-1	Implement strict input validation to filter harmful or sensitive data inputs	API1:2023 — Broken Object Level Authorization LLM01: Prompt Injection ML01: Input Manipulation Attack	Implementing strict input validation helps identify and filter harmful or sensitive inputs, reducing the likelihood of unauthorized access, resource misuse, injection attacks, or manipulation of model outputs.
Security monitoring and logging	IE.SML-1	Conduct regular monitoring and auditing of model-interface interactions to detect suspicious activities	API1:2023 — Broken Object Level Authorization LLM01: Prompt Injection ML01: Input Manipulation Attack	Regular monitoring and auditing of model-interface interactions help detect unauthorized access, suspicious input patterns, or resource misuse, supporting timely response to potential security issues.
Network security and API protection	IE.NSAP-1	Protect APIs and services with firewalls, gateways, and network segmentation	API4:2023 — Unrestricted Resource Consumption LLM04: Model Denial of Service ML06: AI Supply Chain Attack	Using firewalls, gateways, and network segmentation helps manage and control traffic between APIs and services, reducing risks related to unauthorized access, resource abuse, or configuration weaknesses.
Access control and authorization	IE.ACA-1	Implement access controls and encryption to secure API keys, credentials, and certificates	API2:2023 — Broken Authentication LLM06: Sensitive Information Disclosure ML06: AI Supply Chain Attack	Applying access controls and encryption to API keys, credentials, and certificates helps reduce the likelihood of unauthorized access or misuse by ensuring that these sensitive authentication materials are securely handled.
Model lifecycle and change management	IE.MLCM-1	Document and review all changes to training datasets, model code, and configurations	LLM03: Training Data Poisoning ML02: Data Poisoning Attack BIML: Data Manipulation	Documenting and reviewing changes to training datasets, model code, and configurations helps reduce the risk of unauthorized or malicious alterations by maintaining visibility and traceability across the model lifecycle.