Baseline Security Controls: Develop
The develop phase turns design intent into trained and evaluated models. Developmental security focuses on the environments, data, and processes used to train models and evaluate their behavior so that weaknesses do not propagate into production.
Elements of the develop phase include
- Model training: the end-to-end training pipeline, including data preparation, tuning, checkpoints, and compute that produces model weights and artifacts
- Evaluation: test-and-evaluation procedures (e.g., metrics, safety or knowledge hazard tests, red-teaming checks, regressions) used to assess model behavior before release.
Model Training
Model Training
| Subcategory | Identifier | Action or Control | Why the Control Is Relevant |
|---|---|---|---|
| Access control and governance | MT.ACG-2 | Enforce RBAC and least privilege to restrict access to ML repositories and training environments | Applying RBAC and least‑privilege controls to model repositories and training environments helps limit unauthorized access and reduce opportunities for misuse or tampering. |
| Infrastructure security management | MT.ISM-1 | Implement and maintain security measures for infrastructure, including data protection and monitoring | Proper configuration and monitoring of infrastructure support early detection of misconfigurations, help maintain accurate inventory, and reduce risks associated with insecure supply chain components. |
| MT.ISM-2 | Regularly scan and patch vulnerabilities in model training environments | Regularly scanning and patching vulnerabilities helps identify and address weaknesses in the model training environment, reducing the risk of exploitation through compromised components or misconfigurations. | |
| Privacy‑preserving techniques | MT.PPT-1 | Use anonymization, pseudonymization, and data minimization in training processes | Using anonymization, pseudonymization, and data minimization practices helps protect sensitive information and reduces data integrity and privacy risks across AI systems. |
| Supply chain and package security | MT.SCPS-1 | Sign Git commits with GNU Privacy Guard (GPG) keys and load only trusted serialized ML models | Verifying signatures on source code and loading only trusted serialized models helps strengthen integrity and authenticity controls, reducing the likelihood of compromised components entering the pipeline. |
| MT.SCPS-2 | Update and review secure packages before using them in virtual environments for development | Reviewing and updating packages before use helps ensure that only secure components are introduced, limiting the risk that compromised packages create vulnerabilities within the model. | |
| Training environment security | MT.TES-2 | Segment and isolate model training environments to ensure boundary protection | Segmenting and isolating model training environments helps reduce unauthorized access and limits the potential impact of compromised components on the training process. |
Evaluation
Evaluation
| Subcategory | Identifier | Action or Control | Why the Control Is Relevant |
|---|---|---|---|
| Adversarial robustness testing | E.ART-1 | Simulate attacks to identify security weaknesses in the model and systems | Simulating attacks helps identify vulnerabilities in how the model responds to crafted inputs, reducing the likelihood of successful manipulation or poisoning. |
| E.ART-2 | Test the model with adversarial examples to check for vulnerabilities | Testing the model with adversarial examples helps reveal and mitigate weaknesses where crafted inputs could mislead the model into making incorrect predictions. | |
| Continuous security monitoring | E.CSM-1 | Implement continuous security monitoring and protection for all systems and networks | Continuous security monitoring helps detect and respond to misconfigurations, supports inventory management, reduces the risk of denial‑of‑service events, and helps identify compromised components in the AI supply chain. |
To view the full list of security controls developed for each phase of the AI lifecycle, see Appendix B.