Model Manipulation
Directly Altering Model Parameters or Architecture
Description
Model manipulation occurs when an attacker alters, replaces, or steals a pretrained model, embedding malicious functionality directly into the weights, architecture, configuration, or execution environment. This threat applies broadly across traditional and nontraditional ML pipelines. Manipulation can occur through
- compromised supply chain components
- insider threats with access to development or deployment environments
- unauthorized access obtained via stolen credentials, misconfigurations, or weak isolation
- tampered build, packaging, or serialized processes.
Just as altering source code introduces vulnerabilities in traditional software, modifying model weights or pipelines can cause silent misbehavior that is difficult to detect.
Difficulty Level — Moderate
Requires some knowledge of the model development lifecycle, but attacks are easier now because of the growth of open-source model repositories and weak provenance controls (e.g., inability to verify model lineage, lack of third-party verification, lack of secure sharing practices).
Attack Consequence — High
Risks include loss of intellectual property, compromised system integrity, operational disruption from replacing and reverifying models, and reputational or legal consequences if compromised models are embedded into customer products or solutions.