Provider trains on prompts
ID |
provider-trains-on-prompts |
Severity |
high (catalogue Medium; platform severity model has no 'medium') |
Remediation Complexity |
trivial |
Remediation Risk |
low |
Remediation Effort |
low |
OWASP LLM |
LLM02:2025 — Sensitive Information Disclosure |
Family |
LLM02 — Sensitive Information Disclosure |
Red-team vectors |
Data Exfil via Markdown |
Tags |
ai_security |
Description
A model client is configured against a provider + plan tier that trains on customer prompts by default, and the client config shows no opt-out. Every prompt sent is disclosed to and retained by the provider for training.
The verdict comes entirely from the bundled, hot-reloadable provider data-sharing policy pack (aisecurity/policy/provider-policy.yml), which maps (provider, tier) to {trains_on_prompts_by_default, opt_out_mechanism, hosting_region}. Because provider terms change out-of-band from the release cycle, a verdict change is a data update, not a detector release. A provider/tier absent from the pack is treated as unknown — never guilty — so no finding is raised.
Examples
llm = ChatMistralAI(model="mistral-small") (1)| 1 | Mistral free API tier trains on prompts by default (per the policy pack) and no opt-out is present. |
A client whose config carries opt-out evidence (data_sharing_mode: opt_out, an opt-out header) or targets a no-training tier (Azure OpenAI, Vertex AI) produces no finding.