LLM output to shell or eval

ID

llm-output-to-shell-or-eval

Severity

critical

Remediation Complexity

medium

Remediation Risk

low

Remediation Effort

medium

OWASP LLM

LLM05:2025 — Improper Output Handling

OWASP ASI

ASI05:2026 (secondary ASI02:2026)

Family

LLM05 — Improper Output Handling

Red-team vectors

Sandbox Escape / RCE

Tags

ai_security

Description

Model output — attacker-influenced by definition — reaches a code-execution sink: eval / exec, os.system, subprocess.*(…​, shell=True), or a JavaScript Function / child_process.exec. This is the highest-impact output-handling class: the model can be steered to emit a payload that executes as code (RCE). Server-side template renders (Jinja2, Mako, Handlebars, EJS, …) are the sink surface of the sibling template-injection detector.

Phase-1 posture is the sink-present shape scoped to an LLM-derived argument, so a sink fed a constant or a non-LLM value produces no finding; the per-finding confidence is medium, reflecting the approximation. The Phase-1.5 TaintFlowDetector upgrades it to flow-confirmed.

Examples

response = client.chat.completions.create(...)
subprocess.run(response.choices[0].message.content, shell=True)   (1)
1 Model output executed in a shell — flagged critical.

subprocess.run("ls -la", shell=True) (constant argument) produces no finding.

Mitigation / Fix

  • Never pass model output to a shell / eval / exec or an unescaped template.

  • Constrain output to a strict allowlist of actions; run in a sandbox; require human approval for side-effecting operations.