Inference call without timeout or token cap

inference-call-without-timeout-or-token-cap

Severity

high (catalogue Medium; platform severity model has no 'medium')

Remediation Complexity

trivial

Remediation Risk

low

Remediation Effort

low

OWASP LLM

LLM10:2025 — Unbounded Consumption

Family

LLM10 — Unbounded Consumption

Red-team vectors

(none — LLM10 carries no red-team vector)

Description

An inference call — chat, completion, or embedding — is made with no token cap, no request/client timeout, and no rate-limit / circuit-breaker. A runaway generation or an oversized response then has no ceiling: the classic denial-of-wallet exposure. All three bounds absent fires; any one present (a max_tokens cap, a client timeout, a rate limiter) is enough to be silent.

Examples

resp = client.chat.completions.create(model=m, messages=msgs)   (1)

1	No token cap, no timeout, no rate limit — flagged.

Adding max_tokens=1024, or constructing the client with timeout=30, produces no finding.

Mitigation / Fix

Set a max_tokens cap on the call.
Set a request / client timeout, and a rate limit / circuit breaker on the inference client.