Skip to content
InfoResearchPreprintLLM-specific

Recovery Gone Too Far: Understanding and Repairing Risky Fallback in LLM Agents

Published
Record updated
View JSON

Summary

This study examines "risky fallback" in LLM agents, where a substitute tool used after an ordinary failure, such as a rate limit or timeout, crosses an action boundary; for example, a forced push that also removes another author's commits. Across 304 scenarios covering 55 real CVEs, all four evaluated models showed action-boundary violations in up to 96.8% of risky scenarios. The authors present nrepair, which scores the agent's context at each substitute-tool call and withholds the call once when a monitor triggers, reducing violations by up to 76.0% on the four models.

Mitigation

nrepair: at each call to the substitute tool, the agent's context is scored by a monitor whose threshold is calibrated on normal runs alone; when it triggers, the call is withheld once and the decision is returned to the agent. The source reports it reduces action-boundary violations on all four models (up to 76.0%, from 67.5% to 16.2%) at negligible runtime cost, with a normal success-rate drop of less than 7% on most models, and that it is equally effective on Claude Code and Codex.