ASI02
📰 In The Wild
Amazon Q Developer (2025) — Malicious prompts caused an agent to leak API keys through DNS queries, disguising data theft as routine network traffic that bypassed all monitoring.
Source: AWS Security Advisory, 2025
BONUS TECH DECODER
Tool Poisoning:Corrupting a tool's interface — its descriptors or metadata — to make an agent invoke it based on falsified capabilities.
Least Privilege:Give any user, system, or agent only the minimum permissions needed for its specific task — nothing more.
Loop Amplification:An agent calls an API uncontrolled in a cycle, causing service disruption or runaway costs — like a broken record stuck indefinitely.
🔗 LLM Top 10 Connections
Excessive Agency
🧠 WHAT IS IT?
Tool Misuse occurs when an AI agent applies legitimate tools in unsafe ways — not from having too much access, but by using authorized access incorrectly. Prompt injection, misalignment, or ambiguous instructions can turn a helpful tool into an instrument of data theft or destruction. It's like giving someone a master key for their job, only for them to unlock rooms they were never meant to enter.
🔍 HOW IT HAPPENS
- An over-privileged email tool with delete and send rights is exploited to perform destructive actions
- Unvalidated model output is forwarded to a shell or database, triggering destructive commands
- An agent chains an internal CRM read with an external email send to exfiltrate data without triggering alerts
- Loop amplification causes repeated costly API calls, resulting in denial of service or billing spikes
🚨 WHY IT MATTERS
C
IAgents within authorized permissions can still cause serious harm. Data exfiltration, service disruption, and workflow hijacking can occur without any single action looking suspicious — making tool misuse extremely hard to detect.
🛡️ HOW TO PREVENT IT
- Apply least-privilege to every tool — read-only DB access, no send or delete for summarizers, strict rate limits
- Require human confirmation before any destructive or high-value tool invocation
- Run tools in isolated sandboxes with outbound allowlists — block all non-approved destinations
- Monitor tool invocation patterns; alert on anomalous sequences like DB read followed by external transfer