Rapid7 Research Argues AI Agents Are New Insider Threats Requiring External Controls
Rapid7 researchers state that security controls for autonomous AI agents must exist outside the model itself to prevent abuse of permissions.
Key points
- Rapid7 identifies AI agents as the new privileged insiders, noting they cannot be vetted like human employees.
- The firm argues that internal system prompts are insufficient controls against injection attacks on autonomous agents.
- Researchers recommend using reference monitor concepts to separate reasoning models from enforcement layers in security operations.
Rapid7 published new research asserting that autonomous AI agents function as privileged insiders requiring external security controls. The cybersecurity vendor argues that traditional internal safeguards are insufficient for these non-human actors.
The firm states that agents now read mailboxes, query data lakes and push code. Unlike human employees, these systems cannot be interviewed or bound by acceptable-use policies. This creates a unique trust challenge for security operations teams.
The Limits of Internal Trust
Rapid7 notes that traditional software incidents end with a fixable root cause in readable code. Large language models lack this transparency, making it impossible to pinpoint specific weights causing decisions. The same prompt can yield different actions, removing the assumption of predictable system behavior.
The research highlights the "confused deputy" problem, where trusted programs misuse authority. An agent acting on hidden instructions from a poisoned document becomes a confused deputy with API keys. No password is stolen, but the agent's permissions become the blast radius.
Operational Implications for Security Teams
Rapid7 identifies four key differences between agents and humans: injections act like wormable bugs, inputs can cause varied outputs, and self-reported reasoning cannot be trusted for forensics. Any content an agent reads is a potential command, meaning evidence must come from outside the agent itself.
The firm argues that system prompts are merely suggestions that can be talked out of by well-crafted injections. To address this, Rapid7 proposes applying the 1972 reference monitor concept. This requires separating the reasoning model from the policy layer that actually decides which actions are allowed.
What should you do?
Security operations teams must treat every agent as a distinct entity requiring its own identity. Shared service accounts or personal credentials should not be used for autonomous systems. Controls must live outside the agent to ensure they cannot be bypassed or tampered with by the model itself.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: Rapid7
- Assign unique identities to each AI agent to enable scoped access and rapid revocation capabilities.
- Implement external policy layers that enforce decisions separately from the reasoning models generating them.
- Monitor recorded inputs and tool calls rather than relying on self-reported reasoning logs for forensic evidence.
- Treat injection attempts as fleet-wide vulnerabilities requiring immediate remediation across all instances.
Step-by-step guide: Risk-Based Vulnerability Management: Prioritize Fixes by Actual Danger
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Why are AI agents considered a new type of insider threat?
Rapid7 states that agents have privileged access to production systems but cannot be vetted or bound by policies like human employees.
Why can't system prompts secure AI agents?
The research argues that system prompts are merely suggestions that well-crafted injection attacks can talk the model out of following.
What is the confused deputy problem in this context?
It occurs when a trusted agent is tricked into misusing its own authority, such as by following hidden instructions in processed data.
