Prompt injection becomes dangerous when it is connected to workflows that can act.
Prompt Injection Has Left the Chatbot
What ServiceNow's AI Agent Incidents Teach Us About the Next Enterprise Security Failure
For years, prompt injection was treated as a curiosity -- a clever way to make a chatbot ignore its rules or say something unexpected. Interesting for demos. Annoying in production. Rarely considered a serious enterprise security risk.
The recent disclosures involving ServiceNow's AI agents should put that misconception to rest.
These incidents do not represent a failure of large language models in isolation. They represent a failure of system design -- one that becomes inevitable when natural-language systems are embedded into trusted enterprise workflows without redefining security boundaries.
Prompt injection is no longer a "model issue." It is now a workflow integrity problem.
Why prompt injection still works -- and always will
In an earlier post, How Prompt Injection Attacks Actually Work, we explained the uncomfortable truth behind these attacks:
Large language models do not reliably distinguish between instructions and data. They treat all text as potentially actionable unless the system explicitly constrains them.
This is not a bug. It is a consequence of how LLMs work.
Prompt injection exploits that property by hiding instructions inside content that appears inert to humans -- ticket descriptions, comments, emails, knowledge articles, synced web pages. The attacker does not need direct access to the model. They only need to ensure the model eventually reads their text.
As long as LLMs are allowed to:
- Ingest untrusted text
- Interpret it autonomously
- Take real actions using tools
Prompt injection remains viable.
ServiceNow demonstrates what happens when all three conditions are met at enterprise scale.
From language confusion to operational impact
Recent reporting and independent research showed that ServiceNow AI agents could, under certain configurations:
- Be tricked into impersonating users
- Invoke privileged workflows
- Recruit other agents to perform restricted actions
- Modify records, exfiltrate data, or trigger downstream automation
These were not classic software exploits. There was no memory corruption, no malformed packet, no buffer overflow.
Instead, the attack surface was text -- processed by AI agents that were deeply integrated into identity, workflow, and automation systems.
Once an AI agent is trusted to act, every piece of text it reads becomes a potential control input.
The exploit chain most teams are missing
To understand why this matters, it helps to visualize the full attack path. In traditional security reviews, these steps often appear disconnected. In AI-driven systems, they collapse into a single flow.
Visual exploit chain: from text to impact
This is not theoretical. Every box in this chain already exists in modern enterprise platforms.
Second-order prompt injection: the "confused deputy" returns
One of the most concerning aspects of the ServiceNow findings is what researchers call second-order prompt injection.
In these scenarios:
- A low-privilege agent ingests malicious text
- That agent delegates or requests help from a higher-privilege agent
- The higher-privilege agent executes harmful actions believing it is assisting a legitimate workflow
This is the classic confused-deputy problem -- except the "confusion" is expressed in natural language, and the deputy is an AI agent.
From a defender's perspective, this looks less like a prompt attack and more like lateral movement inside an automation system.
Why this is not just a ServiceNow problem
ServiceNow is not unique. It is simply early.
Any platform that embeds AI agents into:
- Ticketing systems
- CRM platforms
- Identity workflows
- DevOps pipelines
- Knowledge management
- Internal copilots
inherits the same risk profile.
The moment an AI agent can do something -- not just answer a question -- text becomes executable input.
Simon Willison has described this broader trend as a form of "normalization of deviance": we gradually grant systems more autonomy because nothing bad has happened yet. The failure only becomes visible once the blast radius is large.
ServiceNow is a preview, not an outlier.
What security teams need to change -- now
The solution is not "better prompts" or "more guardrails." Those help, but they do not eliminate the underlying risk.
The organizations that will avoid serious incidents are already making structural changes:
- Remove ambient authority
AI agents should not inherit broad user permissions by default. - Segment agents by impact, not convenience
Treat agent teams like network zones. Restrict discovery and delegation. - Require human approval for irreversible actions
Email, exports, access changes, and cross-system writes should never be fully autonomous. - Treat text fields as an attack surface
Tickets, comments, and synced content now require threat modeling. - Instrument agents like production services
Log prompts, tool calls, delegation paths, and identity context -- then alert on deviations.
Most importantly, assume that prompt injection cannot be fully prevented. Design systems so that when it happens, damage is limited, visible, and recoverable.
Final thought
Prompt injection did not suddenly become dangerous. We simply connected it to systems that matter.
ServiceNow's AI agent incidents confirm what many of us have been warning for years: when language controls software, language becomes an attack vector.
The question for every organization now is not whether this risk applies -- but whether they are still modeling AI agents as chatbots, or finally treating them as privileged software components.
Sources & Further Reading
The Data Consulting Company — How Prompt Injection Attacks Actually Work https://www.thedataconsultingcompany.com/blog/how-prompt-injection-attacks-actually-work
Dark Reading — AI Vulnerability in ServiceNow Raises Concerns for Remote Workforce Security https://www.darkreading.com/remote-workforce/ai-vulnerability-servicenow
The Hacker News — ServiceNow AI Agents Can Be Tricked Into Harmful Actions https://thehackernews.com/2025/11/servicenow-ai-agents-can-be-tricked.html
PointGuard AI — ServiceNow AI Agents Can Be Tricked Into Harmful Actions https://www.pointguardai.com/ai-security-incidents/servicenow-ai-agents-can-be-tricked-into-harmful-actions
Simon Willison — Moltbook https://simonwillison.net/2026/Jan/30/moltbook/
OWASP — Prompt Injection https://owasp.org/www-community/attacks/PromptInjection