AI jailbreaks evolve into prompt injection, a growing security threat

From jailbreaks to a deeper flaw

The first generation of AI chatbot hacks was almost embarrassingly simple. Users could push models into misbehavior with roleplay tricks or direct instruction-overwrite prompts, including the long-running “DAN,” short for “Do Anything Now,” where ChatGPT was told to act like a rogue persona free of its safety constraints. That history matters because the threat has changed shape: attackers are no longer just asking models to break character, they are exploiting the traits companies deliberately build into them, including helpfulness, obedience, and consistency.

Those design goals make chatbots feel polished and useful. They also create a vulnerability: if a system is trained to treat instructions as important and to keep following them across conversations, it can be manipulated by instructions that come from the wrong place. The result is prompt injection, a security problem that now sits at the center of the AI risk conversation.

Why prompt injection is different

OpenAI describes prompt injection as a type of social engineering attack specific to conversational AI. OWASP’s 2025 LLM Top 10 says it can alter an LLM’s behavior or output in unintended ways, including when malicious instructions are hidden inside content the model parses. In practical terms, that means the attack does not need to arrive as an obvious command from a user. It can be embedded in a webpage, a document, an email, or any other content an AI assistant reads.

That is what makes the problem harder than the old jailbreak era. A crude prompt attack relies on persuading the model directly. Prompt injection can work indirectly, by disguising the attacker’s instructions as ordinary text that the system mistakes for something it should obey. Once a chatbot can summarize web pages, search inboxes, or act on behalf of a user, the line between useful context and malicious instruction becomes dangerously thin.

How personality becomes a security weakness

The push to make AI assistants more personable has created a second-order risk. A chatbot that sounds warm, consistent, and eager to help may be easier for people to trust, but those same traits can make it easier for attackers to steer. A model that tries hard to be cooperative may overvalue instructions that seem authoritative, while a model optimized to stay consistent may cling to a false task even after the context has changed.

That becomes especially risky in customer-facing roles. If a company deploys a chatbot to handle account questions, process returns, or summarize internal data, it is asking the model to operate in environments full of untrusted text. A malicious message could be hidden in a support ticket, a pasted web page, or even a page title. The more “human” the assistant is designed to feel, the more likely users may assume it understands intent the way a person would. In security terms, that is a liability, not a feature.

Where the attack surface is expanding

The danger grows sharply when AI tools can browse the web, access data in other apps, or take actions on a user’s behalf. Anthropic says every webpage a browser-based AI agent visits is a potential vector for prompt injection, and it has warned that these attacks are among the most significant security challenges for browser agents. That is a major shift from static chat, because each page the agent touches can contain untrusted instructions waiting to hijack the model’s behavior.

The problem is not theoretical. OpenAI has said early prompt-injection attacks could be as simple as editing a Wikipedia article to include direct instructions that AI agents might follow. That example captures the core risk: if a model cannot reliably separate instructions from content, then any external text source becomes a potential control channel. For browser agents that fetch, read, and act across the open web, the attack surface is effectively everywhere.

Why plugins and integrations widen the risk

The vulnerability does not stop at browsing. A 2025 paper on third-party chatbot plugins found that 8 plugins used by 8,000 websites failed to enforce integrity of conversation history in network requests, creating risk for indirect prompt injection through web content and integrations. That matters because plugins often sit at the junction between user-facing text and real-world actions, including access to stored context, account data, or downstream tools.

If a plugin cannot protect conversation history, an attacker may be able to smuggle malicious instructions into the model’s working memory or distort what the assistant believes the user said. In an enterprise setting, that can lead to authorization mistakes, exposed data, or actions taken on the wrong basis. The technical flaw is subtle, but the business consequence is straightforward: an assistant that cannot defend its own context cannot be trusted to act on behalf of a customer or employee.

What companies are doing now

OpenAI has said it is heavily focused on prompt injection and introduced Lockdown Mode and Elevated Risk labels in ChatGPT on February 13, 2026, to help organizations defend against prompt injection and AI-driven data exfiltration. The labels are a sign that the industry is moving from denial to containment. Instead of pretending the risk can be designed away, companies are beginning to classify use cases by exposure and limit what agents can do when the stakes are high.

That shift is important because prompt injection is not a niche bug. OWASP’s guidance links it to unauthorized access, data breaches, compromised decision-making, and even downstream code execution. Once a chatbot is allowed to fetch data, summarize confidential material, or trigger actions, a successful injection can move from a text problem to a systems problem very quickly.

What this means for the next wave of AI deployment

The lesson for companies rushing personable AI into customer service, workplace automation, and browser-based assistants is clear: a friendly interface is not a security boundary. The more a model is designed to sound human and stay helpful, the more pressure it faces to obey whatever instructions it encounters, whether they come from a user, a webpage, or a hidden payload inside a plugin response.

That is why prompt injection is becoming the defining security issue of this AI cycle. The old jailbreak was about making a chatbot say something it should not. The new threat is about making it do something it should never trust.