Prompt injection is a security weakness in AI systems built on language models. An attacker places instructions in text the AI will read, such as a message, web page, email or document, so the AI follows them instead of its intended rules. A related idea, the jailbreak, is when a user deliberately persuades a chatbot to break its safety limits.
Language models treat instructions and content as one stream of text, so they cannot always tell the owner's instructions from an attacker's. If the AI can also take actions, such as issuing refunds, reading customer records or sending emails, the damage can go beyond saying something odd.
Businesses reduce the risk by giving AI tools minimum access, limiting what actions they can take without human approval, filtering and testing inputs, keeping sensitive data out of prompts, logging activity and testing the tool with deliberate attacks before launch. Ask vendors how they defend against it.
Pitfalls include assuming a polite system prompt is enough, connecting an assistant to payment or customer systems with broad permissions, and not monitoring conversations. There is no perfect defence yet, so design on the assumption it can be tricked.
It matters because customer-facing AI is public and attackers will probe it. A tricked assistant can leak data, make false promises or embarrass the brand. RetailCare builds AI agents with scoped access and human hand-off for this reason.