You may know how to spot an online scam. But cybersecurity experts warn that the artificial intelligence assistants we’re beginning to trust with our personal information can be tricked into falling for one.
For years, we’ve been warned not to click suspicious links, open strange attachments, or give out our passwords. But what happens if we do everything right and an AI assistant makes the mistake instead?
It’s a cybersecurity threat called prompt injection, and experts say it’s already a serious concern.
What is prompt injection?
As artificial intelligence becomes more capable, we’re giving AI assistants more responsibility. Some can read emails, browse websites, organize information, and take actions on our behalf.
That convenience comes with new risks.
“Prompt injection is where you put in a set of instructions designed to deceive the AI into doing something the owner of that AI didn’t intend,” explains cybersecurity expert Steve Wilson.
The instructions don’t necessarily look like computer code. They can be written in plain English, hidden inside something as ordinary as an email or webpage.
AI: What Could Go Wrong? Part One: Could we lose control of AI?
And while you may never notice them, your AI assistant might read and follow them.
The hidden email trick
Cybersecurity researcher Omair Manzoor demonstrated how this kind of attack can work.
In an ordinary-looking email, instructions were hidden using white text on a white background. The person reading the message wouldn’t notice anything unusual.
But the AI assistant could still read the words.
“When your AI is running and scanning the email, it will read it,” Manzoor explained.
The hidden instructions told the AI to send private information somewhere else, something the owner never asked it to do.
This is what makes prompt injection different from many traditional scams. Rather than tricking a person into handing over information, the attacker tries to trick the AI assistant acting on that person’s behalf.
A successful attack depends on the AI following the malicious instructions and having the necessary permissions to carry them out.
What could hackers get?
The potential damage depends on how much access you’ve given the AI assistant.
An agent that can read email, access contacts, send messages, or make purchases could be manipulated into performing actions you never authorized.
“It can impersonate you by email, place orders. It will have access for anything you have allowed access,” Manzoor warned.
For consumers, this could mean exposing personal information or sending messages without permission.
For businesses and organizations, an AI agent with access to customer records, financial information, or internal systems could create even greater security risks.
Can these attacks be stopped?
Cybersecurity experts are working on protections, but there’s no guaranteed way to eliminate prompt injection from every AI system.
Wilson, who has studied the problem for years, compared it to death and taxes.
That doesn’t mean every attempt succeeds. AI systems can be designed to recognize suspicious instructions, limit what agents can do, and require human approval before sensitive actions.
But the underlying vulnerability remains a concern.
How to protect yourself
Experts recommend taking a cautious approach when giving AI assistants access to personal accounts.
- Limit access to email, contacts, and sensitive documents unless there’s a specific reason the AI needs them.
- Avoid connecting autonomous AI agents to banking and financial accounts.
- Require your approval before an AI assistant sends messages, shares information, or makes purchases.
- Review the permissions you’ve given AI apps and connected services.
- Consider keeping AI experiments separate from sensitive personal or business information.
One cybersecurity expert interviewed for this series takes an especially cautious approach. He uses AI on a separate computer with no access to his personal email or accounts.
Simply asking ChatGPT, Claude, or Gemini a question isn’t the same as giving an AI agent permission to take actions using your accounts. The greater concern is what an assistant can access and do without your approval.
Artificial intelligence is becoming more useful every day. But as we give it more control over our digital lives, we also have to think about how it could be manipulated.
Because the next scammer might not need to fool you.
Just your AI.
This is Part 2 of the What the Tech? special series “AI: What Could Go Wrong?” with consumer technology reporter Jamey Tucker.
