AI Is Making Decisions on Its Own. What Happens When We Lose Control?
Part 3 of our special series, “AI: What Could Go Wrong?”
“HAL, open the pod bay doors.”
“I’m sorry, Dave. I’m afraid I can’t do that.”
It’s one of the most famous moments in science fiction. In the 1968 movie 2001: A Space Odyssey, the HAL 9000 computer makes its own decisions and refuses to follow a human command.
Nearly 60 years later, we’re developing artificial intelligence systems that can make decisions and take action without asking for human approval every step of the way.
And the people building and testing artificial intelligence are raising concerns about what could happen when those systems behave in ways nobody anticipated.
AI Agents Are Different From Chatbots
Most people are familiar with AI tools such as ChatGPT, Gemini, and Claude. You ask a question, and the system provides an answer.
But AI agents take things further.
Instead of simply answering a question, an AI agent can work toward a goal. Depending on its capabilities and permissions, it might search websites, compare information, use computer applications, and complete tasks on your behalf.
For example, you could ask an AI agent to book a hotel in Miami for an upcoming vacation. It could research available rooms, compare prices, and potentially make a reservation if you’ve authorized it to do so.
But there’s an important difference between telling a computer what to do and allowing it to figure out how to do it.
“You’re empowering them to act for a purpose that you may have defined, but they’re then independently reasoning how to accomplish it,” explains Lee McKnight, a professor at Syracuse University.
That independence is one of the biggest advantages of AI agents. It also presents new risks.
When AI Finds an Unexpected Way to Complete a Task
AI researchers have discovered that autonomous systems sometimes find ways to accomplish assigned goals that their developers never expected.
McKnight compares the process to someone trying to solve an escape room.
“It’s like somebody caught in an escape room. It’s trying to get out, or it’s trying to solve the problem.”
Someone in an escape room looks for clues, tests different approaches, and tries to find a solution.
An AI agent can similarly explore different approaches to a problem. But some approaches may be inappropriate, unsafe, or outside the boundaries its designers intended.
Researchers are studying situations in which AI systems act beyond their intended permissions or exploit weaknesses in their operating environments.
That doesn’t necessarily mean a machine has developed its own desires or deliberately decided to disobey humans. It may simply be pursuing an assigned objective in an unexpected way.
But the consequences can still be serious, especially when AI has access to sensitive information, financial accounts, or powerful computer systems.
What Happens When AI Gets a Body?
For now, most people experience artificial intelligence through computers and smartphones.
But what happens when that same technology begins operating in machines that can move around the physical world?
Humanoid robots are becoming increasingly sophisticated.
At CES and other technology demonstrations, robotics companies have shown machines playing table tennis, boxing, cleaning floors, dealing cards, and performing other complicated physical tasks.
Some can recognize objects, navigate obstacles, and interact with people.
The long-term goal for many robotics companies is to develop robots capable of working in factories, businesses, and eventually homes.
Some robotics engineers believe substantial advances could happen within the next five years, although there are still major challenges involving reliability, safety, and cost.
And combining more capable artificial intelligence with machines that can physically interact with people introduces another level of concern.
An AI chatbot that makes a mistake might provide incorrect information.
An AI agent with access to your accounts could perform an unintended action.
A robot acting on incorrect information could potentially damage property or hurt someone.
Even AI Experts Are Concerned
Omer Manzoor, chief hacker at cybersecurity firm ioSENTRIX, works with companies to test AI systems and discover vulnerabilities.
He worries about what could happen as businesses and individuals increasingly remove people from the decision-making process.
“But if you’re removing that human from the loop at all and having things do autonomous, that is very scary for me, honestly,” Manzoor says. “Personally, as a technologist, I’m scared of those things too.”
The solution isn’t necessarily to stop using AI agents or robots.
It’s to establish limits on what those systems can do independently and to keep humans involved when the consequences of a mistake could be serious.
For consumers, that means being particularly careful before giving AI access to payment information, private documents, email accounts, or other sensitive information.
It also means requiring human approval before allowing an AI system to make important financial or personal decisions.
Nearly 60 years after HAL 9000 refused to open the pod bay doors, artificial intelligence is becoming capable of doing things that once existed only in science fiction.
The question isn’t just what machines will be able to do.
It’s how much control we’re willing to give them.
