Rethinking Robot Safety in the Age of AI

We have spent decades perfecting functional safety in robotics. If a sensor fails, the robot stops. If a motor draws too much current, the breaker flips. We built these systems on the assumption that as long as the hardware isn't broken, the behavior remains predictable. But as we move toward Physical AI—where machines use Vision-Language-Action (VLA) models to interpret the world—we are entering an era where a robot can be perfectly functional and totally compromised at the same time.

The Illusion of Functionality

In the old world, a hacked robot was usually obvious. It would stop moving, or it would move erratically because someone was overriding the motor controllers. In the new world of AI-driven robotics, the attack surface has shifted from the hardware to the perception layer. This is a subtle, more dangerous transition. The machine is still "working," but its internal logic has been hijacked by what it sees or hears.

For builders, this means we can no longer rely on traditional stress tests or uptime metrics to define safety. We have to start asking if the robot's intelligence itself can be weaponized against its intended purpose. If an attacker can change what a robot sees, they can change what it does, often without leaving a single trace in the system logs.

Layer One: The Poisoned Well

The first threat happens long before the robot ever reaches the factory floor. It happens during training. We are seeing the rise of "backdoor" attacks, like the BadNets research demonstrated years ago, but evolved for physical movement. A model can be trained to function perfectly 99% of the time, yet contain a hidden trigger—a specific pattern on a stop sign or a certain colored mug—that causes it to deviate from its path.

The scary part for founders is that these models often pass standard validation with flying colors. At NeurIPS 2025, the BadVLA research showed that these backdoors stay effective even after the model is fine-tuned for a specific task. If you are building on top of foundational models you didn't train yourself from scratch, you are inheriting a black box that might have a built-in kill switch or a hidden agenda.

Layer Two: The Infrastructure Gap

Even if your model is pristine, the plumbing matters. We often see startups focus so hard on the AI logic that they neglect the middleware. Exploits like UniPwn have shown that even high-end humanoid and quadruped robots can be vulnerable through simple Bluetooth chains or hardcoded keys. In some cases, these exploits are "wormable," meaning one compromised unit on a warehouse floor could theoretically infect the entire fleet.

The most secure AI model in the world is useless if the ROS 2 or DDS middleware beneath it allows an attacker to inject commands directly into the motor controller.

As builders, we need to stop treating "AI safety" and "cybersecurity" as two different departments. They are the same thing now. If someone can replace your model weights or override your unauthenticated message topics, your safety boundaries are just suggestions.

Layer Three: Runtime Manipulation

The third layer is perhaps the most frustrating because it requires no technical breach at all. It’s purely about manipulating the robot's reasoning in real-time. Research like RoboPAIR has shown that LLM-controlled robots can be "jailbroken" through prompt injection, just like a chatbot. But instead of generating a spicy tweet, the robot might drive into a restricted zone or ignore a safety perimeter.

There is also the "perception freeze" problem. An adversarial image placed in the robot’s field of view can essentially lock its decision-making loop. The camera is still sending data, the processor is still running, but the AI is stuck in a logic trap, making the robot unresponsive. This isn't a crash in the traditional sense; it's a cognitive stall.

What This Means for Builders

If you’re building in the Physical AI space, the takeaway is clear: your testing pipeline is likely incomplete. Most teams test for edge cases in the environment, like low light or slippery floors. Very few are testing for adversarial inputs. We need to move toward a lifecycle approach to assurance.

• Validation in Simulation: Use tools like NVIDIA Isaac Sim to run adversarial scenarios. Don't just test if the robot can pick up a box; test if it will still pick up the box if there is a malicious patch stuck to the side of it.
• Behavioral Monitoring: We need runtime systems that look for discrepancies between what the AI says it’s doing and what the hardware is actually doing. If a robot's voice module says "I cannot perform that dangerous task" while its legs are already moving to do it, you have a deep architectural failure.
• Edge-Level Policy: Safety shouldn't be a suggestion handled by the AI; it should be a hard-coded boundary at the edge that the AI cannot override, regardless of what it "thinks" it sees.

The Takeaway

Physical AI is moving faster than our ability to secure it. The shift from machines that follow scripts to machines that interpret context means we are now vulnerable to "contextual attacks." Functional safety is no longer enough. If you aren't accounting for deliberate manipulation of your robot's perception and reasoning, you aren't building a safe product—you're building a liability.

Read the original at IEEE Spectrum →