A software agent that sends an email to the wrong recipient produces a problem that is solved by a correction email. A robot that drives a car into an obstacle produces a problem that is not solved by anything. The difference is reversibility, and it changes everything about how the agent is designed, operated, and governed.

This piece is about what changes when the agent's output is a physical action rather than a file or a message. It covers the failure cost structure, the latency problems, the perception pipeline, the safety requirements, and the honest limits of what current AI can do in physical contexts. The goal is not to discourage the use of AI in robotics but to be precise about what kinds of problems are suited to it and what kinds are not.

What changes when the agent is physical

The output of a software agent's action is a file, a message, a database record — something that can be overwritten, deleted, or corrected. The output of a physical agent's action is a movement, a force, a temperature change, an electrical signal — something that has already happened in the world and cannot be taken back.

This single difference changes the failure cost structure. A software agent that loops can be stopped. A physical agent at speed, at altitude, or in contact with a human cannot always be stopped without consequences. A software agent that makes a wrong decision produces a wrong file. A physical agent that makes a wrong decision produces a physical event.

The feedback loop is also different. A software agent observes the result of its actions through tool outputs — the file was written, the API returned a 200, the message was delivered. A physical agent observes the result of its actions through sensor data — the arm is at this position, the force sensor reads this value, the camera sees this image. Sensor data is noisy. Tool outputs are usually clean. The agent that reasons under sensor noise is working in a fundamentally different environment than the agent that reasons over clean text.

The latency problem

Software agents run at millisecond speed. A model call that generates a text response in 500 milliseconds is considered slow. A robot arm moving at 1 meter per second covers 0.5 meters in 500 milliseconds. A car at 60 miles per hour covers 13.4 meters in 500 milliseconds.

The latency problem is that the agent's "think" step must complete in time for the "act" step to be safe. In software, slow thinking is an inconvenience. In physical control, slow thinking can be a collision.

The implications are concrete. A physical agent that uses a large, slow model for real-time control will fall behind the physics of its environment. The model that is best at planning may be too slow for control. The model that is fast enough for control may not be good enough at planning.

In practice, physical agent architectures separate the slow path from the fast path. The slow path — mission planning, scene understanding, goal selection — uses the best available model. The fast path — reactive control, motor execution, sensor fusion — runs on fast, narrow models or classical controllers that do not depend on a large language model. The large model suggests; the fast controller executes.

The piece on inside the agent loop covers the loop stages that become time-bounded in a physical context.

The perception problem

A software agent "sees" by reading text. The text is usually correct — a file contains what was written in it, an API returns what was requested. A physical agent "sees" through cameras, lidar, IMU, GPS, force sensors, and proximity sensors. Each of these has a failure mode.

Camera images are affected by lighting, occlusion, motion blur, and lens artifacts. Lidar is affected by reflective surfaces and weather. GPS is affected by signal obstruction and multipath errors. Force sensors drift. Each sensor is a probabilistic estimate of the world, not a ground-truth observation.

The reliability gap between text and sensor data is the core perception problem in physical AI. An agent that reads a document and summarizes it is working with clean, accurate data. An agent that reads a camera image and decides whether to drive through an intersection is working with noisy, incomplete, possibly misleading data.

The mitigations are sensor fusion and redundancy. Multiple sensors that agree on a reading are more reliable than a single sensor. A physical agent that uses cameras, lidar, and IMU together has a better picture of the world than one that uses any single sensor. The cost is complexity: sensor fusion pipelines are harder to build and harder to debug than single-sensor pipelines. Current vision-language models have narrowed this gap substantially — they can read a camera image and produce reasonable descriptions of scene content, obstacles, and intent. But they are not yet reliable enough to serve as the sole perception system for high-speed, safety-critical navigation without classical sensor preprocessing in front of them.

The safety problem

A software agent can be paused. The process is stopped, the model call is cancelled, the file write is interrupted. Pausing a physical agent is not always possible without consequences.

A robot arm that is paused mid-swing holds its position — unless the hold creates a hazard. A drone that is paused at altitude hovers — unless wind or battery state makes hovering unsafe. A vehicle that is paused on a roadway creates an obstacle. The physical world does not have a pause button that is free of consequences.

The safety patterns that apply to physical agents are different from software patterns:

The core safety patterns are: geofencing to bound where the agent can go; force limits so the agent cannot exceed safe physical thresholds regardless of what the planner decides; redundant sensors for independent measurement of the same quantity; fail-safe defaults that stop and wait for human input when uncertain; and real-time monitoring that overrides the agent's outputs when they exceed safe bounds. Each pattern operates independently of the agent's reasoning — the agent's recommendation and the safety system's override are separate channels.

The piece on evaluation, safety, and governance covers the governance framework that physical agents require, including the human-in-the-loop patterns that are mandatory in any physical deployment.

The verification problem

In software, the eval suite is the verification. Unit tests, integration tests, property-based tests — the test suite tells you whether the code works. The tests run in seconds or minutes, they are repeatable, and they cost almost nothing.

In the physical world, the eval suite is real-world testing. A self-driving car that has driven 10 million miles under human supervision has more data than any simulation. But 10 million miles of real-world testing takes years, costs millions of dollars, and cannot be run in a loop. Closed-course testing facilities — proving grounds with controlled scenarios, instrumented vehicles, and repeatable conditions — are the intermediate step. They are faster and cheaper than public-road testing but still expensive and limited in scenario variety.

Simulation is the standard mitigation. Simulated environments let agents practice in a virtual world where crashes do not destroy anything. Simulation is cheap, fast, and repeatable. The problem is sim-to-real transfer — behaviors that work in simulation do not always transfer to the real world. The gap between simulated physics and real physics, between simulated sensors and real sensors, means that a policy trained in simulation needs careful tuning before it works on hardware.

Sim-to-real transfer is an open research problem. It is an active area of work, and the gap is narrowing. But it has not closed. A physical agent that passes a simulation eval is not guaranteed to pass a real-world eval. The simulation must model the relevant physics accurately enough that behaviors learned in simulation generalize — which means the simulation fidelity must be high for the specific scenarios that matter most.

The loop changes

The software agent loop is: read, think, act, observe. The physical agent loop is: perceive, plan, act, sense, update. The difference is not cosmetic.

The practical consequence is a two-level architecture. The high-level planner uses a capable model and runs at a slow rate — once per second or once per ten seconds. The low-level controller uses a fast model or a classical controller and runs at the speed of the physics — 100Hz, 1kHz, or faster. The large model suggests; the fast controller achieves. The planning model and the control model are different models with different latency requirements.

What AI can and cannot do in physical contexts

The honest picture: AI is good at perception and planning in physical contexts. AI is not yet reliable at real-time motor control, low-latency sensor fusion, or safety-critical decisions that must execute within hard latency bounds.

What AI is good at: scene understanding from camera and lidar, natural-language instruction following, path planning in known environments, anomaly detection from sensor streams, and learning manipulation policies for specific tasks in controlled settings.

What AI is not yet reliable at: real-time reactive control, guaranteeing safety under distributional shift, operating in unstructured environments at speed, and any task where a wrong decision has irreversible physical consequences without a human in the loop.

The practical implication: use AI for perception and planning; keep humans in the loop for real-time control and safety-critical decisions.

The operator's posture

The mental model for physical agent operation: treat the agent as a junior engineer who is very fast, very knowledgeable, but has no common sense about physical consequences. The agent's recommendation is usually good. The agent cannot be counted on to catch the edge case where the recommendation is dangerous.

The physical system should be designed to ignore the agent's recommendation when safety demands it. This is not a failure of the agent — it is the correct design. The agent reasons; the system acts. The system has a safety layer that operates independently of what the agent recommends.

What this is not

This piece is not an argument against using AI in robotics. The perception and planning capabilities of modern AI are making physical agents significantly more capable than they were three years ago. The trajectory is toward more AI in physical systems, not less.

This piece is not a technical guide to robot programming. The mechanics of motor control, sensor calibration, and actuator design are outside the scope of a conceptual manual. The scope here is the agent architecture and the operator's design decisions.

This piece is not a safety guarantee. Physical agent safety requires domain-specific analysis by people with expertise in the specific hardware, environment, and failure modes of the system in question. The patterns described here are starting points, not certifications.

See also