A Humanoid Dodged 95% of Thrown Balls Using Its Own Head Camera, Outperforming Tidier Safety Maths

A ball comes in at a humanoid robot standing in a Caltech lab. It has no motion-capture rig watching the room, no external tracker, no operator feeding it coordinates. All it has is a camera on its head and a segmentation model that decides which pixels are ball. Out of that it has to work out where the thing is, continuously, and whether its own body is about to be in the way. Then it moves.

The system is called PAC-MAN, and it comes from Lizhi Yang, Junheng Li and Aaron D. Ames at Caltech's AMBER Lab, posted to arXiv on July 30. Ames is one of the better-known researchers working on control barrier functions, which makes the paper's central result more interesting than it would be from anyone else: the tidier, more mathematically precise way of drawing the safety boundary is the one that does worse when the robot can only glimpse the danger.

Start with what the robot is given. The abstract is precise about it: the deployed policy sees the ball only as segmentation-masked depth from a head-mounted camera. That means a depth image, in which each pixel records how far away that bit of the world is, with a segmentation model masking out everything that is not ball. No position handed in from outside, no velocity estimate from a tracking system in the room. Whatever the camera catches is the whole of what the robot knows about the incoming object.

What a control barrier function does, and what it does not do here

A control barrier function is a piece of maths that encodes a region the system must stay inside, and turns that region into a constraint on what the controller is allowed to command. Used as an online filter, sitting between the policy and the motors and vetoing unsafe commands at every timestep, it is a well-studied way to enforce a safety condition at runtime. That is not how it is used here, and nothing else about this paper matters as much as that difference.

PAC-MAN builds on CBF-RL, a framework the same lab published last year, whose explicit selling point is that the barrier is applied during training rather than at deployment. The reinforcement-learning policy is filtered while it is learning, so it internalises the constraint, and the robot then runs without an online safety filter. That buys practical things: less conservative behaviour, faster convergence, no safety infrastructure to maintain on the machine. What it does not buy is a runtime certificate. The deployed policy is a learned controller that was trained under barrier guidance, and nothing about it is guaranteed once it is switched on.

The paper is straightforward about this. Its own description is "training-time CBF guidance", and the word guarantee does not appear in its text. If a guarantee turns up in a description of this work, someone other than the authors will have put it there.

Where to draw the line

The design question PAC-MAN actually studies is where to put the boundary. One formulation, Joint-CBF, is the more precise of the two and gives the best performance when the ball's position and velocity are known accurately. The other, which the paper calls a lightweight Link-CBF, defines clearance to the body's links instead: coarser, and correspondingly less demanding about knowing exactly where the ball is. Training also includes an adversarial motion regulariser, which pushes the policy to keep behaving when things do not go the way it expects.

Then the researchers took away the accurate ball states. Under fixed-camera observation, with the barrier used only as training guidance, Joint-CBF degrades. It recovers if you give the robot a ball-tracking gimbal, so the camera can follow the ball, or if you put a privileged runtime filter back in. The paper's phrase for the finding is that usable barrier structure depends on perceptual observability. Put plainly: how finely you can afford to specify a safety boundary is set by how well the robot can see, and specifying it more finely than the sensor supports makes things worse rather than better.

The camera did better than expected

A second result is easy to miss and worth stating on its own. On the controlled benchmark, with seeded throws, the camera-only policy came within a few points of a privileged state oracle, a version of the system handed perfect knowledge of where the ball was. The paper's conclusion from that is direct: a fixed onboard camera alone is adequate for evasion. Nobody has to bolt a motion-capture system to the ceiling for this task.

Ninety-five percent, and the other five

The headline number comes from hardware. A lightweight Link-CBF policy was dropped onto a Unitree G1 with no additional training, tolerated imperfect perception, and succeeded on 95% of throws, using semantic segmentation to pick out balls it had not been trained on. Zero-shot transfer from simulation to a real humanoid is not trivial and the number is a real achievement. It is also, read the other way, one throw in 20 connecting with the robot. The G1 is the platform the lab already used for CBF-RL, so the robot is not the new part either.

What dodgeball is and is not

Dodgeball is a proxy task on a controlled benchmark with seeded throws, chosen because it makes a whole-body avoidance problem concrete and filmable. It is not a safety evaluation. There is no experiment involving people in proximity to the robot, no certification of anything, and no analysis of what went wrong in the failures. The results page carrying the video is the researchers' own, which is normal for robotics and is not independent verification. This is a single lab demonstrating its own extension of its own framework, one day old, not peer-reviewed, with no journal reference on the record.

The finding worth carrying away is an engineering one, and it reaches past dodgeball. Teams building safety constraints into learned controllers tend to reach for the most exact formulation available. What this paper reports is that the exact one lost to the coarse one as soon as the robot had to find the danger itself, which is the condition every robot outside a lab is in.