One photograph, one mug, one trial

The photograph is titled Modern kitchen on Wikimedia Commons. Gnangarra took it and published it under CC BY 2.5 AU. The photographer does not endorse the product. In the reconstruction built from it, a pale blue ceramic mug sits on the countertop.

Everything below happened inside a rebuild of that one image.

The rebuild

Atom rebuilt the kitchen from the photograph alone. Dimensions and every surface the camera could not see were inferred, so the result is an editable generated approximation and not a survey of the room. It is an editable Blender scene plus a GLB, with MuJoCo 3.3.2 physics underneath. Contact meshes, masses and friction are explicit estimates, and the run’s evidence records that. The reconstruction is shared under the same license as the photograph.

The robot placed in it is a Franka Panda. The task, in the words the model received, was: “Pick up the pale blue ceramic mug and lift it clear of the countertop.”

The run

The policy was Atom Code, our program-writing door, with GPT-6 Astra as the model. The model received RGB frames rendered by MuJoCo, the measured pose of its gripper and the camera calibration. It did not receive the mug’s position. The reset seed was 7. The clock mode was simulation, meaning physics paused while the model thought.

The programs ran in a bounded interpreter with no arbitrary Python execution, inside a separate process with no provider keys, no network and no access to other episodes. The evidence names the scene version and its hash, so any later run can be checked against exactly this kitchen and not a revised one.

Over the trial the model made 10 calls and wrote 10 programs. It made zero calls to a learned policy. The programs compiled to 329 control steps at 20 Hz, which is 16.45 seconds of simulated motion. The whole run, including setup and thinking, took 4 minutes and 22 seconds of wall time. The median model call took 14.2 seconds and the 95th percentile was 28.2. No request to the provider failed. The model consumed 34,352 input tokens, of which 12,420 were cache reads, and produced 6,075 output tokens.

The trial ended with the termination reason success. The simulator’s predicate for this task, named height-and-both-fingertip-pads-v1 in the evidence, requires the mug to rise at least 8 cm while both fingertip pads are in contact with it. The recorded lift was 8.2 cm. The left pad was in contact and the right pad was in contact. The model’s finish call was a request to stop. The predicate decided.

What zero-shot means here

We describe the run as zero-shot, and the phrase needs its definition attached or it means too much.

It means this. The model had no task-specific training. No saved program was loaded. No skills from earlier runs in the project were available to it. It was placed in a kitchen that did not exist as a scene until Atom built it, asked in plain text to lift the mug, and it did.

It does not mean open-loop, and it does not mean one model call. The model looked at its cameras ten times and revised what it wrote in response to what it saw. Most of those ten programs were only a short move followed by a checkpoint to look again, which is the code door behaving like the agent door when the model is unsure exactly where the mug is. Closed-loop correction from camera feedback is the method here, not a caveat to it. The seed fixed the scene’s reset state. It did not fix the model’s decisions, which are not deterministic, so a second generation from the same reset could have written different programs and arrived at a different result.

It does not mean generalisation either. This is one development trial. It is not a success rate, and we have no aggregate claim to make from it. We claim no hardware transfer, and no policy was trained.

The video and the replay

The presentation video was rendered afterwards. It takes the recorded simulator poses at 20 Hz and re-renders them in Blender 4.5 Cycles inside the original reconstructed kitchen, with presentation lighting. That lighting was not an input to the model, whose frames were the MuJoCo renders, and those original frames are unchanged in the evidence. Setup and the model’s thinking pauses are omitted from the video, and the final frame is held for two seconds. So the video shows 16.45 seconds of motion plus two seconds of stillness, standing in for a run that took 4 minutes and 22 seconds. The caption says so.

After the success, the ten programs were frozen as a saved program. A fresh run of that program, in fresh physics, with the same scene, configuration and seed, succeeded with zero new model calls and produced the same actions and the same joint positions as the original. Nothing was played back from a recording. The replay stepped the physics again from the same reset, ran the frozen programs through the same interpreter, and the predicate was evaluated again. That is a strong statement about reproducibility and a weak one about anything else. Change the scene or the seed and the saved program is a guess that needs its own evaluation.

The evidence ZIP for the run holds the frozen manifest, the exact images the model saw, every command, the transcript with the model identifier the provider returned, the simulator model, the trajectory, the spectator video, the independent grade and the source snapshots, plus a replay helper. The saved program is published beside the showcase as a Python file.

One trial lifted the mug 8.2 centimetres. The replay is the second time it has left this countertop, and it counts as reproducibility of the first trial, not as a second one.