Why we build environments

Every trial on a real robot ends the same way. Someone walks over, picks up the mug, and puts it back where it started.

That walk is short, and it is the shape of the whole problem. A robot arm needs a fresh motor command about a thousand times a second. A model deciding where the arm should go next answers in seconds. Between those two rates sits a physical world that has to be reset by hand, shares one arm between everyone in the lab, breaks mugs, and never runs the same trial twice. You can make the model faster. You cannot make the mug reset itself.

We started Moving Atoms because we think the physical world is the slow part of robot development, and the development loop belongs in software. Not the final test on hardware, which has to happen where the hardware is. The loop: the many attempts before the one you show anyone, the comparison of two models on the same scene, the replay of a run from last week to see what the camera actually saw.

What RobotGym is today

RobotGym builds editable 3D environments from the material a team already has: photos, video, PDFs, Word documents, notes, tables and 3D assets in GLB, OBJ, STL or PLY. You refine the result in chat with Atom, our authoring agent, which adjusts geometry, materials, lighting, robot placement and cameras. The output is an editable Blender scene plus a GLB, with MuJoCo physics underneath. You can then run a language model, today GPT-6 Astra, Claude Fable 5 or Claude Fable 5.1, through one shared harness against that scene with the same limits and the same success check. Every run produces the exact images the model received, its commands, the transcript, the trajectory and a verdict decided by simulator state, all downloadable and replayable. The reference robot is a Franka Panda with a Robotiq 2F-85 gripper. The starter environment is a reconstructed office desk where the task is to stack a purple cylinder on a red can.

That is the product. It is LLM-first, and we say so. We are not policy-agnostic infrastructure for every robot. The workspace exposes one arm and three models, and those are what we describe.

Why environments rather than a smarter model

In our own work, the failures that cost the most time were rarely about a model’s judgement. They were a wrist camera measured from the wrong point, a gripper that quietly opened while the model was thinking, a rotation written in one convention and read in another. None of these shows up in a demo video. All of them show up in the evidence of a run, provided the run kept its evidence.

So we put our effort into the loop around the model rather than into the model. Environments that can be rebuilt from a photograph and edited afterwards. A controller that is identical for every model, so that a comparison between two of them means something. Records that keep the failed attempts next to the successful one, because the failures are most of the information.

What we will write about here

The observation contract: what actually reaches a model in RobotGym, and why we save the exact pixels it saw separately from the view made for people. The two ways a language model can drive an arm, one call per motion or a short program, and what our audits measured for each. The two clocks on every run, simulated and wall, and why we never report one as the other. One public photograph of a kitchen, and what happened when an agent tried to lift the mug in it. And the conventions we build against, written down as how we work rather than as advice to anyone else.

We will use real numbers from real runs, with the qualifiers that belong to them. One recorded trial is one recorded trial. A saved program that replays is a replay, not a second success. And physics pauses while the model thinks. That last sentence will appear in most of what we write here, because it is true of every run we have recorded.