There isn’t enough data to solve robotics. Even if you hired armies of humans.
We’re approaching the problem a different way. Our world foundation models are SOTA at generating virtual environments for robots to train in.
Show it a scene and it continues the scene — things fall at the right speed, liquids spread, cloth settles. This is what a video model already does well, and it is the foundation everything else stands on. It is also, on its own, not enough: a model can render gravity perfectly and still be useless to a robot.
Physics, learned from watching
Now you can intervene. Hand it a movement — this joint, this far, this fast — and it shows you the future that movement produces. Change one number and the future changes with it. This is the line between a model that has watched the world and one a machine can actually plan inside.
Counterfactual, not just plausible
The same model, sampled differently, proposes the movement and the consequence together. That makes it a controller you can interrogate: every command arrives with the future it expects, so you can check the prediction against what actually happened and know which half was wrong.
One set of weights, three behaviours
Rank a hundred versions of the software overnight, without a robotEvaluation
Try a movement a thousand times before touching anything realPlanning
Generate demonstrations of tasks nobody ever performedTraining data
Replay a failure and ask what a different choice would have doneCounterfactual
Practise in imagination, then act once in the worldLearning
Notice when the world disagrees with what was predictedMonitoring
Reach a new machine from a few minutes of watching it moveTransfer
Recover the movements from footage alone, with no sensorsInference
Test the dangerous case without breaking the expensive thingSafety
Hear the contact as well as see itSound
Watches, cannot be steered
They have watched more of the world than any robot ever will, and it shows. But there is no way to intervene in one. You can describe what you want in words, and the evidence is that a model steered by description alone performs indistinguishably from one steered by nothing at all.
Exact, but hand-built
Fast, repeatable, and exactly as rich as whatever somebody modelled by hand. Every object, every surface, every friction coefficient has to be authored. The greasy pan, the floppy bag, the cluttered counter — the long tail that breaks real robots — is the part nobody builds.
Where we work
A simulator that learned the world from footage instead of being authored — so it inherits the long tail for free — and that answers to a control input, so a machine can plan inside it. Trained by reality, steerable like a simulator.
Step one · working now
We start from an open video model and teach it physics it does not have. Out of the box these models render gravity at roughly a fifth of Earth’s — and the published fix is startlingly small: a hundred clips and an afternoon of training moves it most of the way back.
Step two · in build
Then the surgery: movements become a channel the model reads, aligned to the video frame they cause. The test is blunt — change one number in a command and the imagined future has to change with it, in the way physics says it should. Most attempts quietly fail this, which is why we run it on every checkpoint.
Step three · in build
The same weights, sampled to produce the movement as well as the consequence. That is a controller — and an unusually legible one, because every command it issues arrives with the future it expects, which you can hold up against what actually happened.
A two-person company. One of us built recommendation systems at the scale where a small error rate is a large number of people; the other trains models and does the maths. Between us: the training runs, the evaluation harness, and a habit of reading the config file rather than the summary.
Worked on TikTok’s video recommendation algorithm — ranking at a scale where a small error rate is a large number of people. Maths and Computer Science at Oxford, Economics at LSE.
Here he does the model work and the benchmark strategy: what to train, on which data, and the harness that decides whether a change actually helped rather than merely looked like it did.
Reading Maths at Oxford. At GCHQ he trained a network to summarise incoming threats and sort them across seventy-two categories, using Jaccard distance over word embeddings; he has also interned in asset management at J.P. Morgan.
Here he does the training and the measurement: the runs themselves, and the instrumentation that establishes from a model’s own output what it actually learned — which is the part everything else has to be judged on.