Embodied competence — largely yes.
Practical embodied intelligence — not yet.
We ask not only how capable GPT-6-Astra is, but what its capabilities imply for the field: we probe its limits on 12 benchmarks, dissect the harness behind it, validate it on eight real-robot platforms, and prototype what the next generation of embodied benchmarks should measure.
Successful demonstrations are conventionally accessible when training smaller task-specific policies, so GPT-6-Astra may use them too. It studies the training data and builds a frozen knowledge package of task guidance, motion and visual references, executable helpers, recovery procedures and metadata, then retrieves and adapts these resources to each test scene.
We allow up to 10× the official step limit, since standard step limits can significantly underestimate GPT-6-Astra. Its emerging capability for failure recovery lets it translate additional steps into higher success, nearly saturating SIMPLE and the LIBERO family. Extensions continue the same episode — no resets.
Packages are built from each benchmark's training data, so their benefit weakens once test tasks move away from it. Packages that generalize beyond their source tasks remain an open direction.
No task-specific demonstrations; it relies on a package studied from the other four groups. GPT-6-Astra + ICL still benefits from more interaction, but attains the lowest success of all groups.
No task-level package can be built (zero-shot, 10× budget). GPT-6-Astra leads the best listed baseline (42.9%), yet its absolute success remains moderate.
Where test tasks follow the training distribution — LIBERO family, SIMPLE, VLN-CE — demonstrated motions or routes are retrieved almost directly and success approaches saturation: above 95% on the LIBERO family and SIMPLE, around 90% on VLN-CE at 10×.
Standard-budget success on the four common tasks, with shorter successful episodes under ICL.
Even nearly solved benchmarks are expensive: about 1M tokens and nearly an hour per LIBERO-Long++ episode, roughly 9M tokens and nearly two hours on LIBERO-Ultra-Long. Extending the budget adds more, concentrated in difficult episodes that often still fail: on RoboDojo, an extended-budget episode takes more than three hours on average.
About 98% of tokens are cached input, so 1M tokens ≈ 1 USD. Inference latency is not counted in the step budget, so this cost also limits deployment where real-time control is required.
GPT-6-Astra can tell a task is unfinished but not which physical condition is unmet, so its corrections fail to converge. Looking done, or reporting done, is not the same as meeting the task's contact, support and alignment conditions.
Each attempt changes the state that recovery starts from, and some changes cannot be undone. A failed prerequisite blocks every later subtask while retries drain the budget.
Recovery works when GPT-6-Astra changes strategy rather than repeating itself, as in DexJoCo Fold Glasses, where a side approach and a test lift complete the task.
Earlier progress must stay intact while later actions run, yet a later motion can undo it, and small errors in grasping, placement or navigation compound across stages. GPT-6-Astra has to keep rechecking what it has already done rather than treating completed subgoals as permanently resolved.
Letting the VLA run alone is insufficient; GPT-6-Astra reviewing its proposed actions works, but costs tokens. Reviewing four at a time saves 63.5% of tokens over per-action review, and adding ICL gives the best result — at about 6.9× the tokens of the best closed-loop setting without ICL.
Directional commands help and joint angles hurt. Letting the model choose its chunk size beats a fixed chunk in success rate, at a higher token cost.
GPT-6-Astra performs as a generalist, without task- or robot-specific training.
Tool schemas, joint limits, camera placement, safety constraints, hand presets and, on some platforms, task-level guidance such as admissible grasp points — no high-level semantic skills such as pick, place or insert.
Trials are normally limited to 60 minutes. An operator holds an emergency stop, and success is judged from the physical outcome, not the model's own claim.
Without platform-specific training, the same model controls all eight platforms. Rigid pick-and-place is reliable and matches simulation, whereas deformable objects, precise contact and long-horizon mobile tasks remain markedly harder. A second arm extends the range of zero-shot tasks, and GPT-6-Astra re-plans from execution feedback when the scene changes.
In flower insertion, the gripper holds the stem vertically, so its own wrist camera cannot see the vase mouth. On a single arm the insertion fails. On the dual-arm Piper, GPT-6-Astra moves the free arm until its wrist camera sees the vase mouth, then completes the insertion — the viewpoint becomes part of the action space. Teleoperated demonstrations rarely contain this active vision.
On a real Unitree G1 with Wuji hands, GPT-6-Astra commands arm-joint targets and binary hand states for manipulation (2/3), and turns and walks through the vendor velocity interface (3/3) — without a task-trained System 1 policy. This establishes feasibility under slow, feedback-driven supervision rather than reliable real-time visuomotor control.
In simulation, extra steps and failed attempts cost nothing, and demonstrations of the test task exist. On real robots neither holds: physical constraints limit extended execution, and missing demonstrations reshape in-context learning.
Extra steps and retries cost nothing in simulation. On real robots, protective stops and hardware faults end runs, and elapsed time adds tokens, risk, hardware stress and supervision.
Human-written context can stand in for demonstrations, but it is expert prompt engineering. Building context from other sources is an open problem.
Stated as a person would, e.g. “Could you make me breakfast?” Judged on the request (Intent) and on unstated obligations (Duty).
No in-domain demonstrations. The agent learns the scene through its own exploration.
Thinking takes world time, as on a real robot. The judge watches the whole trajectory, even after the agent says it is done.
Please reload the page.
Its own arm and the states it has changed seem to enter its decisions only when they block the next step. A final-state check would miss this, so the judge tracks the whole trajectory.
It plans for the gripper and handle, not its whole body, and repeats and adjusts the same motion. In Section 01, the demonstrations it distills in context show both, so this weakness does not appear there.
It rightly decides to turn off the burner once the egg sets, but thinking takes world time: the burner goes off about 90 s late and the egg chars.
The simulators of Section 01 wait for the model. This world does not, so the action arrives too late.
Measuring deployment rather than isolated tasks raises four open problems for the judge, the simulator and the evaluation budget.
Simulators model key physical facts heuristically; vision-language success detectors reach at most 60% balanced accuracy on contact-rich assembly.
Modeling errors look like agent behavior; simulation has been shown to predict real-robot results only for short tabletop tasks.
If the world keeps running, scores depend on latency, hardware and service load.
Hour-long deployments are expensive to repeat; even short robot trials are typically run too few times to separate policies reliably.
Strong harness design with in-context learning and a sufficient execution budget can push many isolated benchmarks close to saturation. The remaining challenges shift toward transfer across tasks, long-horizon reasoning, failure recovery, and real-world constraints on time, cost and safety. The next frontier is not simply to solve more isolated tasks, but to build embodied systems that operate reliably and efficiently under realistic deployment conditions.
Packages