GPT-6 Astra scores 95% on one robot task, 10% on another

Robocurve has run OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 against physical robot arms, and the results split sharply by task. Astra completed the easy task 19 times out of 20. On the harder task it managed 2 out of 20, exactly matching Fable 5.1. Robocurve published the Astra results on September 4, 2026, and the Fable 5.1 results the day before.

Robocurve is an independent evaluator, not either lab. It describes itself as a Public Benefit Corporation and says it is backed by Y Combinator.

The two tasks and the numbers

Both models drove the same rig: bimanual I2RT YAM arms, six degrees of freedom each, parallel-jaw grippers, three camera views. Each model ran 20 trials per task, scored by human graders on a five-point stage scale.

The first task was picking a red block into a bowl. The second was placing a blue puzzle piece into a matching groove.

Model	Block task	Puzzle task	Cost per run, block
GPT-6 Astra	19/20	2/20	$0.94
Claude Fable 5.1	8/20	2/20	$2.12
Claude Fable 5	1/20	0/20	$2.69

Astra was also quicker. It averaged 2.5 minutes per run on the block task, against 6.8 minutes for Fable 5.1 and 8.2 minutes for Fable 5.

Where the gap disappears

The block numbers look decisive. The puzzle numbers are the interesting ones.

On the puzzle task both current models landed on 2 out of 20. A 95% score and a 40% score collapsed into the same 10%. Whatever separates the models on the easy task stops mattering once the task requires fitting a shape into a slot.

That is a familiar shape. We covered how Astra scores 99.9% or 62.7% on ARC-AGI-3 depending on the harness, and this is the same lesson arriving through a different door. A single headline percentage tells you about the task as much as the model.

Robocurve's Fable 5.1 write-up makes its own narrow claim rather than a broad one. Fable 5.1 "reached later stages of both tasks than Fable 5, completed each task more often, and produced fewer output tokens doing it."

The harness is open source

The evaluation framework, called inspect-robots, is on GitHub under the MIT licence. That matters more than any single score.

The repository holds the task definitions and the scoring code, not just the results. Models plug in through a defined interface, and the framework checks the action space, the observation space and the control rate before a run starts. Each run is recorded with its resolved config, the git revision and package versions.

So the numbers can be re-run and disputed, which is not true of most model comparisons.

What this means for developers

Read the per-task numbers, never the average. If Robocurve had reported one blended figure across both tasks, Astra would look roughly twice as capable as Fable 5.1. On the task that is actually hard, they are identical.

If you are costing an agent that touches the physical world, the cost column deserves as much attention as the success column. Astra ran the block task at $0.94 against $2.12, so it was cheaper and better there. Fable 5.1 cut cache read prices sharply this month, which is the kind of change that moves these figures between one evaluation and the next.

If you are building your own evaluation, clone the harness before writing one. It already handles the parts people get wrong: validating that a policy and an embodiment agree on the action space, and pinning the exact revision a result came from.

And treat 20 trials as what it is. It is enough to separate 19 from 8. It is not enough to separate 2 from 2.