Evaluation doesn’t scale
Every team writes its own scripts and success checks, so two models are almost never measured the same way.
Simulation · Evaluation · Benchmarking · Post-training
Simulation & evaluation infrastructure for Physical AI.
Spin up simulated worlds, run your robot policies on on-demand GPUs, compare checkpoints side by side, and feed what failed back into training, all in one repeatable workflow.
RUN · policy-v3front-cam · 256²
“stack the green block on the amber one”
The bottleneck
Robot models are shipping faster than teams can measure them.
Simulation, evaluation, data, and training usually live in separate tools. Each loop needs bespoke plumbing and costly time on hardware.
Simulation ≠ improvement
Rollouts pile up in sim, but the lessons from failed runs seldom make it back into the data or the next training cycle.
Real-world testing is slow
Hardware time, resets, and safety reviews make it impractical to check every new checkpoint on a physical robot.
One platform, four connected stages
No model is best at everything. Find the one that fits your task.
Simulate
Generate tasks, scenes, and seeded variations ready to run across popular simulators, from tabletop manipulation to full scenes.
Evaluate
Run policies closed-loop on serverless GPUs and capture success, robustness, control rate, memory, cost, and rollout video.
Benchmark
Line models up across embodiments and simulators on a single frontier, with clear trade-offs and ranked recommendations.
Post-train
Turn failed episodes into curated data, fine-tune, and re-run the same suite to confirm the fix actually holds.
New · DataroomNew New New New
See why a policy failed, then turn it into better data.
Evaluation tells you that a run failed. The Dataroom shows you where and why. Import rollouts or datasets, let an agent annotate them, and curate the episodes worth training on next.
Agent annotation
An agent reviews every rollout, splits it into phases, flags grasp and release moments, and labels the failure, no manual tagging.
Import any dataset
Bring in public or private robot datasets from the hub and browse every episode as a searchable, replayable clip.
Failure attribution
Go beyond one success number and see exactly where, and how often, policies break across thousands of episodes.
Curate → retrain
Pick the clips that matter, export a clean dataset, and hand it to the next training run to close the loop.
Bring your own data New
Connect your own dataset account to pull private repos and push recorded work back to your namespace. Credentials stay encrypted and scoped to you.
Decision-grade evidence
The metrics that decide deployment, not just one success rate.
Success + 95% CI
Success rate with confidence intervals, so small gaps are not mistaken for real wins.
Control rate
Achieved control frequency per policy, checked against what your robot actually needs.
Robustness Δ
How much performance drops under lighting, camera, and object perturbations.
GPU memory
Peak VRAM per policy, pointing you to the smallest hardware tier that works.
Cost / success
Spend per successful episode, the figure that tells you whether a model can ship.
Reproducible
Fixed initial states and a config hash let anyone rerun the exact same evaluation.
How it works
Generate → Evaluate → Benchmark → Improve
01 Generate
Describe a task; get a world with seeded variations.
02 Evaluate
Real rollouts on serverless GPUs.
03 Benchmark
Compare head-to-head on one frontier.
04 Improve
Post-train on failures, then re-test.
Who it’s for
Built for the teams shipping Physical AI.
Robotics foundation-model labs
Evaluate every checkpoint across tasks and embodiments without building an internal eval stack.
Applied robotics teams
Pick the right model for your task and hardware before committing time on the real robot.
Research groups
Publish results others can reproduce, with frozen seeds, configs, and shareable rollout video.
Simulation & data teams
Turn rollouts into curated datasets and measure whether new data really moves the needle.
Evaluate your first policy on a real GPU.
Bring a model, choose a task, and get reproducible results back, affordable enough to run on every checkpoint.