Skip to content

For AI labs · How we compare

Robot training data sources compared

There are four common ways to get robot training data: teleoperation farms, synthetic or simulation-only data, crowd labelling platforms, and skilled-human demonstrations in games and simulations, which is what Metagaming offers. No single source wins on everything. Teleop is the most realistic, simulation scales furthest, crowd labelling scales judgments, and game and sim demonstrations are quick to start with ranked human skill, but with a sim-to-real gap. Most teams combine them.

Request a dataset or pilot Data formats and specs

How do the four approaches compare?

Each approach trades realism against cost and speed. The table sums up where each one is strongest.

ApproachCost per hourScaleRealismSkill levelTime to first dataBest use
Teleoperation farmsVariesLimited by robots, rigs and operatorsHighest: real robot, real sensors, real contactTrained operatorsSlower: robots, rigs and operators must be in placeReal-robot policies, contact-rich manipulation, final fine-tuning
Synthetic and sim-onlyVariesVery high: runs in parallelLower on contact, deformable materials and sensor noiseScripted or learned policies; humans optionalFast once the simulator and task are builtPre-training, rare or dangerous scenarios, reinforcement learning at scale
Crowd labellingVariesHigh, for labelsDepends on the data being labelledFrom general to specialist, by platformFast, once you have data to labelLabelling images, video and logs you already have
Game and sim demos (Metagaming)VariesGrows with the trainer pool; starts with a pilotReal human decisions and timing; game or sim physicsVetted adults, ranked on six skills and a five-level ladderTimeline given after the brief, before collectionTask-level demos, edge cases and evaluation labels, alongside real-robot data

Why "varies" for cost: we didn't find independent, published per-hour figures we could cite for any of the four. Cost depends on the task, the hardware, the skill needed, the QA rules and the volume, so compare quotes on the same brief.

Teleoperation farms

Teleoperation gives the most realistic robot data you can get: a person controls a real robot, so the data has the robot's own sensors, joints and contact. Label Studio's 2026 guide calls teleop and kinesthetic demonstrations "the highest-fidelity demonstrations available" (Label Studio).

The cost is speed and hardware. The same guide notes a trained operator might produce dozens of usable episodes a day, so scaling means more robots, rigs and operators. As a public reference point, the DROID dataset's 76,000 trajectories, about 350 hours, took 50 data collectors across 13 institutions 12 months to gather (DROID). Teleop operators often work on site, next to the robots.

Synthetic and simulation-only data

Simulation generates the most data for the least effort once it's built: trajectories run in parallel, and rare or dangerous scenarios cost nothing to repeat.

The limit is transfer to the real world. Label Studio's guide notes that sim-to-real performance degrades as a task involves more physical contact, deformable materials or fine manipulation, and that simulators leave out sensor noise, lighting variation and mechanical inconsistencies. It describes the 2026 research view of simulation and physical data as "complements, not substitutes."

Crowd labelling platforms

Crowd labelling platforms send existing data, such as images, video or logs, to many workers who tag, rate or annotate it. They're fast to start and scale well for labels, and some offer specialist pools.

Their job is labelling, not demonstrating. They're a strong choice when you already have robot data and need it annotated; they don't on their own produce the control demonstrations a policy learns to act from. Label quality depends on the instructions, the workers' expertise and the review steps.

Skilled-human game and sim demonstrations (Metagaming)

Metagaming collects demonstrations from vetted adult trainers playing games and simulations, including custom training games built around your task. Every application is reviewed, and trainers are ranked on measured quality, from Rookie to Genius, and scored on six skills. You get their inputs, timestamps and the task state your brief lists, plus evaluation labels on your robot's attempts.

It's quick to start because no robot or rig is needed, and you can choose trainers by level, skill score and industry. The limits are real, and we'd rather state them:

  • Sim-to-real gap. The physics are the game's or the simulator's, not your robot's. The data is strongest for decisions, timing, paths and task order.
  • Inputs, not joints. Trainers use keyboards, mice, controllers, wheels or VR, so actions need mapping to your robot's action space.
  • Teleop isn't live yet. Teleoperation of real robots is coming soon, starting with trainers at Expert level and above.
  • Pilots first. Every project starts as a small pilot, so you can check the data helps your model before a larger order.

Which approach should you choose?

Choose by the gap in your data, and expect to combine sources:

  • Need real contact and real sensors? Teleoperation, or real-robot data from deployed systems.
  • Need huge volume or dangerous scenarios? Simulation, checked against real data.
  • Have data that needs labels? Crowd labelling, or evaluation labels from skill-ranked trainers when the judgment needs expertise.
  • Need skilled human decisions, edge cases or rankings quickly? Game and sim demonstrations, alongside one of the above.

Shared datasets help too: Open X-Embodiment pools over a million real robot trajectories across 22 robot types (Open X-Embodiment). See what fits by industry: autonomous driving, drones, warehouse robotics, manipulation and humanoids.

Comparing data sources: questions from labs

Which robot training data source is best?

None wins on everything. Teleoperation is the most realistic, simulation scales furthest, crowd labelling scales labels, and skilled-human game and sim demonstrations are quick to start with ranked human skill. Most teams combine two or more.

Can game and sim demonstrations replace real robot data?

No. The physics are the game's or the simulator's, so there's a sim-to-real gap. Game and sim demonstrations work best for decisions, timing, paths and task order, alongside real-robot data.

Does Metagaming offer teleoperation?

Not yet. Teleoperation of real robots is coming soon, starting with trainers at Expert level and above. Tell us your robot in the request form to be first in line.

How much does robot training data cost per hour?

It varies with the task, the hardware, the skill needed, the QA rules and the volume, and we didn't find independent published per-hour figures to cite. Compare quotes on the same brief. For Metagaming, send a brief and we'll email you about scope, format and timeline for a pilot.

We're looking for the most skilled people to train the future.

Think you're one of them? Prove it.

Apply free

Free to apply · 18+ · Every application is reviewed