Skip to content

For AI labs · Manipulation

Human demonstration data for robot arms and manipulation

Metagaming collects human demonstration data for robot arms from vetted adult trainers: part placement, assembly steps, gentle-grasp and folding demos in games, simulations and VR, plus human judgments on your robot's attempts. It fits manufacturing, lab automation, agriculture and home robots, and every project starts as a small pilot.

Request a dataset or pilot Data formats and specs

What's the data problem in manipulation?

Manipulation needs many demonstrations, and real ones are slow to collect. The DROID team, for example, gathered 76,000 teleoperated trajectories, about 350 hours, with 50 data collectors over 12 months (DROID).

Simulation scales faster, but transfer to real robots gets harder as a task involves more physical contact, deformable materials or fine manipulation, according to Label Studio's 2026 guide to robot training data.

Game, sim and VR demonstrations sit between the two. They're fast to collect and capture what a skilled person decides: which part goes first, where it's placed, how a task is sequenced. They don't capture real contact forces, so they work best for task structure and judgment, alongside real-robot data for the final contact-rich steps.

Which Metagaming data fits manipulation?

All four data types open for pilots apply to manipulation, and teleoperation is coming soon.

  • Pilots open

    Human demonstrations

    Trainers place, assemble, fold or sort in a game, simulator or VR. You get their inputs, timestamps and the task state your brief lists.

  • Pilots open

    Gameplay input data

    Inputs and timing from skilled players in precision and VR games, recorded only in the games a mission names.

  • Pilots open

    Evaluation labels

    Trainers judge your robot's grasps and placements: what went right, what went wrong, and which of two runs was better. Inspectors label defects.

  • Pilots open

    Custom training games

    We build a game around your task, such as your part set and fixture, so trainers produce exactly the demonstrations your policy needs.

Coming soon

Teleoperation sessions

Trainers at Expert level and above will control real robot arms remotely, live, under your instructions. Most teleop work elsewhere is done on site; tell us your robot in the request form to be first in line.

What could a manipulation pilot look like?

These are examples of what a pilot could look like, not past projects. Your brief sets the real scope.

  • Assembly sequence demos

    Trainers assemble your parts in a custom training game, with each placement, its order and its timing recorded as one episode.

  • Folding and tidying in VR

    Trainers fold and tidy in a VR task built for home robots, giving you the steps and order a careful person uses.

  • Grasp and defect judgments

    Trainers choose the better of two grasp attempts on your robot, and quality inspectors label defects.

Which trainer skills does it use?

Manipulation missions draw on trainers with strong scores in steady hands, accuracy, timing and packing, four of the six skills we measure per trainer. Scores come from our skill tests, such as the steady hand test, and from quality review on missions.

We recruit skilled gamers, including VR players, and people whose trade is careful handwork: machinists, assemblers and quality inspectors, lab techs, and pickers and growers. Where a mission needs a trade certificate or professional license, we check it before that mission. Filtering by license or certification is coming soon, with the first missions that need one.

You can filter trainers by level, skill score and industry. Labs see a trainer name, skills and scores, never a real name.

How does a pilot work?

A pilot follows the same five steps as every Metagaming project, at a small size, so you can check fit before a larger order.

  1. 01

    Brief

    You tell us the robot, the task, the data type, the volume and the timeline.

  2. 02

    Custom mission

    We design the mission or training game and agree the QA rules with you.

  3. 03

    Collection

    We match the mission to accepted trainers with the right level, skill scores and industry.

  4. 04

    QA

    Every task is checked against the agreed rules. Work that fails isn't delivered.

  5. 05

    Delivery

    You receive the data in the agreed format, with each session's consent record.

Manipulation data: questions from labs

Does game or sim data capture contact forces?

No. Game, sim and VR demonstrations capture inputs, timing, order and the task state your brief lists, not real contact forces. They work best for task structure and judgment, alongside real-robot data for contact-rich steps.

Which industries does this cover?

Manufacturing (part placement and defect labels), agriculture (gentle-grasp demos), home robots (folding and tidying demos in VR) and lab automation (pipetting and sample-handling demos).

Can trainers label defects?

Yes. Evaluation labels include judgments on your robot's attempts and defect labels from trainers with inspection experience. The labels and allowed answers are agreed in the brief.

What format do we get?

The format is agreed in the brief. LeRobot, RLDS, HDF5, MCAP and JSON or CSV are available on request. See data formats and specs.

We're looking for the most skilled people to train the future.

Think you're one of them? Prove it.

Apply free

Free to apply · 18+ · Every application is reviewed