Human motion mapped onto a robot · real capture

Case study · Software and data

Veles internal R&D

First, we trained a robot model on our own data.

Before offering robotics data to anyone, we tested every step ourselves. We recorded human motion with cameras in our studio and mapped it onto a humanoid robot. We exported it in formats robot teams train on, and trained one robot model from it.

The constraint

No client yet, so we ran it on ourselves first.

This one had no customer. Before offering robotics data to anyone, we wanted to run every step in our own R&D. We recorded motion in our studio, mapped it onto a humanoid, exported it in open formats and trained a robot model.

Who owned what

All four steps, in-house.

Client
Veles internal R&D; no external customer yet
Veles
Recording, mapping to the robot, export and training, all in-house

What we delivered

The four steps, and their limits.

  • Record: OptiTrack camera-based motion capture of body, hands and props.
  • Map: the human motion moved onto a humanoid robot, across 78 joints.
  • Export: saved in open robot-learning formats. Ten clips went through the pipeline.
  • Train: one policy checkpoint (a saved robot control model), trained on two clips.
Mapped to robot
78 joints
Clips
Ten
Trained model
One · trained on two clips

Verified result

One robot model, trained on two clips.

We haven't yet tested the model in action (a closed-loop evaluation), and there is no external customer yet. This page shows our own run, so robot teams can see what we have done, not what we promise.

Verified
One robot model (policy checkpoint), trained on two clips
Tested in action (closed loop)
Not run yet

For robot-learning teams

Need human motion data for a robot-learning task?

Send the task list. Jan Kontek runs Robotics data.