Runway Trains Robots Using Internet Videos, Performance Nearly Matches That From Professional Datasets

Insight Beating AI News Flash: Runway has released the robot model Praxis-1, applying the expertise from training AI video models to real-world robots. It first learns how objects move and how humans perform tasks from a large number of regular videos, then translates that knowledge into robot movements. Runway conducted an object placement test: after pre-training with internet videos, the robot’s average error was 16.1 cm; when using custom-collected teleoperation videos of robots, the error was 16.0 cm. The test covered 93 evaluation sets, with results nearly identical across the two training approaches. If this line of work can be scaled, the data gap plaguing robot training could expand by several orders of magnitude. Collecting data via human-operated robot teleoperation requires specialized equipment, space, and labor, while regular videos are already abundant online. Runway aims to repurpose these videos—originally not intended for robot control—into training materials for robots to master movements. Praxis-1 has been tested on multiple robot models from Noble Machines, Standard Bots, and Ultra, including robotic arms, dual-arm robots, and mobile robots. Runway also showcased that the same strategy can adapt from a studio setting to a kitchen to complete tasks. The model is currently only accepting early test applications; its weights are set to be released in the coming months, with parameter size and more comprehensive independent evaluations still undisclosed.