Collecting robot training data is dirty, unglamorous work. Some AI labs are already paying XDOF to do it

Two weeks ago, OpenAI said it would relaunch the robotics programme it shuttered successful 2021 — the latest awesome that the biggest AI labs are racing to thatch machines to run successful the carnal world. But gathering susceptible robots requires thing the AI manufacture doesn’t yet have, which is the grooming information to lucifer that utilized for connection models.

That spread is creating a caller benignant of infrastructure business. Unlike LLMs that were trained connected a immense oversea of publically disposable text, robots request information that captures carnal interaction, and that benignant of information hardly exists. YouTube videos and footage captured by gig workers are low-fidelity and hard to reconcile with the carnal world.

XDOF (pronounced “ecks-doff”), emerging from stealth today, is betting that the adjacent large bottleneck successful AI isn’t models oregon chips, but the information feedback loop needed to thatch robots however to interact with the carnal world.

The startup aims to physique the information pipelines, postulation tools, and annotation systems that frontier labs and robotics companies can’t easy physique themselves — and has raised $70 cardinal from Thrive Capital, Spark Capital, a16z, Lux, and WndrCo to bash it. Co-founder and CEO Philippe Wu says XDOF, which has astir 60 employees, is already moving with 20 customers including respective frontier AI labs, but cannot sanction them.

“All of the apical labs are trying to prosecute robotics,” Wu said. “We’ve already seen immoderate of the downfalls of falling a small spot down successful the connection exemplary contention … you don’t privation to beryllium successful this benignant of concern wherever you prosecute this exertion excessively late, and everyone is successful this vessel wherever carnal AI is the adjacent frontier.”

Wu ran into this occupation himself arsenic a PhD pupil astatine UC Berkeley. His absorption was connected enabling robots to larn skills from large-scale information sets. There was conscionable 1 problem.

“We didn’t person large-scale information to enactment with,” helium told TechCrunch. “There was this chicken-and-egg occupation — we archetypal needed to really cod information earlier we could adjacent inquire however to bid a instauration exemplary for robotics.”

Wu and his aboriginal XDOF co-founder and CTO, Fred Shentu, worked connected a task called GELLO, a low-cost teleoperation strategy that lets a quality relation power a robotic limb to make grooming data. “It ended up becoming a precise influential insubstantial successful robotics, due to the fact that a batch of radical had akin needs and bottlenecks, and galore started leveraging this benignant of instrumentality for information collection,” Wu said.

Spotting the opportunity, Wu, Shentu, and 3rd co-founder and Chief Operating Officer Nemo Jin launched XDOF successful October 2024 to supply a information ecosystem for companies pursuing robotics models. Mindful that information proviso unsocial tin beryllium a dead-end business, the institution is besides focused connected information cleaning, tooling, and annotation — creating a self-reinforcing feedback loop for robot trainers.

As a starting point, the institution is partnering with UC Berkeley’s AI Research laboratory to merchandise what it believes is the largest postulation of high-quality robot grooming information ever assembled, dubbed ABC. It includes 130,000 trajectories of robot manipulation data, 300 hours of simulation, and 100 hours of evaluations. That benignant of scaled-up pre-training information has ne'er been disposable to academia before.

“We’ve seen successful language, representation generation, and different fields, that erstwhile models and information are released, the assemblage achieves things that you wouldn’t needfully person expected,” David McAllister, a Berkeley PhD pupil who helped signifier the release, told TechCrunch.

The squad has already utilized the information to bid robots connected benchmark tasks similar folding T-shirts and flattening boxes, oregon loading AirPods into their cases.

Unlimited degrees of freedom

The institution plans to enactment crossed 3 tiers of a information pyramid. The astir invaluable tier is teleoperation information collected connected the existent robot being deployed; adjacent comes teleoperated robots gathering much wide data, arsenic with GELLO; and yet “egocentric” information gathered by humans performing mundane tasks, for which XDOF plans to physique its ain wearable sensors.

“Your camera prime is going to impact the prime of your information — which is going to impact however your hand-tracking algorithm performs,” Wu said. “If you don’t plan the hardware good from the start, the information you cod mightiness person precise circumstantial problems that you didn’t anticipate.”

The institution plans to prosecute and bid armies of teleoperators and egocentric information operators astir the satellite — a labor-intensive exemplary that raises an evident question: Why aren’t the large labs doing this information accumulation enactment themselves?

“You request a warehouse of hundreds of thousands of quadrate feet with hundreds of robots,” Wu said. “You request to support these robots, calibrate their carnal parameters, and decently bid operators.”

It’s a build-out that requires focus, capital, and operational standard that astir AI labs would alternatively outsource — which is precisely the marketplace XDOF is betting on.

The sanction XDOF is simply a play connected the robotics word “degrees of freedom,” which describes the fig of autarkic motions a robot tin perform. Your arm, from enarthrosis to wrist, has seven degrees of freedom. Humanoid robotics institution Figure.AI’s latest robot has 30. The X successful the company’s sanction captures its ambition: “Arbitrary degrees of freedom, unlimited degrees of freedom,” Wu says.