A global leader in general-purpose humanoid robotics is engineering systems for the complexity of everyday life — moving beyond controlled labs and into the home. The objective: mastering routine, high-dexterity tasks like laundry, dishwashing and general cleaning in unpredictable environments.
These tasks require fine motor control and adaptive sequencing, along with the ability to handle variation across objects and environments. Training depends on egocentric data, human demonstrations that capture how tasks are actually performed, from hand movements to motion and sequencing, in real-world settings.
The challenge
Overcoming data constraints
In practice, the push to scale humanoid autonomy faces critical constraints in data volume, environmental diversity and signal consistency.
- Physical behavior data doesn’t scale easily. It cannot be scraped or easily synthesized but must be captured through real-world egocentric data collection or instrumented setups, which introduces cost and coordination overhead, along with variability in signal quality.
- Generalization requires environmental diversity. Household environments vary widely across geographies. Differences in objects, layouts and lighting all affect how tasks are performed. Without sufficient distributional coverage, models risk overfitting to a narrow range of scenarios.
- High-signal egocentric data is naturally inconsistent. The variance in how tasks are framed and executed reduce data usability for training unless the capture process is structured and continuously refined through expert-led validation.
The approach
Building and running a human-in-the-loop pipeline
To break through the limits, we built a structured, pre-training data collection, ensuring collected demonstrations were usable for downstream training.
Geo-distributed data collection: We activated a global contributor base to capture demonstrations across a wide range of household environments, increasing distributional coverage and improving robustness for real-world deployment.
Continuous recruitment and throughput management: We maintained active recruitment to sustain data throughput and expand environmental coverage over time, allowing the dataset to evolve alongside model requirements.
Standardized capture protocols and device control: We standardized recording protocols for POV framing, task boundaries and lighting. By managing device hardware and issuing headmounts when needed, we normalized resolution and field of view to preserve the fidelity of interaction data.
Ongoing training and feedback loops: We implemented a daily review cadence and provided continuous feedback to contributors, reinforcing adherence to capture standards and improving signal quality over time.
Iterative learning integration: We aligned data collection with model iteration cycles, using human feedback to inform corrections and support incremental refinement in both dataset quality and learned task performance.
