Autonomy is a ratio

Every Walden robot operates with a remote assistant available. Most of the time, the robot acts autonomously. When it encounters something novel or uncertain, a remote person can step in, keeping useful work moving and generating some of the most valuable training data in physical AI. As our models learn, the ratio of robots to remote assistants improves continuously, allowing us to scale faster than an all-or-nothing approach. This makes reliable high-quality remote teleop an important differentiator, enabling our robots to do more valuable work, and generate more valuable data, more quickly than robots that can’t ask for remote help.

First-person view of a Walden robot receiving brief help from a remote assistant.

This pattern has deep roots. Manufacturing learned decades ago that the best automation detects its own trouble and summons a person. Toyota calls this jidoka, "automation with a human touch," where a machine stops itself and calls for help rather than continuing and compounding a problem. Autonomous driving uses a related playbook to scale: Waymo's fleet operates with remote agents who provide guidance when a vehicle encounters difficulty it can't resolve alone, currently at a reported ratio of one person to dozens of vehicles. Inspired by this, we’re building our robots to pull their own andon cords, backed by a remote assistance stack that provides fast and seamless resolution.

Crucially, when a person assists a robot in this way, that correction is “on-policy”: it demonstrates desired behavior near, and recovery from, failure states where data is most needed. Replicating this after the fact is impractical at scale; the remote assistant paradigm allows learning from this most valuable data by construction. That lets us deliberately deploy at the moving frontier of current capability, where data novelty matters most. Here, deployment compounds in three ways. It generates high-value, hard-to-replace data to improve autonomy, builds operational capability to deploy at increasing scale, and provides immediate business and customer value - because the useful work being done by deployed robots is our product.

What data?

I want to talk a bit more about the data aspect here, and why this on-policy corrective data is so important. If you’re in physical AI, you get asked about data constantly. One of the most frequent questions I get is something along the lines of “What do you think of x data modality?” where x is generally something like human video, robot simulation, or UMI/wearables. The short (and somewhat superficial) answer here is they’re all useful, and each can be a valuable part of a training mixture - all are present in our Walden datalake.

That's not the whole story though. Unlike in the digital space, vast quantities of diverse robot data are not a pre-existing resource and must be acquired deliberately. The important question then isn’t about the usefulness of data, it’s about opportunity cost and uniqueness. What data provides the highest ROI and where is there greatest or least flexibility in the training mixture?

These nuances matter in the digital space as well. It’s why digital AI companies that have trained their models on meaningful fractions of the entire internet are still doing things like paying mathematicians by the hour, licensing decades of news archives, and signing strategic private data deals.

The seeming paradox - that large-scale training amplifies, but does not replace, the need for domain-specific data - that drives this bimodal approach to data in the digital space also holds deep implications for physical AI. It’s why deployment data from robots doing useful work in the world, with human feedback, is the most critical data source in robotics. This data is both highly valuable and non-fungible; it’s required to ground diverse multimodal training mixtures into the specific behaviors and domains in the field where robot performance matters, and it’s data that cannot easily be replaced by other sources.

Physical AI isn't (that) special. Data is just hard

State-of-the-art physical AI shares much in architecture and data philosophy with the digital space; there’s just significantly less physical-world data currently available. The archetypal examples are large transformer-based policies that process multimodal task prompts and robot sensor readings to directly command robot motion - generally via iterative denoising. These models are trained on large amounts of multimodal data from many sources, including robots in deployment, other robot embodiments, human video, simulation, and handheld capture. This training data mixture is crucial for robustness, generalization, sample efficiency, and a whole host of other important attributes.

As physical AI advances, we’ll see a continually shifting frontier of capabilities. Sample efficiency will improve, as will flexibility and generalization. Some types of behavior that are well represented in large-scale datasets, like basic pick and place, may soon be employed broadly without necessitating additional in-domain data. Other types of behavior, such as those that require complex force modulation, rely on non-visual feedback, operate at the limits of physical ability, involve unusual task constraints, or have delayed failure observability, will require significantly more in-domain physical data.

As this plays out, capabilities will fall into three groups. Some will sit safely inside the moving frontier, where generalization from past experience is reliable. Others will remain outside it and not yet be reliable enough to productize. The most valuable for data and expansion will lie along the boundary, where deployment and on-policy experience produce the human and environmental feedback needed to push the frontier outward. We built our deployment-first data strategy at Walden around this phenomenon: our bet is that diverse deployment on real-world tasks is the most effective way to move the frontier of physical AI capability outward.

What's next

I’m excited to - finally - be talking more about what we’re building at Walden. Our mission is to create robots that make people’s lives better and help us all live and work more purposefully. To succeed, our robots must be good at the use cases people actually care about. As a result, we’re building Walden around a core bet: the best physical AI will be built by whoever best deploys it.

Relatedly, we’re hiring! It’s something of a cliché to say “the team is amazing,” but I will anyway: Walden truly is special. I’m unreasonably fortunate to learn every day from exceptional colleagues across hardware, software, AI, robotics, product, and design. We’re looking for equally exceptional people to join us as we push the frontier of what’s possible while building on the incredible dynamic we already have.

More to come,

Ben Burchfiel

Co-founder and CTO, Walden Robotics