LBM 2.0: Accelerating learning in deployment
We’re excited to share a look at our next-gen physical AI, LBM 2.0: Sage, our newest multimodal LBM powering Walden robots in production. Sage is a step-change in capability, learning rapidly on the job and benefiting more from deployment experience than weaker models. While Sage was designed for improved performance in deployment, we find that it is also a surprisingly powerful steerable generalist, capable of a wide variety of manipulation tasks and exhibiting strong compositional generalization. Sage also exhibits sparks of emergent capability such as collision avoidance and multi-step problem solving.
At a Toyota factory in North America, Walden robots work eight-hour production shifts on an engine manufacturing line. This is the type of environment Walden robots are deployed into, with unforgiving quality bars and strict cycle-time requirements. Our latest physical AI model for robot autonomy, LBM 2.0: Sage, is designed to improve performance in these settings - its primary design goal is stronger learning from deployment experience.
In deployment, Sage provides a large jump in performance, with mean time between intervention (MTBI) improving more than 4x over our preceding model generation. These improvements also reflect a broader jump in manipulation capability. Sage performs demanding manipulation tasks that earlier models struggle to complete, combines physical skills more flexibly, and learns far faster in deployment. In addition to strong deployment capability, Sage is surprisingly general, demonstrating significant compositional transfer and exhibiting sparks of emergent behavior, such as obstacle avoidance and goal-driven environmental manipulation. We also find evidence that stronger models like Sage are more robust deployment learners, able to learn more efficiently from deployment data and continuing to improve after weaker models saturate.
LBM 2.0: Sage is a 4.7-billion-parameter multimodal model trained on 100,000 hours of diverse data spanning cross-embodiment robot behavior, egocentric human recordings, UMI trajectories, simulation, and intervention traces. Task adaptation and corrective experience from our robots build on that foundation, grounding broad capabilities in the exacting requirements of deployment domains. Sage’s data recipe, training curriculum, action head, and model context are specifically designed to effectively combine heterogeneous data sources together with on-the-job learning and intervention data.
In an illustrative case study, one of the jobs our robots perform is transferring unfinished automotive engine camshafts into a conveyor for processing. The robot must extract individual parts from tightly packed dunnage (sometimes moving neighboring parts to free a grasp), identify the correct orientation, transfer the camshaft between hands, and place it into a conveyor fixture with millimeter-level accuracy. A placement error can wedge the part, each cycle must meet the line’s timing and quality requirements, and dropping parts is unacceptable.
We track both the fraction of time spent in autonomy and the mean time between interventions (MTBI). The first measures how much of the work proceeds under autonomous control; the second measures how frequently remote operator assistance is required. Intervention duration provides a complementary measure of how much assistance each correction requires. Together, these metrics capture the two key constraints on fleet scalability: how often robots need human help and how much operator time that help consumes.
Figure 1. LBM 2.0: Sage is a step-change in deployment. Recorded session autonomy and mean time between interventions across three model generations.
In this deployment, our first two generations of autonomy improved MTBI, one of the most crucial deployment metrics, from 22.6 to 78.5 seconds. LBM 2.0: Sage raises that to 340.3 seconds, more than five minutes between interventions on this task. At this level, a deployed fleet of 50 robots can be supported by as few as 3 remote assistants. We also observe significantly more behavioral robustness with the ability to change manipulation strategy if required, including repositioning difficult camshafts before grasping them.
Stronger Models are Stronger Deployment Learners
The central result of our experiments is that scaling pretraining improves the gains from deployment experience. After task adaptation, our strongest model nearly doubles its MTBI through deployment learning, while the smallest-data model improves only slightly despite receiving more corrective data.
1 of 5: Gearbox assembly
Figure 2. Examples from our manufacturing task suite. Sage autonomously performs machine tending, kitting, component handling, and assembly tasks, combining coordinated whole-body motion with dexterous manipulation.
To study the relationship between pretraining scale and learning from deployment, we compare LBM 2.0: Sage, pretrained on 100,000 hours of data, with Sage-1k and Sage-10k, pretrained on 1,000 and 10,000 hours, respectively. For consistency, all three use the same 4.7-billion-parameter architecture and training recipe, with Sage-1k being representative of our deployed gen 0 model and Sage-10k of our previous gen 1 model.
Figure 3. Comparing pretraining scale and deployment learning. Our model comparison pipeline, task-specific demonstrations provide initial adaptation; interventions collected during deployment provide experience for further training.
Our comparison covers seven customer-derived tasks: restocking and sorting, laser-cutter tending, 3D-printer tending, robot charging, and three kitting workflows involving robot assembly. These tasks combine mobility, precise manipulation, and sequences of actions in which mistakes can disrupt the work that follows.
First, we evaluate the generalist model after pretraining. Second, we adapt the pretrained model using an average of 6.2 hours of trainable demonstration data per task, and evaluate again. Third, we deploy each adapted model for three eight-hour shifts per task, with remote assistance when needed. The resulting corrective data is added to the training set for subsequent model updates, and a final evaluation measures the improvements from deployment learning.
Broad pretraining already makes a substantial difference. Before task adaptation, mean autonomy in our generalist evaluation increases from 25.1% for Sage-1k to 38.0% for Sage-10k and 51.2% for Sage. Sage can perform more of the work autonomously from the outset, although sustained execution still requires frequent assistance.
Figure 4. Broader pretraining improves initial capability. Mean autonomy before task-specific adaptation increases with pretraining scale.
The models separate further after learning from deployment. Sage reaches approximately 89% autonomy and 80 seconds MTBI, compared with 80% and 56 seconds for Sage-10k, and 67% and 28 seconds for Sage-1k. This nearly doubles Sage’s MTBI from approximately 41 seconds after task adaptation, while Sage-1k improves only slightly, from 27 to 28 seconds. Across these experiments, scaling pretraining substantially increases how much models improve from deployment experience.
Figure 5. Stronger pretrained models gain more from deployment experience. Autonomy and MTBI are measured before adaptation, after initial adaptation, and after deployment learning. The models follow the same deployment schedule, but accumulate different amounts of corrective training data because they require different amounts of assistance.
Over the same deployment schedule, Sage achieves higher performance while accumulating fewer corrective demonstrations, because it requires fewer interventions. Even from this stronger starting point, approximately 1.4 additional hours of intervention data per task nearly doubles its MTBI. This is the practical value of a stronger generalist: it starts from a higher level of capability, requires less assistance during learning, and derives greater gains from the experience it collects.
The differences are also visible in task execution. In laser-cutter tending, Sage progresses through opening the machine, handling the material, and closing the lid while Sage-1k spends longer on intermediate steps and frequently needs assistance. In assembling a gearbox, Sage aligns the chain with the gear teeth and seats it autonomously; Sage-1k repeatedly attempts the alignment and requires interventions.
1 of 2: Gearbox chain placement
Figure 6. More capable execution on demanding workflows. Side-by-side examples compare Sage-1k and Sage on laser-cutter tending and gearbox chain placement, illustrating differences in manipulation and task progression.
Requesting assistance is also part of the model’s capabilities. As described in our call-for-help work, a shared multimodal representation supports both action generation and the decision to request help. During a many-hour evaluation on a mock camshaft conveyor, the system recorded zero uncaught failures despite situations requiring assistance. The resulting experience can improve both task execution and recognition of when help is needed.
Sparks of emergent physical capability
Our goal for Sage was to maximize performance on customer tasks in deployment. However, as we spent more time exploring Sage through zero-shot prompting, we were continually surprised and delighted by the steerability and diverse manipulation capabilities of the base model. “Follow the person in yellow” requires identifying the intended person and moving with them. “Fetch a Coke and hand it over” combines object selection, retrieval, transport, and interaction with a person. The policy also exhibits information gathering behavior such as looking down at the floor when an object falls out of view, or looking around for a person when prompted “hand x to me.”
Figure 7. A broad repertoire directed through language. Examples combine object selection, dexterous manipulation, whole-body movement, and interaction with people. Playback speeds are indicated on screen.
In one instance, we invited non-Walden guests to interact with our robot in a kitchenette by submitting prompts directly to the policy. Over the course of an evening, the robot was given 88 unscripted prompts. Requests included “Can you open a protein bar for me,” “Pour me a beer,” “Put the cell phone in the freezer,” and “Bring the trash can to the table and sweep the popcorn into the bin.” Across these interactions, the robot achieved 99.2% autonomy.
We also see initial examples of emergent capability. Sage navigates around people but moves objects that obstruct its goal, suggesting distinct responses to people and movable obstacles that emerge directly from data. It takes intermediate steps to complete a task: moving a chair to reach sticky notes, reorienting a bottle to adjust its grasp, righting a tipped bin before resuming cleanup, or returning a fridge shelf and closing the door after retrieving a Coke. Prompted to “pick up another mug and bring it to the front table,” it searches the kitchenette, finds a mug, and brings it over.
1 of 8: Responding to a person
Figure 8. Sparks of emergent capability. Sage navigates around people, moves obstacles, searches for objects, and takes intermediate steps to complete tasks and recover from setbacks.
This breadth matters for deployment. A model that can reach, reorient, transfer, and manipulate objects in more ways has more capabilities available when adapting to novel situations and new workflows. These abilities also appear to correlate strongly with the ability to absorb data from the wild more effectively.
Measuring progress across tasks and environments
Evaluation is crucial for rapid capability development, and we’ve invested heavily in automating this pipeline. We automatically pull metrics from every robot session across the fleet, and complement this with robust structured testing in both real and simulation. Simulation plays a key role in providing a repeatable way to probe generalist capabilities, specific behavioral affordances, and estimate downstream deployment performance before shipping a model update.
Our simulation benchmark-generation pipeline uses an agentic workflow to construct diverse scenes and tasks. We then vary objects, lighting, distractors, backgrounds, and instructions to test each policy under a range of conditions. This lets us assess whether policy improvements hold across these variations.
Figure 9. Evaluating capabilities across diverse simulated environments. Language-directed manipulation tasks include controlled variations in objects, lighting, distractors, backgrounds, and language instructions. The same benchmark enables comparisons across model generations.
Scaling pretraining produces substantial gains on these evaluations. On the zero-shot benchmark, success rises from 12.0% at approximately 1,000 hours of pretraining data to 31.8% at 10,000 hours and 56.5% at 100,000 hours. Success on the observed-task benchmark rises from 50.0% to 72.3% and 88.3% at the same respective scales. These measurements are taken before any task-specific adaptation.
Figure 10. Scaling pretraining improves performance on observed and zero-shot tasks. Peak success rates across Sage pretraining runs increase with dataset size. Success on the zero-shot benchmark rises from 12.0% to 56.5%.
We also reconstruct deployment environments for targeted evaluation. Our real-to-simulation pipeline combines reconstructed workspaces, system identification, physics simulation in Drake, and photorealistic sensor simulation. These digital twins let us compare policy checkpoints across repeated trials and controlled changes to operating conditions.
For camshaft handling, checkpoint rankings in simulation match those observed in production. This helps us prioritize policies for physical evaluation and investigate the task stages where failures persist. Taken together, these capabilities allow us to complete the cycle of learning, testing, and deploying back into production in as little as a day, allowing for a highly reactive autonomy cycle.
Figure 11. Testing deployment policies in simulation. Reconstructed workspaces support repeatable evaluation of camshaft handling and tasks involving navigation, retrieval, and placement.
Broad benchmarks measure whether the generalist is becoming more capable. Reconstructed workspaces test how those capabilities translate to particular jobs. Together with physical evaluations and deployment experience, they give us a faster path from a promising model to improved performance at a customer site.
More Capable Robots at Work
LBM 2.0 connects progress in broad robot learning to measurable improvements in production. It performs demanding workflows more effectively, requires less frequent assistance, and gains more from the experience collected during deployment.
That combination expands the work we can take on. Each improvement to the foundation model gives new deployments a more capable starting point and a faster rate of improvement. Each deployment supplies experience grounded in real production requirements, and the cycle compounds.
