Frontier general-purpose models such as GPT-6 Astra have shown promising zero-shot performance on some manipulation tasks. Deployment, however, also requires reliable policies. Even leading models can make repeated errors in contact-rich interaction, alignment, and long-horizon task execution.
Human action demonstrations can improve reliability. But if every improvement requires new demonstrations, the supply of expert operators will not keep up with large-scale robot deployment.
Physical Intelligence is also considering this problem. In a recent interview, co-founder Sergey Levine described an initial stage in which robots acquire basic skills from high-quality human demonstrations. Once those skills are established, robots can learn through trial and error in the real world, using the data they collect themselves.
Following a similar line of thought, Tsinghua University, AgiBot, and Infinigence AI proposed Collective Experience Learning (CEL) and tested a prototype of the approach with 96 robots.
CEL paper cover
In a multi-stage industrial task involving mobile-phone packaging, the trained policy was evaluated in 120 trials on two same-model robots that had not participated in training. Success rate rose from 79.2% to 98.3%. Failures fell from 25 to 2 over the same number of trials, a 92% reduction.
The experiments also showed a benefit from scale. With the same amount of elapsed real-world training time, increasing the training fleet from 16 to 96 robots raised evaluation success rate from 86.7% to 98.3%. More robots produced more reliable task execution.
CEL overview
Technical report: https://arxiv.org/abs/2610.25736
Project website: https://cel-robot-learning.github.io/
01 Collective experience as a source of action supervision
Existing approaches to improving robot policies rely heavily on human intervention. Both imitation learning and human-in-the-loop reinforcement learning require people to demonstrate tasks or intervene when a robot makes a mistake. Operators need to perform actions fluently, decide when to intervene, and balance autonomous exploration with human demonstrations. Experienced operators are difficult to scale at the same rate as the number of training robots.
CEL changes where action supervision comes from during continued learning.
Suppose one robot struggles to place a phone while another has successfully completed a similar operation in a similar situation. The successful experience can help improve their shared policy, allowing practice by one robot to benefit the fleet.
People then shift from repeatedly providing actions and assisting exploration to judging outcomes.
CEL compared with earlier learning approaches
Making this idea work requires solving two problems. How can the learner identify useful experiences from large amounts of execution data with only 0/1 feedback? And how can an online robot fleet keep collecting experience despite limited communication and local failures?
02 Learning from sparse feedback
A task can last more than 3 minutes and contain over 4,000 steps, yet its final feedback is only one bit: 1 for success and 0 for failure. This distinguishes completed tasks from failed ones, but does not identify which parts of the policy need improvement or which successful experiences deserve more attention.
The team addresses this with Failure-guided Experience Retrieval Fine-Tuning (FERT). Successful trajectories supply actions to learn from; failed attempts identify situations where further learning is needed.
FERT framework
In FERT, robots execute tasks asynchronously using the initial policy. Successful trajectories enter a shared experience pool for subsequent fine-tuning. Failed trajectories identify where the model needs improvement. The algorithm extracts a sequence of observations near the end of a failed attempt, jointly encodes camera images and robot joint states, and retrieves similar action segments from successful experiences.
FERT uses successful and failed trajectories together. Successful segments matching a failure context receive higher sampling priority, directing training toward difficult situations.
Updated policies are sent back to the robots, which collect further successful and failed experiences. The value of collective experience depends both on accumulating data and on retrieving the right experience for the policy’s current weaknesses.
03 A training system for 96 robots
A fleet collecting experience continuously must handle differences in execution and reset times, network fluctuations, charging, and occasional hardware failures.
Restarting the entire training run after every local failure would make large-scale collaborative training impractical.
The Fault-tolerant Asynchronous Training System (FATS) dynamically coordinates 100 robots, 50 edge nodes, and 6 cloud servers. It treats robots as training resources alongside GPUs and schedules these resources within one system.
3.1 SwarmLink: Moving data between robots and training
The SwarmLink data plane connects environment, inference, and training nodes. Each robot submits data asynchronously after an execution round. Training can update the policy using the experiences that have arrived, without waiting for every robot to finish.
The system supports flexible placement across cloud and edge resources. Training and inference components can run in either location. Network bandwidth and topology can be configured to balance bandwidth-intensive weight synchronization against the latency requirements of frequent exchanges between inference and environment processes.
SwarmLink design
3.2 HotSwarm: Joining and leaving without stopping the fleet
The HotSwarm control plane manages nodes and worker processes as they enter and leave the system. When a robot pauses, disconnects, or rejoins, the system retains its registration and resource configuration, updates worker processes, and repairs affected connections.
In a 96-robot system experiment, the fleet collected 9,834 execution episodes. There were 30 failures involving 22 robots, but the resulting downtime accounted for only 1.3% of total robot-hours. Experience collection and training continued despite hardware failures.
HotSwarm design
04 Results on five real-world tasks
The experiments cover flower arrangement, clothes folding, phone charging, stool assembly, and phone packaging across household and industrial settings. These tasks involve deformable objects, precise manipulation, tool use, and long execution sequences.
Success rates on the five test tasks
The team ran 600 evaluations on two robots that had not participated in training, totaling nearly 30 robot-hours. There were 15 task failures, giving an overall success rate of 97.5% and a failure rate of 2.5%. A single robot completed clothes-folding and phone-charging tasks continuously without a task failure for 3 hours 34 minutes and 3 hours 19 minutes, respectively.
Conventional offline fine-tuning (SFT) achieved an average success rate of 36.3%. Multi-robot HG-DAgger using human demonstrations raised this to 81.0%. FERT under the CEL approach improved performance further. On every task, the final model exceeded the mean performance of six human teleoperators. Continued learning from collective experience produced more reliable execution without requiring further human action demonstrations.
05 Scaling trends in online robot learning
If the fleet can supply its own learning experience, can adding robots improve the policy further?
Real-world robot-learning scaling curves
The team progressively increased the number of training robots. At the same elapsed real-world interaction time, FERT showed a clear scaling trend on all five tasks: larger fleets achieved higher task success rates. HG-DAgger, which depends on human demonstrations, showed more limited scaling. Demonstration-collection capacity became a bottleneck as the fleet grew.
On phone packaging, the most difficult task, increasing the FERT training fleet from 16 to 96 robots raised evaluation success rate from 86.7% to 98.3%. With interaction time held constant, more robots directly improved task execution.
This suggests another way to scale embodied learning. Alongside collecting more human demonstrations, researchers can expand fleets of robots with basic autonomous skills and use their real-world interactions as a continuing source of experience.
06 Robots can continue learning together after deployment
Reliable, repeatable execution requires improvement through real interaction beyond an initial successful demonstration. CEL explores how robots that already have basic skills can contribute experience to a shared policy, turning individual practice into knowledge the fleet can use.
Deployment sites then become a source of learning data. Successful and failed task executions can feed into subsequent training.
The 96-robot experiments connect experience collection, experience retrieval, and sustained operation. They show that a larger robot fleet can improve learning in real-world settings. As more robots contribute experience, fleet size can become another way to improve policy capability.