A robot chose the right box using its partner’s view, while sharing 80% fewer visual tokens

The robot doing the picking could not see which box was the target. Its partner could. In Astribot’s CoMind, also called Be My Eye, two S1 robots shared visual information during a controlled experiment. A person placed a blue plate into one of three boxes, hidden from the acting robot’s view. With the new method, it chose correctly in 16 of 18 trials, compared with seven without sharing and 13 when given generic visual features from its partner. These were real robots, not simulated agents.

The system learns to compress observations into information useful for the partner’s next action. It sends 32 visual feature tokens instead of the comparison method’s 160, an 80% reduction. Using the paper’s representation format, that means 144 KiB rather than 720 KiB per direction at each decision step, excluding network-protocol overhead. During training, errors in the receiver’s actions also update the sender’s information encoder. The sender is therefore trained to preserve details that help its partner act, rather than merely describe the scene.

In a second task, the robots took turns leading and following as they picked colored markers. The follower had to infer both the color and the moment to move from its partner’s shared information. CoMind completed all 48 two-stage trials; the no-sharing version completed none. The larger, generic visual-sharing baseline also completed every trial. Here, the achievement was preserving successful coordination with a smaller message, rather than inventing an ability the comparison system lacked.

The researchers also showed refrigerator restocking, box assembly, and blanket folding and packing. Such work illustrates why a second robot’s perspective could matter when a cupboard door or carried object blocks a household helper’s view. Those demonstrations did not include the same success-rate comparisons as the controlled tests, and larger teams remain future work. The paper appeared November 9, followed by Chinese coverage in QbitAI on November 13. These are the developer’s research results, not independent evidence of reliable everyday home use.

Independently rewritten and explained from the CC BY 4.0 paper by Baifu Huang, Shilin Fang, and Jianan Wang. License: https://creativecommons.org/licenses/by/4.0/ . Source times: paper, November 9, 2026, 20:19:15 Beijing / 07:19:15 EST; QbitAI, November 13, 2026, 14:29:23 Beijing / 01:29:23 EST.