Next-Generation Retail Robots Tap Cloud AI as Human Training Takes Center Stage
A recent convenience-store trial run hosted on Amazon’s cloud infrastructure has highlighted both the remarkable potential of next-generation retail robots and the heavy reliance they still have on human training.
Amazon Web Services (AWS) spent a portion of this year collaborating with Tokyo-based robotics firm Telexistence to test whether a machine stationed at a shop counter can learn to think one step ahead. Telexistence is an established player in the automation space, reporting that it has deployed more than 300 robots across major Japanese convenience-store chains. While its current fleet primarily focuses on restocking drinks, the company is actively developing newer humanoid models designed to scan items and bag customer purchases directly at the register.
Despite the advanced hardware and machine learning algorithms powering these systems, the vast majority of what these robots know still comes directly from people. Human operators routinely steer the machines remotely through real-world tasks, capturing the operational sessions to serve as fundamental training data for future automation.
Integrating World Action Models in the Cloud
In a technical write-up published by AWS in September, engineers from Telexistence and scientists from the AWS Generative AI Innovation Center detailed an experiment integrating one of NVIDIA’s newest models into this operational setup. The model, known as DreamZero, operates as a world action model. Before executing any physical movement, the system predicts what its onboard cameras should encounter a fraction of a second later, planning its arm movements accordingly. While standard retail robots typically react strictly to what is immediately in front of them, DreamZero attempts to anticipate environmental changes and user actions.
Amazon provided the necessary high-performance computing resources and technical expertise for the trial. The model was fine-tuned and tested on AWS servers running advanced NVIDIA chips through the Physical AI Fellowship, a collaborative initiative managed by AWS, NVIDIA, and MassRobotics. Telexistence noted that it was selected as one of nine companies for the 2026 cohort and holds the distinction of being the first participant from Japan.
The trial results, however, underscored the significant technical hurdles that still face physical AI developers. In a simulated environment, the top-performing version of the model successfully completed a block-picking task 45% of the time. When the same model was trained using recordings captured from Telexistence’s physical robots and subsequently tested in that identical simulator, its success rate dropped to zero.
Engineering teams pointed to mismatches between the physical robot and its digital twin, alongside data noise within the recordings, as primary factors behind the discrepancy. On the fastest hardware evaluated during the tests, each operational step took approximately 583 milliseconds—a processing speed that engineers noted is close to the threshold required for true real-time robotic control. AWS clarified that these performance metrics stem strictly from internal testing environments.
Genki Sano, co-founder and chief technology officer at Telexistence, acknowledged the nuanced outcome of the experiment, noting that the work effectively mapped out "where DreamZero helps, where the real-to-sim gap bites."
Who Teaches the Robots
For years, Telexistence has marketed remote operation as a viable staffing solution for labor-strapped industries. When the company first introduced its Model-T shelf-stacking robot, executives emphasized that store staff could manage inventory from virtually anywhere with an internet connection. However, the recent DreamZero trial sheds light on the reverse side of that operational equation: the extent to which autonomous machines depend on the quality of data fed to them by human workers.
During the AWS trial, the engineering team analyzed 183 recorded demonstrations of the block-picking task. Human reviewers ultimately discarded 91 of those recordings, ruling nearly half of them unusable. In several instances, the robot remained idle for prolonged periods during the recording session. In others, the remote operator inadvertently selected a block of the incorrect color, or a bystander’s hand drifted directly into the camera’s field of view.
To streamline this quality-control process, the engineers developed an automated AI reviewer to screen the footage. The algorithmic filter flagged 72 recordings, or roughly 39.3% of the total dataset, demonstrating a level of accuracy close to what is required for a preliminary filter. However, the automated reviewer still struggled with basic visual distinctions, such as differentiating between blue and purple blocks.

From a human resources and operational perspective, these training workflows outline an entirely new category of labor. Workers must perform physical tasks cleanly and consistently under camera surveillance, while others must review, curate, and clean the resulting footage. Neither of these roles aligns neatly with traditional retail organizational charts. Establishing these functions will inevitably require formal standards, specialized training protocols, and structured compensation models.
Prior reporting from human resources publications has emphasized that when employees are filmed to generate robotic training datasets, employers must treat the recording sessions as formal workplace monitoring. Clear agreements regarding data ownership, financial compensation for training labor, and restrictions on whether performance-related footage can be utilized in standard workplace evaluations are critical considerations for organizations venturing into physical AI.
Why the Pressure Is Building
Japan serves as an ideal test market for retail automation primarily due to its accelerating demographic shifts and labor shortages. According to projections published in 2023 by the Recruit Works Institute—a research organization operated by Japanese employment firm Recruit—the nation faces a projected shortfall of approximately 3.41 million workers by 2030, a deficit expected to swell past 11 million workers by 2040.
In response to these tightening labor markets, Seven-Eleven Japan, which operates a massive network exceeding 20,000 stores, finalized an agreement in September 2025 to co-develop a humanoid robot alongside Telexistence. The resulting machine, designated as Astra, is slated for commercial deployment in convenience stores by 2029. Representatives from both companies stated that the ultimate objective is to fundamentally transform the daily responsibilities of store employees, offloading routine and physical tasks to machines so that human staff can concentrate on customer service aspects that require interpersonal engagement.
Retail environments in the United States are similarly transforming, driven by distinct labor market pressures. Projections from the U.S. Bureau of Labor Statistics indicate that overall cashier employment will decline by 10% between 2024 and 2034, shifting from approximately 3.16 million jobs down to 2.84 million. Nevertheless, persistent employee turnover continues to generate high volumes of hiring activity, with the agency anticipating roughly 542,600 job openings annually throughout the decade.
Human resources professionals and corporate leaders are urged to ensure that employers honor commitments to elevate the quality of work for remaining human personnel. Previous workplace studies highlighted in industry research indicate that working in close coordination with autonomous robots can occasionally diminish employees’ sense of personal autonomy and job meaning. Consequently, redesigning the human role in an automated workplace requires just as much strategic planning as deploying the robot hardware itself.
How Far Off Is It?
Despite the optimistic demonstrations frequently highlighted in promotional videos, industry insiders note that widespread commercial deployment remains some distance away. Footage from behind-the-scenes development cycles reveals the persistent operational challenges that rarely appear in polished advertising campaigns. In a June technical update regarding its autonomous bagging robot, Telexistence shared unedited footage played at both normal speed and eightfold speed, explicitly stating that the machine is not yet operating at speeds suitable for active commercial deployment.
The physical movements of the robots remain prone to occasional jerks, and shifting an item by just a few centimeters can result in a sharp drop in operational accuracy. Sano characterized these public demonstrations strictly as a foundational starting point rather than a finished product ready for mass deployment.
Market research firms maintain a measured outlook regarding commercial timelines. Industry analysts at Gartner have projected that fewer than 20 companies globally will successfully scale humanoid robots into active production stages within supply chain and manufacturing sectors through 2028.
Ultimately, whether these technologies represent the immediate future of convenience store operations in markets like Japan depends heavily on whether major retail chains can adhere to ambitious deployment timelines toward the end of the decade. For the broader business community, however, the more immediate operational shift highlighted by these trials lies in the creation of specialized roles required to train, monitor, and maintain the robotic systems set to enter the modern workplace.
| Source | : | Open Passport |
| Author | : | Lina Irawan |
| Editor | : | Open Passport Editorial Team |
