Peter Dürr could barely follow the table-tennis ball as it zoomed across the net, each strike’s trajectory designed to perplex the opponent. This was no ordinary match: Taira Mayuka, one of the top players in the world, was on one side—on the other, was a robot called Ace.

Mayuka launched a twisting smash that should have nailed a point. But in the blink of an eye, Ace answered with a return that kept the game alive. “Yes!” Dürr pumped his fist, knowing his team had engineered a historic moment for robotics.

Sony AI’s Ace is the latest autonomous system to be pitted against humans in a game. Since Deep Blue defeated chess champion Garry Kasparov in 1997, AI has trounced humans in Jeopardy, Go, StarCraft II, and car-racing simulations.

Ace has now taken these virtual victories into the real world.

Up against seven top human players, the AI-controlled robot arm beat three in multiple adrenaline-pumping games. Ace is an “important milestone,” wrote Carlos H. C. Ribeiro and Esther Colombini at the Aeronautics Institute of Technology and University of Campinas, respectively, who were not involved in the study.

Ace joins a humanoid robot that crushed the world record for a half marathon in Beijing last week. Neither project is focused on creating elite robotic athletes. Their main goal is to build next-generation autonomous machines that operate fluidly in the physical world.

“We wanted to prove that AI doesn’t just exist in virtual spaces,” Michael Spranger, president of Sony AI, said in a press release. “It’s not just tech you interact with in the virtual world—you can actually have a physical experience, and the technology is ready for that.”

Fast and Furious

Robots have come a long way. The clumsy, bumbling humanoids are gone, replaced by agile machines that can navigate all kinds of terrain. Autonomous vehicles once baffled by our roads now cruise the streets. Dexterous robotic arms are increasingly used for surgery, warehouse operations, or even delivering your lunch.

AI is a big part of that leap in capability. Robots are no longer strictly preprogrammed machines. They can now learn, adapt, make decisions, with generative AI models helping them understand what they’re looking at and, increasingly, how to interact with it. They’re a little less like yesterday’s rigid machines, and more like curious kids: Taking in a messy world, figuring it out, and getting better over time.

But compared to humans, robots still struggle to react on the fly, especially in fast-paced games like table tennis. The sport is a brutal mix of speed, perception, and precision. Players must read the ball and strike in a split second. There’s no margin for error. Too much power or the wrong angle, and the ball flies off the table. Too predictable, and you’ve likely handed your opponent the next point.

Professional players can smash shots up to 67 miles per hour and impart “a massive amount of spin on the ball,” exceeding 160 rotations a second, Dürr told Nature, making it tough for rookie humans and robots to react in time.

To Dürr, building a robot that could compete with elite human players was a “dream project” that “would challenge us to push the individual component technologies to their limits.”

Give Me Your Best Shot

Ace seamlessly fuses AI-based software and hardware.

For its eyes, the team placed cameras outside the court that could cover the entire playing area and track the ball’s position about 200 times per second. They also used an event-based image sensor to capture the ball’s spin. Together, these give the “robot the information it needs to anticipate where the ball is going to go, and plan how to hit it back,” said Dürr.

All that data feeds into multiple AI algorithms: Ace’s “brain.” One of these algorithms, borrowed from image processing, focuses on key parts of each frame to increase processing speed. Another, a deep reinforcement algorithm, learned to play table tennis in simulated matches. (Think student and coach: The model decides how to swing, where to aim, and how hard to hit. The “coach” gives feedback—good or bad—without demonstrating any moves.)

“So basically, we shoot a ball in simulation at our robot and let it do random things. At the beginning, it doesn’t know how to react…But eventually, it maybe be lucky enough to hit the ball back on the table,” said Dürr. And over countless iterations, it improves its play.

Expert players coached Ace too. In table tennis, the initial toss sets up the serve. Ace learned from human demonstrations adapted to its mechanics, so every toss follows the game’s rules.

After thousands of simulated hours, and with the help of yet another algorithm to weed out poor plays, the team built a library of realistic serves for Ace to draw upon.

The last component was the arm itself—and off-the-shelf didn’t work. “There’s nothing on the market that would let us play at the level we wanted to play,” said Dürr. So they built their own robot from the ground up. The lightweight, six-jointed arm can whip a racket at over 20 meters (roughly 66 feet) per second and react roughly 11 times faster than a person.

All assembled, Ace is a table-tennis powerhouse—but not unbeatable. Against five elite and two professional players, it dominated the less-experienced elites but fell to the pros. In the months since the team wrote up their results, the robot continued improving against top-tier competition.

Ace didn’t win by simply being faster than humans. Rather, it won by being inventive. It created different kinds of spins, varied its returns, and consistently landed the ball on target. When Olympic table-tennis player, Kinjiro Nakamura, watched Ace play, he was mesmerized by the robot’s unconventional moves. “No one else would have been able to do that. I didn’t think it was possible,” he said. But if a robot can pull it off, maybe humans can too.

For Colombini, who worked on soccer-playing robots, that kind of agility and improvisation is the real goal. Robots need to think on their feet and easily navigate the physical world to work safely with people. “I need the skills and the abilities of these robots, learned in these environments that are easy for us to see how they are evolving,” she said. “So, sports are just a proxy for what we want.”