Google's Willow quantum chip breakthrough is hidden behind a questionable benchmark

Google debuted Willow, its latest quantum chip, on Wednesday, and if you’ve spent any time online since, you’ve undoubtedly run into some breathless reporting about it. Willow “crushes classical computers on a cosmic timescale,” proclaims one headline; Google “unveils ‘mind-boggling’ quantum computer chip,” reads another. It’s all anchored by a claim that Willow can complete a computation that would theoretically take a classical computer significantly more time than the 14 billion years the universe has existed. But, as you can probably guess, what the chip represents is not so simple.

First, with Willow, Google makes no claim of quantum supremacy, something the company did when it publicly debuted its previous generation quantum computer, Sycamore, back in 2019. You may recall that, at the time, Google publicized how it took Sycamore just 200 seconds to perform a calculation that would have theoretically taken the world’s then-fastest supercomputer 10,000 years to complete. That feat, the company said, demonstrated that it had created a quantum computer that could solve problems the best classical computers could not even attempt. In other words, Google had achieved quantum supremacy.

However, that claim quickly ended in controversy, with one researcher calling the company’s announcement “indefensible” and “just plain wrong,” and Google has since avoided talking about quantum supremacy. Instead, it just says it has achieved “beyond classical computation.” Part of the issue was that Sycamore was not a general-purpose quantum computer; instead, it was designed to surpass classical computers in a single task known as random circuit sampling or RCS. The thing about RCS is that, in Google’s own words, it has “no known real-world applications.” Yet, here again, the company is touting RCS performance.

Google says Willow can complete its latest RCS benchmark in under five minutes. By contrast, the company estimates it would take Frontier, currently the world’s second most powerful supercomputer, 10 septillion years to complete the same task. That number, Google says, “lends credence to the notion that quantum computation occurs in many parallel universes, in line with the idea that we live in a multiverse.”

More practically, Google tries to make the case that RCS performance should be the metric by which all quantum computers are judged. According to Hartmut Neven, the founder of Google Quantum AI, “it’s an entry point. If you can’t win on random circuit sampling, you can’t win on any other algorithm either.” He adds RCS is “now widely used as a standard in the field.“

However, other companies, including IBM and Honeywell, instead use a metric called quantum volume to tout their breakthroughs. They claim it points to a more holistic understanding of a machine’s capabilities by factoring in how its qubits interact with one another. Unfortunately, you won’t find any mention of quantum volume in the spec sheet Google shared for Willow, making comparisons difficult.

To that point, the far more impressive claim Google is making today is that Willow is “below the threshold.” To date, the problem that has plagued every attempt to build a useful quantum computer is that the quantum bits they’re based on are difficult to control. They only hold their quantum state for fractions of a second, and the more qubits are added to a system, the more likely it is to produce errors. However, with Willow, Google says it has found a way to reduce errors as it adds more qubits to the system. According to the company, Willow is the first time this has been done.

“As the first system below threshold, this is the most convincing prototype for a scalable logical qubit built to date. It’s a strong sign that useful, very large quantum computers can indeed be built,” says Neven. “Willow brings us closer to running practical, commercially-relevant algorithms that can’t be replicated on conventional computers.”

That’s the real breakthrough here, and one that points to a future where quantum computers could solve problems that have tangible effects on people's lives. That future, however, isn't here just yet, and even Google admits it has more work to do before it gets there.