Physicists Solve a Big Quantum Mystery. Now, Old Results Don’t Add Up.

New calculations seem to have put a 25-year-old particle physics puzzle to rest. But they’ve also created a clash with other experimental results. Introduction For 25 years, physicists have been puzzled by an apparent one-part-in-a-million problem. Their expectations of the way that certain particles should wobble in a magnetic field were clashing with what they saw in experiments. The discrepancy was an electrifying hint that they might be seeing evidence of unknown particles. Then in 2021, that hint seemed to evaporate. When researchers updated the way they did their theoretical calculations, they found that their predictions matched the experimental results much more precisely than before, to one part in 100 billion. But that, in turn, has created another puzzle: The old calculations seem perfectly valid. So why don’t they match the new calculations? Those older predictions were not purely based on theory; they were also inferred from other experiments. If the older calculations conflicted with newer results, and the older calculations were based on experimental data, was something strange going on in those old experiments? One promising clue comes from a particle collider in Siberia, which has recently started seeing its experiments dramatically diverge from what it and other colliders saw in the past. Its results have sparked a flurry of activity as physicists try to determine whether the conflicting measurements are a side effect of different experimental procedures, or a sign that new particles are popping up after all. Weird Wobbles The particle at the center of the mystery is the muon, a heavier cousin of the electron. A muon behaves a bit like a tiny bar magnet. Spin one in a circle inside a magnetic field and the magnetism will make it wobble, tracing out its own, smaller circles. The sizes of these circles are determined by a number called a “g-factor.” If the muon sat isolated from other particles, its g-factor would be exactly 2. But quantum theory requires that all other particles influence the g-factor. As the muon wobbles, it releases particles such as photons, which are too short-lived to show up in detectors. These can release other particles, which can release still more particles. The muon quickly reabsorbs all these fleeting particles, and the only trace they leave behind is that the muon wobbles a little bit more. Through these intricate chains of emission and reabsorption, every particle in existence has some small effect on the movement of the muon. That makes the precise size of the excess wobble, the muon’s “g–2,” invaluable as a window into the quantum world. “The measurement of muon g–2 is a proxy for saying how many particles exist in the universe,” said Alex Keshavarzi, a senior research fellow at University College London. So when an experiment at Brookhaven National Laboratory on Long Island measured the muon’s g-factor in 2001, physicists were thrilled that it came out larger than expected. To some, it hinted that new particles — perhaps even particles that could account for dark matter — were at work. Physicists set out to check the result with an even more precise measurement. In 2013, Brookhaven’s 50-foot-wide magnetic ring was moved via an elaborate series of barges and trucks to Fermi National Accelerator Laboratory (Fermilab) in Illinois, where an upgraded version of the experiment would take even more data. To prepare for that new experiment, physicists also made a huge effort to understand the theoretical prediction that disagreed with the data. Their challenge was to understand the muon’s chains of emission and reabsorption in extreme detail. In particular, how much do the particles associated with each of nature’s four fundamental forces participate in these chains? The calculation is straightforward for three of nature’s four forces. Gravity is so weak that physicists can ignore it outright. And both the electromagnetic force and the weak nuclear force can be deduced using a standard technique. The strong force, however, is not so easy to deal with. That force tightly binds particles known as quarks into composite particles such as protons and neutrons. Standard theoretical techniques don’t work on the strong force. So physicists have to get creative. In his doctoral thesis in 2018, Keshavarzi helped hone an alternative way of understanding the strong force, called the data-driven method. In this method, physicists don’t try to predict how often muons will emit and absorb groups of quarks. They go out and measure it. The main way that happens is by colliding electrons and their antimatter partners, positrons. The matter and antimatter annihilate each other, creating other particles, including bundles of quarks. If lots of quarks appear, physicists know they have a tight quantum link to particles such as electrons and positrons. In short, the more quarks appear in electron-positron collisions, the more strongly they will affect the muon. Using the data-driven method, physicists set out to calculate the expected size of the muon’s magnetic wobble. That prediction, which was released in June 2020, sharply differed from Fermilab’s precise experimental measurement, which came out in April 2021. The discrepancy was so strong that it nearly crossed the stringent threshold required for physicists to claim they had discovered new particles. But a different theoretical calculation would tell a different story. Wrangling Lattices Not all physicists pursued the data-driven method to calculate the muon’s wobble. Some were working on a more purely theoretical technique to make their prediction. The approach resembles what happens in weather forecasting. While it is possible, in principle, to understand the weather by keeping track of the precise contour of every breeze in the atmosphere, in practice that task is absurd. Instead, meteorologists divide the atmosphere into big boxes — a 3D grid — and calculate how each box changes on average over time. Likewise, it’s too hard for physicists to keep track of every strong-force interaction between every pair of quarks. So physicists use a technique called lattice QCD (short for quantum chromodynamics, the theory of the strong force), to use a big grid to simulate the overall behavior of quarks. In 2014, a collaboration among researchers in Budapest, Hungary; Marseille, France; and Wuppertal, Germany — the BMW group — started on a project to use lattice QCD to calculate the muon g-factor. At first, their predictions were 10 times fuzzier than data-driven inferences. Low-energy particles tend to spread out, so capturing their possible positions requires using a huge lattice. High-energy particles need a comparatively smaller grid, but one with an extremely fine mesh. “Back then, it was unimaginable that one day lattice would reach the same precision” as the data-driven method, said Kalman Szabo, a professor at Wuppertal who was involved in the effort. It took a decade of developing clever computational techniques — and waiting for increased computing power — for the BMW group to wrangle grids that were both sufficiently big and sufficiently detailed. But wrangle them they did. In 2021, on the same day that Fermilab released its updated muon g–2 measurement, the BMW group’s result appeared in the journal Nature. According to the BMW group’s lattice calculation, Fermilab’s muons were wobbling exactly as they should. Since then, independent lattice groups have published matching calculations. Today, many physicists believe the muon mystery is no more: According to the lattice simulations, the muon’s extra wobble can be explained entirely by the emission and reabsorption of known particles obeying the known laws of the known forces. So why does the data-driven method indicate otherwise? Inconsistent Experiments To figure out what’s going on, physicists are drilling into the electron-positron collisions driving the data-driven method. These collisions are supposed to be a direct window into quark behavior, but calculations based on this data disagree with both the latest experimental results and BMW’s prediction. So what’s really going on in the aftermath of those collisions? In the city of Novosibirsk in southern Siberia, the VEPP-2000 collider has been crashing electrons into positrons on and off since the turn of the millennium. It’s a relatively gentle collider, operating at 6,000 times lower energy than CERN’s Large Hadron Collider, near Geneva. The VEPP-2000 features two detectors that precisely count how often certain bundles of quarks, known as pions, pop out of the electron-positron crashes — data that physicists have been using to infer how much the strong force was messing with muons. In 2010, physicists installed a completely new detector. They then used it to more precisely measure this pion production rate, which they published in 2023. After the refresh, they found that the rate changed significantly. Fedor Ignatov, a physicist at the University of Liverpool in the UK, was a member of the team that measured a mysterious new rate of pion production at the VEPP-2000 collider. Courtesy of Fedor Ignatov “It was a surprise. No one expected it to be like that,” said Fedor Ignatov, a physicist at the University of Liverpool in the UK and member of the team. Physicists had seen faint hints that something strange was going on with the pion rate. They noticed that measurements of it from experiments in Italy and the United States were starting to drift apart. But the dramatic divergence of the new measurement from the detector’s own past results, along with the collaboration’s claim of high precision, made the situation hard to ignore. Physicists have pored over the result. “No measurement has been scrutinized more,” Keshavarzi said. So far, no problems have been found. Members of the BMW group then used a lattice simulation to predict the pion rate in 2024. Their calculation largely agrees with the most recent results. And preliminary data from the other detector at the VEPP-2000 collider also seem to match the new rate. Meanwhile, a 2023 analysis of data collected earlier at yet another experiment at a collider, BABAR in California, sits in striking agreement with the older rate. All of this leaves physicists wondering what’s really going on in all these collider experiments. The discrepancies point either to signs of unknown particles meddling with the quarks, or to overlooked details generating the mistaken impression that quarks are misbehaving. Either way, particle physicists can’t rest until they have solved the new electron-positron mystery, and figured out whether the old pion rate, or the new pion rate, is the right one. “There are four decades of measurements that preceded that, that were all done in different ways, that were all done by different people, that were all done by different experiments, that all paint a completely different picture,” Keshavarzi said. “There is so much still left to do.”