Meyrin, Switzerland, sits serenely near the Swiss-French border, surrounded by green fields and the beautiful Rhône river. But a hundred meters beneath the surface, protons traveling at nearly the speed of light collide and create spectacular displays of subatomic fireworks inside the experimental detectors of the Large Hadron Collider at CERN, the European particle physics laboratory.

One detector, called ATLAS, is five stories tall and has the largest volume of any particle detector in the world. It captures the trajectory of particles from collisions that happen a billion times a second and measures their energy and momentum. Those collisions produce incredible amounts of data for researchers to scour, searching for evidence of new physics. For decades, scientists at ATLAS have been optimizing ways to archive their analysis of that data so these rich datasets can be reused and reinterpreted.

Twenty years ago, during a panel discussion at CERN’s First Workshop on Confidence Limits, participants unanimously agreed to start publishing likelihood functions with their experimental results. These functions are essential to particle physics research because they encode all the information physicists need to statistically analyze their data through the lens of a particular hypothesis. This includes allowing them to distinguish signal (interesting events that may be clues to new physics) from background (everything else) and to quantify the significance of a result.

As it turns out, though, getting a room full of particle physicists to agree to publish this information was the easiest part. 

In fact, it was not until 2020 that ATLAS researchers actually started publishing likelihood functions along with their experimental results. These “open likelihoods” are freely available on the open-access site HEPData as part of a push to make LHC results more transparent and available to the wider community.

“One of my goals in physics is to try and make it more accessible,” says Giordon Stark, a postdoctoral researcher at the University of California, Santa Cruz, who is on the development team for the open-source software used to publish the likelihood functions.

The US Department of Energy's Office of Science and the National Science Foundation support US involvement in the ATLAS experiment.

Stark says releasing the full likelihoods is a good step toward his goal. 

The problem with randomness 

Why are likelihoods so essential? Because particle collision experiments are inherently random. Unlike in a deterministic experiment, where a researcher does “x” and expects “y” to happen, in a random experiment (like throwing dice or colliding beams of protons), a researcher can do “x” the same way every time but can only predict the random outcome probabilistically.

Because of the inherent randomness of particle interactions in the ATLAS detector, physicists need to construct what is called a “probability model” to mathematically describe the experiment and form meaningful conclusions about how the resulting data relate to a theory. 

The probability model is a mathematical representation of all the possible outcomes. It’s represented by the expression p(x|θ): the probability “p” of obtaining data “x,” given the parameters “θ.” 

The data are observations from the ATLAS detector, while the parameters are everything influencing the system, from the laws of physics to the calibration constants of the detector. A few of these parameters are central to a physicist’s model (they’re called “parameters of interest”—things like the mass of the Higgs boson), but hundreds of other “nuisance parameters” (things like detector responses, calibration constants and the behavior of the particles themselves) also need to be taken into account.

When experimentally observed data are plugged into the probability model, they return a likelihood function, which determines the values of the model’s parameters that best describe the observed data. 

Importantly, the process answers the question of how likely it would be for a physicist’s theory to have produced the data they observe. 

A new tool comes to the rescue 

When you consider the hundreds of parameters in an ATLAS analysis, each with their respective uncertainties, along with the layers of functions relating the parameters to each other, calculating the likelihoods gets pretty complicated—and so does presenting them. While likelihoods for one or two parameters can be plotted on a graph, this clearly isn’t possible when there are hundreds of them—making the question of how to publish the likelihoods much more challenging than whether this should be done. 

In 2011, ATLAS researchers Kyle Cranmer, Wouter Verkerke and their team released two tools to help with this. One, called the RooFit Workspace, allowed researchers to save their likelihoods in a digital file. The other, called HistFactory, made it easier for users to construct a likelihood function for their theory. Since then, the HistFactory concept has evolved into an open-source software package, spearheaded by Stark and fellow physicists Matthew Feickert and Lukas Heinrich, called pyhf [pronounced in three syllables: py h f]. 

Cranmer says it’s important to understand that pyhf isn’t some magical black box where you put data in and get a likelihood out. Researchers need to make lots of decisions in the process, and “every little bit of that likelihood function should be tied to part of the justification that you have for it and the story that you’re telling as a scientist,” he says.

After interpreting these decisions, pyhf exports the probability model in a plain-text, easy-to-read format called JSON that can be read across a range of platforms, making it easier for other researchers to access the likelihood function and see how the analysis was done.

“The important part is that it’s readable,” says Cranmer. “The stuff you’re reading is not some random, technical gobbledygook. It’s tied to how a physicist thinks about the analysis.”

Making old data do new work

Before the RooFit Workspace came along, the thousands of researchers involved in the ATLAS collaboration had no standardized way to format and store data likelihood functions. Much of the meticulous data analysis was done by PhD students who eventually graduated and left for new positions, taking their intimate familiarity with likelihood construction along with them.  

Without the full likelihood function, it’s impossible to reproduce or reinterpret ATLAS data from published results without having to make possibly crude approximations. But with the layers of rich metadata embedded in the pyhf likelihoods, including background estimates, systematic uncertainty and observed data counts from the detector, scientists have everything they need to mathematically reconstruct the analysis. This allows them to reproduce and reinterpret previously published results without repeating the time-consuming and expensive process of analyzing the data from scratch.  

Public likelihoods also provide fantastic opportunities for reinterpretation by theorists, says Sabine Kraml, a theoretical physicist at the Laboratory of Subatomic Physics and Cosmology in Grenoble, France, who has been involved with helping establish how LHC data, including the likelihoods, should be presented.

With full likelihood functions, theorists can calculate how well their theories fit the data collected by the detector “at a completely different level of reliability and precision," says Kraml. 

To understand just how much more sophisticated and complex the analysis becomes, she says, consider the difference between a simple song and a full orchestral symphony.

Although this precise model-fitting is limited to theories that share the same statistical model as the one originally tested by the experiment—“It’s a restricted playground,” Cranmer says—there is a work-around. Full likelihoods can be put through an additional round of processing called recasting, using a service Cranmer proposed called RECAST, which generates a new likelihood function in the context of a physicist’s theory. Armed with this new likelihood, scientists can test their theories against existing ATLAS data, searching for new physics in old datasets.

So far, two ATLAS searches have been repurposed using RECAST. One used a dark-matter search to study a Higgs boson decaying to bottom quarks. The other used a search for displaced hadronic jets to look at three new physics models.

Cranmer says he hopes the ATLAS experimental community will continue to publish their likelihoods and take advantage of RECAST so the wider scientific community can test more and more theories.

All Clifford Charles Butler and George Rochester knew for sure was that they’d discovered something new. Photographs from their cloud-chamber experiments at the University of Manchester revealed the tracks of two particles that behaved unlike anything they’d seen before. 

When the two physicists published their results in 1947 in the journal Nature, they could have had only the dimmest notion that their discovery would in time upend the world’s understanding of elementary particle physics.

Observations of kaons, as the particles came to be known, and other similar particles led physicists on a path toward the discoveries of new quantum properties of matter, new particles—including quarks—and the downfall of a once-sacrosanct construct called CP symmetry, an exact relationship between the laws of physics for matter and those for antimatter. Today, high-precision experiments with kaons are helping researchers probe the limits of the same Standard Model the particles helped usher in. 

But back in Butler and Rochester’s day, physicists were mostly left scratching their heads, says Helen Quinn, an emerita professor of physics and astrophysics at the US Department of Energy’s SLAC National Accelerator Laboratory.

“Kaons didn’t fit any picture” physicists had at the time, she says. In fact, when physicists realized they needed a new quantum property to describe the particles, it “was called ‘strangeness,’ because the particles had always seemed a bit strange.”

The particle zoo expands

By the start of the 1940s, it seemed like physicists were getting a handle on the fundamental particles and their interactions. They knew about electrons, protons and neutrons, as well as neutrinos and even positrons, the “antiparticles” of electrons Paul Dirac had predicted in the 1920s. They understood that there were forces beyond gravity and electromagnetism, the strong and weak nuclear forces, and were working to better understand them.

But puzzles emerged as unexpected new particles appeared. Physicists discovered muons in cosmic rays using a cloud chamber experiment in 1936. (The name “cloud chamber” comes from the fact that electrically charged particles travelling through water vapor form tiny trails of clouds in their wake.) They found pions by similar means in 1947.

That same year, Butler and Rochester announced they’d found particles they called V⁺ and V⁰. From a set of “unusual fork[s]” in their data, they inferred the existence of two fairly massive particles, one positively charged and the other neutral, that had broken apart into other particles.

The particles had a number of curious features. For one thing, they were heavy—around five times the mass of a muon—which led to another puzzle. Ordinarily, heavier particles have shorter lifetimes, meaning that they stick around for less time before decaying into other, lighter particles. But as experiments continued, researchers discovered that despite their heft, the particles had relatively long lifetimes. 

Another odd feature: The particles were easy to make, but physicists never seemed to be able to produce just one of them at a time. Smash a pion and a proton together, for example, and you could create the new particles, but only in pairs. At the same time, they could decay independently of each other.

A strange new world

In the 1950s, Murray Gell-Mann, Kazuo Nishijima, Abraham Pais and others devised a way to explain some of the curious behaviors kaons and other newly discovered particles exhibited. The idea was that these particles had a property called “strangeness.” Today, physicists understand strangeness as a fundamental, quantum number associated with a particle. Some particles have strangeness equal to zero, but other particles could have strangeness equal to +1, -1, or in principle any other integer. 

Importantly, strangeness has to remain constant when particles are produced through strong nuclear forces, but not when they decay through weak nuclear forces.

In the example above, in which a pion and a proton collide, both of those particles have strangeness equal to 0. What’s more, that interaction is governed by the strong force, so the strangeness of the resulting particles has to add up to zero as well. For instance, the products could include a neutral kaon, which has strangeness 1, and a lambda particle, which has strangeness -1, which cancels out the strangeness of the kaon.

That explained why strange particles always appeared in pairs—one particle’s strangeness has to be canceled out by another’s. The fact that they’re built through strong interactions but decay through weak interactions, which tend to take longer to play out, explained the relatively long decay times.

These observations led to several more fundamental insights, says Jonathan Rosner, a theoretical physicist at the University of Chicago. As Gell-Mann and colleagues developed their theory, they saw they could organize groups of particles into bunches related by strangeness and electric charge, a scheme known today as The Eightfold Way. Efforts to explain this organization led to the prediction of an underlying set of particles: quarks.

The long and short of it

Another important feature of the strangeness theory: When scientists found that strange kaons could decay into, for example, ordinary pions, they surmised that the weak nuclear interaction, unlike the strong nuclear interaction, did not need to keep strangeness constant. This observation set in motion a series of theoretical and experimental developments that physicists are still grappling with today.

Building on theories that suggested the neutral kaon ought to have an antiparticle with opposite strangeness to the standard neutral kaon, Gell-Mann and Pais reasoned that the neutral kaon could, through complex processes involving weak interactions, transform into its own antiparticle. 

The scheme has a significant consequence: It implies that there are two new particles—actually different combinations of the neutral kaon and its antiparticle—with different lifetimes. K-long, as it’s now called, lasts on average about 50 billionths of a second, while K-short lasts just under one-tenth of a billionth of a second before breaking apart. The prediction of these particles was among Gell-Mann’s favorite results, Rosner says, because of how easily they emerged out of basic quantum physics. 

A symmetry of nature, dethroned

One of the important things about K-long and K-short, at least in Gell-Mann and Pais’s theory, was that they obeyed something called CP symmetry. Roughly, CP symmetry says that if one were to switch every particle with its antiparticle and flip space around into a sort of mirror-image universe, the laws of physics would remain the same. CP symmetry holds in all classical physics, and it was CP’s quantum variant that motivated Gell-Mann and Pais. (Technically, Gell-Mann and Pais were originally motivated by C symmetry alone, but they had to update their theory once experiments determined that weak interactions violated both charge conjugation and parity symmetry—but in such a way that CP itself seemed to remain a good symmetry.) 

Ironically, a result motivated by CP symmetry led to its downfall: In 1964, James Cronin, Val Fitch and collaborators working at Brookhaven National Laboratory discovered that the K-long could—very rarely—break up into two pions, a reaction that violates CP symmetry. Kaon decays did violate CP symmetry after all.

The cosmic gift that keeps on giving

By the early 1970s, Quinn says, physicists developing the Standard Model needed a way to incorporate CP violation. In 1973 Makoto Kobayashi and Toshihide Maskawa, building on work by Nicola Cabibbo, proposed the solution: The Standard Model needed an extra pair of quarks beyond what they had already theorized. They also predicted that certain quarks could decay through weak interactions into other quarks in ways that violate CP symmetry. Throughout the 1980s and ’90s, kaon experiments such as KTeV at Fermilab and NA48 at CERN—along with B-meson experiments such as BaBar at SLAC and Belle at KEK—probed how such interactions led to CP violation. 

Over the years, theorists had also made ever more precise predictions about the various ways kaons could break apart. So precise are these predictions, says Yau Wah, a physicist at the University of Chicago, that searching for rare kaon decays remains among the best ways to test the Standard Model. 

“The merit of [these tests] is because the Standard Model is too successful,” says Wah, who works on the K0TO experiment, a Japanese project to search for neutral kaons decaying into a pion, neutrino and antineutrino. 

In the next five years, Wah says, K0TO will likely be in a position to say whether the Standard Model’s tight predictions related to that decay are correct. If not, it could indicate new sources of CP violation beyond the Standard Model.

Such studies are also a good way to probe physics at extremely high energy scales, since any deviations from current predictions would require new particles with enormous masses—perhaps a million times that of the proton, says Cristina Lazzeroni, a physicist at CERN’s NA62 experiment, which focuses on rare decays of charged kaons. 

Lazzeroni says NA62 has already found some evidence of positively charged kaons decaying into positively charged pions and two neutrinos, and that in the next five years they plan to probe Standard Model physics to a level of accuracy that will allow them to see whether there is new physics to be found. What started in a simple cloud chamber three-quarters of a century ago is continuing in some of the most precise experiments ever done, and kaons are likely to keep on giving for years to come.