According to the second law of thermodynamics, the total entropy of a closed process can increase or stay the same but never decrease. The second law guarantees, for example, that an egg can wobble off a table and leave a mess on the floor but that such a mess will never spontaneously form an egg and leap back on the table. Or that air will escape a balloon but never, on its own accord, inflate it. Since at least the 19th century, physicists have been investigating the role of entropy in information theory — studying the energy transactions of adding or erasing bits from computers, for example.
The thermodynamics of computation is a research focus of physicist and SFI Professor David Wolpert, and for the last few years he’s been collaborating with Artemy Kolchinsky, physicist and former SFI Postdoctoral Fellow, to better understand the connection between thermodynamics and information processing in computation. Their latest exploration of the topic, published in Physical Review E, looks at applying these ideas to a wide range of classical and quantum areas, including quantum thermodynamics.
“Computing systems are designed specifically to lose information about their past as they evolve,” says Wolpert.
If a person inputs “2+2” into a calculator and then hits “enter,” the computer outputs the answer: 4. At the same time, the machine loses information about the input since not only 2+2 but also 3+1 (and other pairs of numbers) can produce the same output. From the answer alone, the machine can’t report which pair of numbers acted as input. In 1961, IBM physicist Rolf Landauer discovered that when information is erased, as during such a calculation, the entropy of the calculator decreases (by losing information), which means the entropy of the environment must increase.
“If you erase a bit of information, you have to generate a little bit of heat,” says Kolchinsky.
Wolpert and Kolchinsky wanted to know: What is that energy cost of erasing information for a given system? Landauer derived an equation for the minimum amount of energy that is produced during erasure, but the SFI duo found that most systems actually produce more. “There’s a cost that appears beyond Landauer’s bound,” says Kolchinsky.
The only way to achieve Landauer’s minimum loss of energy, he says, is to design a computer with a certain task in mind. If the computer carries out some other calculation, then it will generate additional entropy. Kolchinsky and Wolpert have demonstrated that two computers might carry out the same calculation, for example, but differ in entropy production because of their expectations for inputs. The researchers call this a “mismatch cost,” or the cost of being wrong.
“It’s the cost between what the machine is built for and what you use it for,” says Kolchinsky.
In past papers, the duo has proven that this mismatch cost is a general phenomenon that can be explored in a variety of systems, not only in information theory but also in physics or biology. They’ve found a fundamental relationship between thermodynamic irreversibility—the case in which entropy increases — and logical irreversibility — the case in computation in which the initial state is lost. In a sense, they’ve strengthened the second law of thermodynamics.
In their latest publication, Kolchinsky and Wolpert demonstrate that this fundamental relationship extends even more broadly than they’d previously thought, including to the thermodynamics of quantum computers. Information in quantum computers is vulnerable to loss or errors due to statistical fluctuations or quantum noise, which is why physicists are searching for new methods of error correction. A better understanding of mismatch cost, Kolchinsky says, could lead to a better understanding of how to predict and correct those errors.
“There’s this deep relationship between the physics and information theory,” says Kolchinksy.
Read the paper, "Dependence of integrated, instantaneous, and fluctuating entropy production on the initial state in quantum and classical processes", in Physical Review E (November 9, 2021) doi: org/10.1103/PhysRevE.104.054107