Well, so the past two posts I’ve done have been touching this philosophical question which, in many ways, is at the heart of science and knowledge (and even reality, as we shall soon see). Our first post mainly focused on what we meant by the act of measuring, and we touched somewhat on different kinds of measurements, and different metrics we could use. The second post related measurement to information, and we started discussing concepts that might have been a bit more overwhelming like irreversibility and entropy, with even a demon showing up.

This post is something like an extension of these ideas, a further exploration of thermodynamic demonology, if you will. For you see, Laplace’s demon was not, in fact, the sole demon plaguing and threatening physicists’ notebooks and blackboards. It was also threatening free will, but well, that might be a minor concern for some. Or rather, it was not fully defeated by the establishment of thermodynamics. And while lurking in the shadows for over half a century until rearing its head again, in the mind of one fellow called James Clerk Maxwell. Hence, this rebirth of the all-knowing demon came to be known as Maxwell’s demon. This iteration was clever, more sophisticated and above all, more cautious. It would not assert it’s all-knowing ability, no. Rather it would attack the very foundation of what had dealt with it in the past.

Laplace’s demon’s aim was to know everything from the understanding of physical laws, which were ultimately, its downfall. Maxwell’s demon was however, quite vindictive. Its goal was to tear down the laws which had previously put a stop to its maddening cosmic symphony.

Enough narrative cuckolding! Let’s get down to the details. The main flaw for Laplace’s demon was its reliance on the reversible nature of physical processes, and more, its passive role. It didn’t act upon the systems, being solely a passive observer. Because the system evolves naturally, irreversible processes can occur, removing the possibility for time prediction. Even if it could know all the physical states in a given instant, that information would not tell him anything of the system a few iterations in the future, nor could he know of the system past.

Maxwell’s demon’s key difference was its ability to control the evolution of the system. Just a short caveat before going into the actual thought experiment. Maxwell’s demon doesn’t actually deal with the causality of the universe, the scope of the system is much smaller, but it has universal implications.

There are several ways to state the second law of thermodynamics, all of which in a specific framework, but in this discussion, there are two relevant ones. The first statement is a blend of Clausius’ and Kelvin’s formulation, which basically state the same thing: Heat cannot pass from a cold body to a warmer body, without some external agent. The second statement, which is essentially Planck’s formulation is: In an isolated system, the entropy cannot decrease, and in a reversible process, it remains constant. By clearly having both statements of the 2nd law, it is now possible to see there is a clear relation between heat, temperature and entropy.

Where does Maxwell’s demon come? Right here, in the part where it says “Heat cannot pass (…) from cold (…) to warmer (…)”. To understand it better, we are going to set up a container filled with gas in it, some ideal gas, where you can think all the particles are essentially elastic balls. No weird quantum stuff yet. So you have this container and it has two chambers A and B. There is also this tiny door controlled by a button on the outside. The Maxwell’s demon is also incredibly intelligent, and is thus able to know exactly where every single particle is and how fast it is moving. And because he has the control in its hands, he can open the door whenever he wants. So he decides to open the door only when the fastest particle from B is moving towards A, or when the slowest particle from A is moving towards B, but not otherwise. Over a certain time, A and B, which were previously in thermal equilibrium, will change, with A being colder and B being warmer.

Maxwell’s demon doesn’t need to be all-knowing, just intelligent enough to know every single state of every single gas particle in the container. Now, there are several things to clarify here. The first one being the relation between the speed at which particles move and the temperature of a gas. Where does it come from?

Well, temperature is not a fundamental property, and by that we mean that it doesn’t appear when we break things down at the smallest scale, not that it isn’t important. Temperature is not something a single particle has and it doesn’t even make sense, but it is a measure of energy, more accurately of the average energy a particle has in an ideal gas. That relation is established by something called the equipartition theorem, which states that the temperature of a system and the average energy of a particle in that system are directly proportional to each other. So, an higher temperature means that, on average, the particles have more energy. If you remember your high school physics, you learn about something called the kinetic energy, which says that things with more energy move faster if they have the same mass. So a fairly direct relation can be established between temperature and speed, where in a system where temperature increases, the average speed of a particle also increases.

That implies that if we were to change the average speed in a chamber, say, by putting all the fast balls on one side, and all the slow balls in the other, we would increase the temperature difference between the chambers, as if heat was flowing from the colder chamber to the hot one. If we go back a few sentences, we can see that violates the second law of thermodynamics. Now we are in quite a pickle, for while Laplace’s demon was dealt with and is now a settled issue, the same can’t be said about Maxwell’s demon. It remained and lived on, and remains, to this century, a thought-provoking puzzle, and arguably tying thermodynamics with information to this day.

So, is that it? We lost? Well, not exactly. While Maxwell’s demon was a lot tougher to handle, it ultimately fell, not completely, but assuredly. And to find how we manage to defeat this genie of eldritch knowledge capabilities, we needed to change what we consider system.

You see, when I stated the second law of thermodynamics, there is one part that appeared in Planck’s formulation right in the beginning, which holds the key to our solution. “In an isolated system”. This suggests that entropy can sometimes decrease, but only if the system is not isolated. In thermodynamics, an isolated system is one in which there is no exchange of energy nor particles, where the classical principles of conservation of mass and energy hold. Other kinds of system explored in thermodynamics include closed system, in which there is exchange of energy but no exchange of particles, and open systems, in which there is exchange of energy and particles. And the neat thing is if you have an isolated system, while the global entropy will never decrease, you may have local variations where entropy does decrease. That’s because every system can be broken down into small systems, and in the same fashion, every system can be part of a larger system.

So perhaps this container with two chambers is not the full system we should consider, but rather the two chambers + the demon. The demon is itself part of our question, and rather than asking if the entropy of the container decreases we should ask, “Does the entropy of the container and the demon increase in total?”. Now does it?

Well, that is where the connection between thermodynamics and information shows up. Because if there is no exchange of energy between the demon and the container, which is closed (no exchange of particles either), how can the entropy of the demon increase, if it does at all? Well, the information that the demon acquires by “measuring” the particles is itself the reason why it’s entropy increases. There may not be an exchange of energy or particles, but there is an exchange of information. And that information requires memory to be stored. That storage leads to an overall increase in the demon’s entropy which offsets the decrease in entropy that is locally observed in the chamber, thus the total entropy of the system does not decrease and the second law of thermodynamics is saved, at last.

The argument for how the access to information and storage is linked to an entropy increase may be a bit more complex that I should delve into in this post, and to this day there is research and discussion around how to relate these two seemingly disconnected subjects.

I will admit I am too lazy to write up the sources below, but this post has been long overdue. If I forget to add them, please comment below reminding me to do so. Thanks in advance.