The Second Law of Thermodynamics and Quantum Feedback Control: Maxwell's Demon with Weak Measurements
aa r X i v : . [ qu a n t - ph ] A ug The Second Law of Thermodynamics and Quantum Feedback Control:Maxwell’s Demon with Weak Measurements
Kurt Jacobs
Department of Physics, University of Massachusetts at Boston, Boston, MA 02125, USA
Recently Sagawa and Ueda [Phys. Rev. Lett. , 080403 (2008)] derived a bound on the workthat can be extracted from a quantum system with the use of feedback control. They left open thequestion of whether this bound could be achieved for every measurement that could be made bythe controller. We show that it can, and that this follows straightforwardly from recent work onMaxwell’s demon by Alicki et al. [Open Syst. Inform. Dynam. , 205 (2004)], for both discreteand continuous feedback control. Our analysis also shows that bare, efficient measurements alwaysdo non-negative work on a system in equilibrium, but do not add heat. PACS numbers: 05.30.-d, 05.70.Ln, 03.67.-a, 03.65.Ta
The amount of work that can be extracted from a ther-modynamical system, when it undergoes a process takingit from an initial equilibrium state S at temperature T ,to a final equilibrium state S at temperature T , is givenby the difference in the (Helmholtz) free energy, F , be-tween these states, where F ≡ E − T S, (1)with E , T and S being, respectively, the average inter-nal energy, temperature and entropy of the system. Forquantum systems the entropy is the von Neumann en-tropy, S = − Tr[ ρ ln ρ ], where ρ is the density matrix ofthe state. The simplest example of this is the work ob-tained by the (quasi-equilibrium) expansion of a gas at afixed temperature (isothermal). In this case the internalenergy of the gas remains constant, and the work doneby the gas is ( S − S ) /T = F − F = ∆ F [1].This relation between maximum work and free energyis true for the traditional thermodynamic processes —that is, ones that do not have access to the microstatesof the system. If we measure the system, so as to ob-tain information about the underlying microstate, andperform actions based on this information, then we canextract more work. This is a process of feedback con-trol [2]. Of course, in this situation, a feedback controlleris merely a Maxwell demon [3–11]. Sagawa and Ueda re-cently showed that the amount of extra work that canbe extracted by a feedback controller, over and abovethe free-energy difference, is bounded by a measure ofthe information extracted by the controller [12]. In theiranalysis this bound was only achieved for certain specialclasses of measurements (e.g. von Nuemann measure-ments). Here we show, by generalizing the protocol ofAlicki et al. [9], that feedback controllers can always sat-urate the bound, irrespective of the measurement theymake.The following analysis is divided into three parts. Thefirst part introduces some important thermodynamic def-initions. In the second part we derive the relationshipbetween free energy and work extraction with feedbackcontrol, building upon previous results by Alicki et al. [9]on Maxwell’s demon. Lastly, we treat the feedback con- troller as purely quantum mechanical, eliminating the useof quantum measurement theory, to show that the sec-ond law is preserved by the control process, in agreementwith Landauer’s erasure principle [4, 13–17]. Quantum Mechanics, Work, and Heat : The averageenergy of a quantum system is given by E = Tr[ Hρ ],where H is the Hamiltonian. We can therefore write dE = Tr[ dHρ ] + Tr[ Hdρ ]. In the past the work doneon the system has been equated with the first term, dW ≡ Tr[ dHρ ], and the heat entering the system withthe second: dQ ≡ Tr[
Hdρ ] [9, 10]. These identificationsare not subtle enough for our analysis here, however, be-cause of transformations induced by the measurement.This necessitates splitting dρ into a number of parts.Transformations that change the eigenbasis of ρ , suchas unitary operations, and preserve the populations ofthe eigenstates (and thus the entropy of the system) cor-respond to work done on or by the system. Alterna-tively, processes that leave the eigenbasis of ρ fixed, butchange the populations correspond to adding or subtract-ing heat, or information extraction.To examine the work done on a system by a quan-tum measurement, note first that the transformation of ρ caused by any efficient measurement can be written as ρ → AρA † / Tr[ A † Aρ ] , (2)for some operator A . The polar decomposition theoremallows us to write A = U P , where U is unitary, and P ispositive, and this allows us to break up the transforma-tion into a unitary part (work done) and an informationextraction part that reduces, on average, the entropy.The operator U can also include a unitary feedback oper-ation based upon the measurement result, and, therefore,can also be undone by the use of feedback. We will re-fer to measurements that have no unitary part as “bare”measurements [18].To fully isolate the work done by the measurement wemust consider the action of a positive operator P in alittle more detail. If P commutes with the density ma-trix, the only change is to the entropy. But if P does notcommute with ρ , then the action of the measurementchanges both the eigenbasis of the density matrix, doingwork, and generates the entropy change (extracting in-formation). We will show in the analysis below that theaverage work done on a system in thermal equilibriumby a bare measurement is always non-negative.The thermal equilibrium state of a system at temper-ature T is ρ T = e − H/ ( kT ) Tr (cid:2) e − H/ ( kT ) (cid:3) , (3)where H is the system Hamiltonian. This state cap-tures the fundamental assumption of statistical mechan-ics (that all accessible microstates are equally likely).Given this assumption, it tells us how the entropy willchange with other quantities in a quasi-equilibrium pro-cess. Extracting Work with Feedback Control:
To obtain theamount of work that can be extracted by a feedback con-troller, we will start our system in an equilibrium state ρ T , denoting the initial average energy by E and the ini-tial entropy by S . The first action of the controller isto measure the state, using an arbitrary quantum mea-surement described by a set of operators { P n } , satisfying P n P n = 1. The operators P n are all positive, since theunitary operator associated with each measurement out-come will be determined by the feedback chosen by thecontroller. After the measurement result, which occurswith probablity p n = Tr[ P n ρ ], the state is ρ n = P n ρP n /p n . (4)This is no-longer an equilibrium state, but its entropyand average energy are well-defined (its temperature isnot). Call its entropy S n and its average energy E n .Note that in general E n = E , because the energy willhave been changed by the measurement; this is the workdone on the system by the measurement process. We willreturn to this later.Now comes the first part of the feedback control pro-cess. The controller performs work (reversible opera-tions) on the system to transform ρ n to an equilibriumstate at temperature T . This is achieved by i) per-forming a unitary operation to transform the eigenba-sis of ρ n to the energy eigenbasis; ii) re-ordering thepopulations of the energy states so that these popula-tions decrease monotonically with increasing energy. Wewill denote the populations (the eigenvalues of ρ n ) as λ nj , and the corresponding energy levels of the systemas ε nj ; iii) adjusting the Hamiltonian so that the sep-arations between adjacent energy levels are such as toset P nj ( T ) ≡ exp( − ε nj / ( kT )) /Z = λ nj , where Z = P j exp( − ε nj / ( kT )) is the partition function, and wehave defined P nj ( T ) as the populations of the energyeigenstates required for the system to be in thermal equi-librium at temperature T ; iv) adjusting the Hamiltonianto produce an overall energy shift of the levels so as toreturn the average value of the energy to the initial value, E . This leaves the system in thermal equilibrium at tem-perature T , since the values of the populations, P nj ( T ), required for this to be true remain unchanged by the en-ergy shift.The above feedback extracts net work from the systemof ∆ E n = E n − E , preserves the entropy of the state, S n ,and leaves the state in equilibrium at temperature T .In the second part of the feedback process, the con-troller performs an isothermal expansion of the system(decreasing the separation of the energy levels at fixedtemperature), so as to return the entropy to the ini-tial value, S . This brings the system precisely back toits initial thermal state, since the energy, temperature,and entropy of the system have all returned to their ini-tial values. The isothermal expansion extracts ∆ W n = T ( S − S n ) of work from the system. The total work ex-tracted by the feedback controller in this cycle, given themeasurement result n , is thus ∆ W n = T ( S − S n ) + ∆ E n .Of course, the important quantity is the total average work extracted by the feedback, where the average is overthe possible measurement results. This is,∆ W = T S − X n p n S n ! + X n p n ∆ E n . (5)We now examine the second contribution, the work ex-tracted deterministically by the feedback, P n p n ∆ E n .This is simply the average increase in the energy ofthe system caused by the measurement, ∆ E meas = P n p n ( E n − E ), being extracted back by the controller. Ifthe measurement is classical, so that all the measurementoperators, P n , commute with the initial state ρ T (that is,the controller measures the systems energy), then the av-erage density matrix after the measurement is the sameas the initial state: ρ after = P n p n ρ n = ρ T . From this itfollows immediately that ∆ E meas = 0. Thus, as expectedfrom our previous discussion of work and energy, whenthe measurement does not change the eigenbasis of thedensity matrix, then it does not, on average, add energyto the system. When the measurement operators do notcommute with ρ T , then one has S ( ρ after ) ≥ S ( ρ T ), a re-sult shown by Ando [19]. Because the equilibrium state, ρ T , is the state with the maximum entropy given a fixedvalue of the average energy, it follows that E after ≥ E .We therefore have∆ E meas = E after − E ≥ . (6)Because the controller can always extract back as workall the energy added to the system by the measurementin a closed cycle (Eq.(5)), to preserve the second lawof thermodynamics we must interpret ∆ E meas as workadded to the system, not heat. This is consistent withour observation that the action of a positive measurementoperator induces a transformation of the density matrixeigenbasis.The total work extracted by the controller in a singlecycle is the work extracted by the feedback process, mi-nus the work done on the system by the measurement,and is therefore ∆ W fb = T ( S − P n p n S n ). This is fora cycle in which the system starts in equilibrium with agiven free energy, and returns back to its initial state. Itnow follows immediately that the work extractable by afeedback controller when starting in state S with freeenergy F , and ending in state S with free energy F , is∆ W fb = ∆ F + T S − X n p n S n ! , (7)where ∆ F = F − F . The right hand side of this equationis the upper bound derived in [12].The quantity ∆ S meas ≡ S − P n p n S n , being the aver-age entropy reduction provided by the measurement, isalways non-negative for efficient measurements, a resultdue to Ozawa [20–22]. This is a key quantity in quantumfeedback control even outside thermodynamical consid-erations [23], and reduces to the classical mutual infor-mation when the measurement is classical. The Second Law:
The feedback control process wehave just described reduces the entropy of the bath, onaverage, by ∆ S meas during the isothermal expansion ofthe system (the system gains this amount of entropy fromthe bath). Since the final entropy of the system is thesame as its initial entropy, the whole process will breakthe second law of thermodynamics (reduce the entropyof the universe), if the entropy of the controller does notincrease by at least ∆ S meas . The simplest way to showthat the entropy of the controller does increase by therequired amount is to treat the controller fully quantummechanically. This allows us to treat the whole feedbackprocess without using quantum measurement theory. Aspointed out by Wiseman, any feedback control processbased on explicit measurements (that is, with a controllerwhose states are classically distinguishable, and thus donot exist in superposition states) can always be imple-mented with a quantum controller, without any explicitmeasurements [24].We will denote the controller as C , and the system as S . The measurement process is completely described bya unitary operation acting on the space of both systems.The controller has N states, | n i , n = 0 , . . . , N −
1, where N is the number of measurement results. The initialstate of the controller is | i , and that of the system is,of course, ρ T . A joint unitary operation correlates thesystems so that the joint state becomes ρ CS = X n p n | n ih n | ⊗ ρ n + X n,m = n | n ih m | ⊗ σ nm . (8)That this is possible is guaranteed by the fact that the ρ n are given by Eq.(4) [25]. The σ nm are matrices with thesame dimension as the ρ n , but we will not require anyfurther details about them. A second joint unitary opera-tion now performs feedback, applying a different unitarytransformation to S depending on the state of the con-troller (each state, | n i , of the controller is the equivalentof measurement result n in our previous analysis). Thisunitary has the form U fb = X n | n ih n | ⊗ U n , (9) where U n acts on the system. These unitaries performthe reordering of the eigenvalues of ρ n , and the change inthe system Hamiltonian (the energy levels) to bring thesystem into a thermal equilibrium state and adjust theaverage energy.The controller then performs the final part of the feed-back in which it expands S isothermally to extract thework. This cannot be described purely as a unitary op-eration, because it leaves the bath in a state of differententropy for each value of n . Because these different statesof the bath are necessarily macroscopically distinct (theyhave different entropy, and are therefore macroscopicallydistinguishable), this fully decoheres the controller in thebasis | n i . This can be described using a unitary of theform given in Eq.(9) that maximally entangles the con-trollers basis states, | n i , with an auxiliary system of thesame size, followed by tracing over the auxiliary system.Now, the result of the feedback operation on each state ρ n is to transform it to a final state ρ final n with entropy S ,temperature T and average energy E . Since the temper-ature and entropy of all the ρ final n are the same, they havethe same set of eigenvalues (the same distribution of pop-ulations). Since the average energy is also the same forall these states, they must also have the same set of en-ergy levels. Thus for every value of n (for each state | n i ofthe controller) the system has the same final Hamiltonianand the same final state, ρ T . Because of this the stateof the system and controller factor, and we can write thefinal joint state as ρ final C ⊗ ρ T . Because the system is atthermal equilibrium, the joint state of the system andbath also factors. However, the state of the controllerand the bath does not factor - this is because, in general,the bath transfers a different amount of entropy to thesystem for each value of n , and is thus left in a differentstate for each value of n (as discussed above). Since theprobability is p n that the state of the controller is | n i ,and since the different states of the bath are classicallydistinguishable, the final state of the three systems is ρ final = X n p n | n ih n | ⊗ ρ bath n ! ⊗ ρ T . (10)If we denote the initial entropy of the bath as S B , theentropy of each final bath state ρ bath n is S B − ( S − S n ).The total entropy of the final state is therefore S [ ρ final ] = S ( { p n } ) + X n p n S n + S B , (11)where S ( { p n } ) ≡ − P n p n ln p n is the entropy of the dis-tribution of measurement results. Since the total initialentropy of all three systems is S + S B , the total changein the entropy of the universe for the cycle is∆ S tot = S ( { p n } ) − ∆ S meas . (12)The second law then follows from Nielsen’s result [21],which states that for every measurement, S ( { p n } ) is anupper bound on ∆ S meas , and thus ∆ S tot ≥ | i . To do this it simply connects itselfto a fourth system with dimension N in a fixed state, | i , performs a (unitary) swap operation between itselfand this fourth system, and then dumps the fourth sys-tem into the thermal bath. This leaves the controller instate | i with zero entropy, and increases the entropy ofthe bath by S ( { p n } ). The feedback control cycle is nowcomplete: work ∆ W = T ∆ S meas has been extracted, thecontroller and system are back in their initial states, andthe entropy of the bath has increased by ∆ S tot .We note that the feedback control cycle is only thermo-dynamically efficient (preserves the entropy of the uni-verse on average) when S ( { p n } ) = ∆ S meas . This isonly true if the measurement operators P n commute with ρ T [21], so that the measurement is classical. This meansthat the feedback controller only preserves the entropy ofthe universe when it makes measurements of energy.We have so far only explicitly considered feedback con-trol with efficient measurements. An inefficient measure-ment is one in which the controller makes an efficientmeasurement, but throws away some information aboutthe measurement result [25]. All inefficient measure-ments can be described by the set of operators A nj , where P nj A † nj A nj = I , and as before n labels the measure-ment results. The final state of the system given result n is ρ n = P j A nj ρ T A † nj /p n , with p n = P j Tr[ A † nj A nj ρ T ].With these new definitions of ρ n and p n , the above analy-sis of the feedback cycle goes through unchanged, exceptthat S n is not necessarily less than S . In this case, theability of the controller to extract work from the sys-tem can be reduced by the measurement, rather than in-creased. Because of this, inefficient measurements canadd heat to a system, as well as doing work.Lastly, we note that we have performed all our analysiswith feedback from a “single-shot” measurement. Thisis usually referred to as “discrete” feedback control, todistinguish it from feedback control that uses continuousmeasurement [26]. However, the analysis we have pre-sented can be easily modified to derive the same resultfor continuous feedback control. All we have to do isobserve that each step in the feedback cycle can be per-formed infinitesimally. (A single infinitesimal time-stepof a continuous measurement is described by a measure-ment in which all the operators, A n , are infinitesimallyclose to the identity [26].). Our results above thus applyto all feedback control, whether discrete or continuous. [1] As an introduction to thermodynamics, the author rec-ommends the lecture notes by Sze Tan, available at < http://home.comcast.net/˜szemengtan/ > .[2] An introduction to quantum feedback control is givenin K. Jacobs and A. Shabani, Contemp. Phys. , 435(2008).[3] C. H. Bennett, Int. J. Theor. Phys. , 905 (1982).[4] W. H. Zurek, in Frontiers of Non-Equilibrium StatisticalPhysics , edited by G. T. More and M. O. Scully (PlenumPress, New York, 1984).[5] H. S. Leff and A. F. Rex, Am. J. Phys. , 994 (1994).[6] T. Ando, Linear Algebr. Appl. , 163 (1989).[7] V. Vedral, Proc. R. Soc. Lond. A , 969 (2000).[8] M. O. Scully, Phys. Rev. Lett. , 220601 (2001).[9] R. Alicki, M. Horodecki, P. Horodecki, and R. Horodecki,Open Sys. & Information Dyn. , 205 (2004).[10] T. D. Kieu, Phys. Rev. Lett. , 140403 (2004).[11] H. T. Quan, Y. D. Wang, Y. X. Liu, C. P. Sun, andF. Nori, Phys. Rev. Lett. , 180402 (2006).[12] T. Sagawa and M. Ueda, Phys. Rev. Lett. , 080403(2008).[13] R. Landauer, IBM Jl. Res. Develop. , 183 (1961). [14] B. Piechocinska, Phys. Rev. A , 062314 (2000).[15] K. Shizume, Phys. Rev. E , 3495 (1995).[16] K. Jacobs, Eprint: arXiv:quant-ph/0512105 (2005).[17] M. B. Plenio and V. Vitelli, Contemp. Phys. , 25(2001).[18] K. Jacobs, Phys. Rev. A , 044101 (2005).[19] S. Lloyd, Phys. Rev. A , 3374 (1997).[20] M. Ozawa, J. Math. Phys. , 759 (1986).[21] M. A. Nielsen, Phys. Rev. A , 022114 (2001).[22] C. A. Fuchs and K. Jacobs, Phys. Rev. A , 062305(2001).[23] A. C. Doherty, K. Jacobs, and G. Jungman, Phys. Rev.A , 062306 (2001).[24] H. M. Wiseman, Ph.D. diss., The University of Queens-land, Brisbane (1994).[25] M. A. Nielsen and I. L. Chuang, Quantum Computationand Quantum Information (Cambridge University Press,2000).[26] K. Jacobs and D. A. Steck, Contemp. Phys.47