John von Neumann's 1950s Change to Philosopher of Computation
JJohn von Neumann’s 1950s Change to Philosopher of Computation
Steven Meyer [email protected]
August 25, 2020
Abstract:
John von Neumann’s transformation from a logician of quantum mechanics (QM) in the 1920sto a natural philosopher of computation in the 1950s is discussed. The paper argues for revision of thehistorical image of Neumann to portray his change to an anti formalist philosopher of computation.Neumann abandoned Hilbert’s programme that knowledge could be expressed as logical predicates. Thechange is described by relating Neumann’s criticism of Carnap’s logicism and by discussing Neumann’srejection of the Turing Machine model of computation. Probably under the influence of the founders ofmodern physics in particular Wolfgang Pauli and Werner Heisenberg at the Advanced Study Institute,Neumann changed to a natural philosopher of computation. Neumann’s writings from his development ofthe now almost universal von Neumann computer architecture are discussed to show his 1950s view ofalgorithms as physicalized entities. The paper concludes by quoting Neumann’s statements criticizingmechanistic evolution and criticizing neural networks.
Keywords:
John von Neumann, Von Neumann computer architecture, Philosophy of computation, Hilbert’sprogramme, Turing Machine computation model, MRAM computation model, Rudolf Carnap, Antiformalism, Natural philosophy, Bohr’s complementarity, P?=NP problem.
1. Introduction
John von Neumann’s transformation from a logician of quantum mechanics (QM) in the1920s to a natural philosopher of computation in the 1950s is best expressed by a story Neumanntold about Wolfgang Pauli’s criticism of formal mathematics. "If a mathematical proof is whatmatters in physics, you would be a great physicist." (Thirring[2001], p. 5). Neumann’stransitioned from adherent of David Hilbert’s programme that all knowledge could beaxiomatized as predicate formulas in the 1920s to the first computer scientist is described. Thevon Neumann computer architecture (Aspray[1990]) anticipated current theoretical models ofcomputation. Neumann also criticized formal automata and machine learning.This paper argues that the wide spread popularity of computers and artificial intelligencehas resulted in suppression of Neumann history. I argue that there is a need for much moredetailed historical study of Neumann. This paper is a first step in the historical revision of the1950s Neumann. In my view, to often the historical image of Neumann only includes his preWW II work on formal quantum logic.Neumann’s transition to philosopher of computation became his dominant passion drivingchanges in his other philosophical positions. However, he remained an applied mathematicianpublishing earlier work even from the late 1940s and continued to work on problems from appliedmathematics. One example is Neumann’s lecture to the International Congress of Mathematics in1954 titled "Unsolved problems in Mathematics" (Neumann[1954]). He considered infinitaryproblems in Hilbert spaces while mentioning that they did not correspond to physical reality (p.241). Other contributions of the later Neumann such as his contribution to computer patents isnot discussed because it requires legal expertise. Neumann’s contribution to econometrics is notdiscussed. 2 -
2. Background
Kurt Godel’s incompleteness results in the late 1920s (Godel[1931]) and discussions withphysicists in the 1930s were some factors motivating Neumann’s change. Neumann’s transition isbest expressed by his strong criticism of Carnap’s predicate formula based conception ofinformation (Kohler[2001]). The paper discusses Neumann’s criticism of Carnap and argues thatKohler’s explanation of the reasons for the rejection of Hilbert’s programme is incorrect becausein the 1950s while working with physicists at the Advanced Study Institute, Neumann rejectedpredicate formula based knowledge. The von Neumann computer architecture explicitlyimproves the Turing Machine model (TM).
3. Neumann Study of Natural Philosophy
Natural philosophy studies physical reality. Before the development of modern physics,there was no need for natural philosophy because Newtonian physics described a fixed and causalworld in terms of Newton’s laws. Max Planck and Albert Einstein called themselves naturalphilosophers because their study of physics involved studying philosophical concepts andscientific methods. It was clear in the late 19th century that Newtonian physics was inadequate insome areas. The new modern physics redefined philosophical concepts such as causality andsimultaneity and modified empirical studies to include methodological study of experiments(Heisenberg[1958] Chapter 6 for a more detailed description of this change).The philosophical changes in physics remain the subject of debate. One important debaterelated to computation involves search for one unified physical theory that was probably due toEinstein. Heisenberg provided a modern explanation of unified physical theories unified bymathematical groups (Heisenberg[1958], pp. 105-107). In contrast, David Bohm held the viewview that nature is an infinity of different qualitative realities (Bohm[1957]).John von Neumann undertook the study of natural philosophy as part of his development ofthe modern digital computer starting in the early 1940s. Neumann was influenced by thefounders of modern physics and physicists were influenced by Neumann in their concepts andmethods used in calculating field properties. Neumann’s natural philosophy of computation isusually studied in the computer science (CS) area as operations research. In contrast toNeumann’s view, the other CS research programme is based on mathematical logic oftenexpressed as the Church Turing Thesis. Logic based CS is now predominant.
4. Neumann’s Criticism of Carnap’s Conception of Information Shows Change
Eckehart Kohler in his excellent and detailed paper "Why von Neumann rejected Carnap’sDualism of Information Concepts" (Kohler[2001]) argues that Neumann’s (and Pauli’s) veryharsh criticism of Carnap was incorrect because Neumann wrongly assumed information only hasphysical meaning. Both Neumann and Pauli were so sure of their criticism that theyrecommended Carnap not publish his study of information. Neumann’s criticism shows hischanged views concerning logic and empiricism.Neumann’s opposition to Carnap’s argument that information has a formal logicalcomponent as well as an experimental physical component provides the clearest explication of hischanged philosophy. Kohler’s mistake is that starting after world II physicists and appliedmathematicians including Neumann rejected the very idea that formal sentence based logic candescribe reality. Their criticism was not that Godel’s results (Godel[1931]) made formula basedpropositional logic no longer absolute, but that logic failed as a method for describing the world.By the early 1950s Neumann viewed the world as empirical for which growth of 3 -knowledge required experiments. I interpret Kohler and Carnap as arguing that Hilbert’sprogramme could still work in conjunction with the dual idea that information has both a logicalelement and an empirical element (p. 117). Both Carnap and Kohler are attempting to saveHilbert’s programme that they problem shift to be interpreted semantically. Kohler characterizedCarnap’s logical component of information as ’logical and mathematical sentences.’ (p. 98) Thisis why Neumann attacked Carnap’s dualism.Neumann’s rejection of his 1920s belief in Hilbert’s programme can be seen from theadvice Neumann gav e Claude Shannon to use the term entropy for one of the functions involvedin Shannon’s definition of information in spite of Shannon’s misgivings (p. 105). The advice wasgiven around 1949 before Neumann’s anti formalist views were fully developed, but showsNeumann’s shift to empiricism. Shannon avoided the issue of the meaning of information byexplaining it as simply a type of mathematical coding function.The background change to anti-formalism did not just occur at the Institute of AdvancedStudy in Princeton, but was common among physicists and applied mathematicians after WWII.Formal propositional logic was viewed as problematic not only because systems of formalsentences could be used to mechanically derive inconsistent results (Godel1931], see also PaulFinsler’s earlier results that were not tied to Russell’s propositional calculus. Finsler[1996],Breger[1992]), but also because it was believed that knowledge not derivable from formalsystems (from axioms plus formal propositions) exists. One obvious example that can not beunderstood in terms of formal mathematics is instantaneous wav e function collapse in quantummechanics.After Neumann’s death in the 1960s, empirical alternatives to logicism were developed.Finsler proved that the continuum hypothesis is true by defining a continuum that wasaxiomatically defined, but different from the standard continuum definition (Finsler[1968]).Einstein expressed earlier the viewpoint of the founders of modern physics in his 1921 lecture ongeometry and experience (Einstein[1921]). Einstein argued that formal mathematics(propositional logic based rationality) was incomplete in a physical sense. However, Einsteinexpressed a contrary view in other writing. George Polya who was Neumann’s teacherencouraged Imre Lakatos to solidify the ideas expressed by Neumann and Wolfgang Pauli thatpossibly actually originated with Polya. The idea in the 1960s was called quasi-empiricalmathematics. Polya encouraged Lakatos to rewrite his thesis
Proofs and Refutations. as a simplerbook because it would solidify empirical mathematics in place of logicism (Polya[1975]).
5. Neumann Conception of Computation Closest to MRAM Model
Starting in the early 1940s, Neumann became focused on the possibilities of digitalcomputers (Aspray[1990] for a detailed history). There is currently no evidence of Neumann’sexplicit discussion of his philosophy of computation. However, it is known that Neumanndiscussed his thinking about abstract properties a computer architecture should have anddiscussed problems with neural networks as models for computation. Neumann rejected that TMmodel because it used logics rather than physical properties (Kohler[2001], p. 104). Kohlerexplains Neumann’s view of logic (algorithms) as physicalized entities (p. 116). Neumann’sexplicit rejection of formal neural networks also sheds light on his computational philosophy(Aspray[1990], note 94, p. 321).In the early 1950s, the possibility was considered that computational errors in constructedcomputers were physically inherent conceptually similar to entropy. Since Neumann worked withWolfgang Pauli and Werner Heisenberg who developed the Bohr interpretation of QM, the earlyview of errors may have expressed their view of Bohr’s complementarity. It turned out the errors 4 -were circuit design mistakes and environment background caused errors that were repairableusing error correcting codes.However, the problem of inherent computational errors has recently re-emerged as errors inquantum computers. The errors (complementarity between classical macro physics and quantummicro physics) may actually represent new physical reality that can be measured with the newmethods of cold atom physics (Monroe[2018]). It is possible that Neumann working with Pauliand Heisenberg anticipated quantum computing. I think Arthur Fine’s characterization ofcomplementarity as only a conceptual device is incorrect (Fine[1996], pp. 20, 21, 124).This paper provides an analysis of Neumann’s thinking by discussing a formalization of theNeumann computer architecture using a modern abstract model called MRAMs (random accessmachines with unbounded cell size and unit time multiply) first studied by Hartmanis and Simon(Hartmanis[1974][1974a]). MRAM’s are the closest studied model to the Neumann computerarchitecture. Neumann explicitly listed the MRAM model properties in his list of requirementsfor his now almost universally used von Neumann computer architecture.In contrast to TMs that have an unbounded number of bounded size unary representingmemory cells, Neumann assumed that a computer would have a finite number of binary encodedunbounded size memory cells (computers need to be built large enough for the given problem).Neumann also argued that some sort of intuition needed to be built into programs instead of bruteforce searching (Aspray[1990], p. 62). Unlike TMs, Neumann’s computer design provides bitselects and indexing.
6. Consequences of Lack of Study of Neumann’s 1950s Change
There are modern consequences of Neumann’s philosophy and indirectly lack of study ofthe 1950s Neumann. In the MRAM model, the P?=NP problem does not exist (or the answer isthat there is no difference between the class of problems solvable by non deterministic guessingversus the class solvable by deterministic searching) (Meyer[2016]). Many modern philosophicalquestions assume implicitly that non deterministic TMs are more powerful than deterministicmachines. For example, Shor’s quantum computation algorithms assume P!=NP (Shor[1996]).
7. Examples of Neumann’s 1950s Thinking
An interesting story related to Neumann’s philosophy is that advocates of the importance ofthe P?=NP problem (does guessing speed up computation by more than a polynomial bound)found a letter in the Godel Archive to Neumann that they interpret as Godel supporting theimportance of the P?=NP problem. However, in a letter (unfortunately undated) from Neumannto Oswald Veblen relating to an Institute of Advanced Study permanent appointment for Godel,Neumann shows skepticism toward Godel’s later work (Hartmanis[1989] for the Godel P?=NPletter. Neumann[2005], p. 276 for the Neumann letter).To me the most important consequence of Neumann’s transformation is that he correctlyunderstood that TMs are very weak (slow) computing machines. It is true that TMs are universalin the Church Turing sense (Copeland[2015]). Any TM or MRAM can calculate anythingrecursively computable in the Church Turing sense (for background the question ’are two regularexpressions equivalent’ is outside NP but computable). The thesis is often expressed as any TMcan simulate any computing device. The problem is that Neumann understood the importance ofcomputational efficiency so TM’s lack of computational power is a negative factor. TMs areslower than von Neumann computers because the von Neumann architecture’s indexed datastructures obviate the need for non deterministic TM guessing.Neumann also criticized other models of computation such as primitive automata and 5 -neural networks. The criticism is based on rejection of the relevance of predicate logic formulasin describing reality. It is not clear if Neumann rejected the Church Turing thesis. He seems notto have discussed it. The paper explains this skepticism by analyzing Neumann’s dislike ofCarnap’s philosophy. Here are two specific examples of Neumann’s criticism of automata.
He (Neumann) led the biologist to the window of his study and said: ’Can you see the beautiful whitevilla over there on the hill? It arose by pure chance. It took millions of years for the hill to beformed; trees grew, decayed and grew again, then the wind covered the top of the hill with sand,stones were probably deposited on it by a volcanic process, and accident decreed that they shouldcome to lie on top of one another. And so it went on. I know, of course, that accidental processesthrough the eons generally produce quite different results. But on this one occasion they led to theappearance of this country house, and people moved in and live there at this very moment.’ (storytold in Heisenberg[1971] p. 111)
The insight that a formal neuron network can do anything which you can describe in words is a veryimportant insight and simplifies matters enormously at low complication levels. It is by no meanscertain that it is a simplification on high complication levels. It is perfectly possible that on highcomplication levels the value of the theorem is in the rev erse direction, namely, that you can expresslogics in terms of these efforts and the converse may not be true. (Neumann quoted in Aspray[1990],note 94, p. 321)
8. References
Aspray[1990] Aspray, W.
John von Neumann and The Origins of Modern Computing.
MIT Press, 1990.Breger[1992] Breger, H. A Restoration that failed: Paul Finsler’s theory of sets. InGillies, D. ed.
Revolutions in Mathematics.
Oxford, 1992, 249-264.Bohm[1957] Bohm, D.
Causality and Chance in Modern Physics.
Van Nostrand, 1957.Copeland[2015] Copeland, J. The Church-Turing thesis.
The Stanford Encyclopedia ofPhilosophy (Summer 2015 Edition), http://plato.stanford.edu/archives/sum2015/entries/church-turing
Einstein[1921] Einstein, A. Geometry and Experience.
Lecture before Prussian Academyof Sciences.
Berlin, January 27, 1921, URL March 2020:
Fine[1996] Fine, A.
The Shaky Game Einstein Realism and the Quantum Theory.
Chicago Press, 1996.Finsler[1969] Finsler, P., 1969. Ueber die Unabhaengigkeit der Continuumshypothese.
Dialectica 23 , 67-78.Finsler[1996] Finsler, P. (Booth, D. and Ziegler, R. eds.)
Finsler set theory: Platonismand Circularity.
Birkhauser, 1996.Godel[1931] Godel, K. On Formally Undecidable Propositions of the PrincipiaMathematica and Related Systems. I. in Davis, M. (ed.)
The Undecidable:Basic Papers on Undecidable Propositions, Unsolvable Problems andComputable Functions.
Dover, 5-38, 1965.Hartmanis[1974] Hartmanis, J. and Simon, J. On the Structure of Feasible Computations.
Lecture Notes in Computer Science, Vol. 26.
Also Cornell EcommonsURL March 2020: https://ecommons.cornell.edu/handle/1813/6050, 1974,1-49.Hartmanis[1974a] Hartmanis, J. and Simon, J. The Power of Multiplication In RandomAccess Machines.
IEEE, Oct. 1974, 13-23.Hartmanis[1989] Hartmanis, J. Godel, von Neumann and the P=?NP Problem.
SIGACTStructural Complexity Column.
April 1989. Cornell Ecommons URLMarch 2020: eCommons - Cornell digital repositoryhttps://ecommons.cornell.edu/handle/1813/6910Heisenberg[1958] Heisenberg, W.
Physics and Philosophy - The Revolution in ModernScience.
Prometheus, 1958.Heisenberg[1971] Heisenberg, W.
Physics and Beyond - Encounters and Conversations.
Harper, 1971.Kohler[2001] Kohler, E. Why von Neumann Rejected Carnap’s Dualism of InformationConcepts. In Redei, M. and Stoltzner, M. (eds.)
John von Neumann andthe Foundations of Quantum Physics.
Vienna Circle Institute Yearbook 8,Kluwer, 2001, 97-134.Lakatos[1976] Lakatos, I.
Proofs and Refutations: The Logic of Mathematical Discovery.
John von Neumann and the Foundations ofQuantum Physics.
Vienna Circle Institute Yearbook 8, Kluwer, 2001,231-245.Neumann[2005] Von Neumann, J. (Redei, M. ed.)
John Von Neumann: Selected Letters.History of Mathematics Series, Vol. 27,
American Mathematical Society,2005.Polya[1976] Polya, G. John Watkins Archive call number WATKINS file 514.Correspondence between George Polya and John Watkins on Lakatosphilosophy of mathematics, London School of Economics, archive date2010.Shor[1996] Shor, P. Polynomial-Time Algorithms for Prime Factorization and DiscreteLogarithms on a Quantum Computer. arXiv:quant-ph/9508027v2[quant-ph], 1996.Thirring[2001] Thirring, W. J. v. Neumann’s Influence in Mathematical Physics. InRedei, M. and Stoltzner, M. (eds.)