Don't choose theories: Normative inductive reasoning and the status of physical theories
aa r X i v : . [ phy s i c s . h i s t - ph ] J a n Don’t choose theories: Normative inductivereasoning and the status of physical theories
Andr´e C. R. MartinsNISC – EACH – Universidade de S˜ao Paulo,Rua Arlindo B´etio, 1000, 03828–000, S˜ao Paulo, BrazilJanuary 3, 2017
Abstract
Evaluating theories in physics used to be easy. Our theories providedvery distinct predictions. Experimental accuracy was so small that worry-ing about epistemological problems was not necessary. That is no longerthe case. The underdeterminacy problem between string theory and thestandard model for current possible experimental energies is one example.We need modern inductive methods for this problem, Bayesian methodsor the equivalent Solomonoff induction. To illustrate the proper way towork with induction problems I will use the concepts of Solomoff induc-tion to study the status of string theory. Previous attempts have focusedon the Bayesian solution. And they run into the question of why stringtheory is widely accepted with no data backing it. Logically unsupportedadditions to the Bayesian method were proposed. I will show here that,by studying the problem from the point of view of the Solomonoff induc-tion those additions can be understood much better. They are not waysto update probabilities. Instead, they are considerations about the priorsas well as heuristics to attempt to deal with our finite resources. For thegeneral problem, Solomonoff induction also makes it clear that there isno demarcation problem. Every possible idea can be part of a properscientific theory. It is just the case that data makes some ideas extremelyimprobable. Theories where that does not happen must not be discarded.Rejecting ideas is just wrong.Keywords: Theory choice, String theory, Bayesian methods, Solomonoffinduction
Describing physics as a successful history among sciences is too much of a com-mon place. But it also carries some truth to it. Even more than fifty years ago,Wigner [1] was already perplexed by the amazing agreement between theoreti-cal predictions and experiments. To the point he considered our mathematical1uccess as unreasonable. Perhaps even more surprising than the number of dig-its in the agreement between experiment and theory, we have had cases wherewe have discovered unexpected objects and entities. The existence of Neptunewas predicted because the calculated orbit of Uranus did not match its observedbehavior. Dirac predicted anti-particles as a second solution to his equationsbefore they were observed. Physicists love those stories. We all know them well,as great examples of the power of our calculations. Cautionary tales, such asthe belief that light needed an aether to propagate, are often told as steps to abetter theory and the lesson in caution, lost.And it is not in the precision of our theories that physicists can claim success.In most cases, our theories have provided us with predictions that, given the ac-curacy of our experiments, were also very discriminating. Newtonian Mechanicsand General relativity provide almost identical predictions for many systems.But there are plenty of cases where the expected behavior is very different. Andwe can make experiments that show General Relativity is a better theory. Thiscan be said with certainty or, more exactly, we can be so close to certainty thatthe difference is pure technicality. Chances smaller than 1 in 10 can be calledzero, without much abuse of language, after all. We have had success both inobtaining very precise theories, as well as very discriminating ones. Under thesecircumstances, epistemological considerations were not necessary. Everythingworks fine if you know epistemology well. But things also work fine if you knownothing about epistemological problems. We have been living in a epistemicallysafe world.But that is not always the case. As we look at the status of our currenttheories and how the community works with them, it is not hard to notice thatthere is something odd going on. We have never-ending discussions about thestatus of string theory. Those discussion even include the question of whetherit is an acceptable theory [2]. We currently can not make observations thatwould provide evidence that could allow us to choose between string theory andthe standard model. Based on that, some physicists claim string theory is notscientific. And science, we were taught, must be testable. And it must provideone single answer. Or must it not?Inductive reasoning, such as Bayesian methods, can help us answer that ques-tion [3]. Indeed, recent efforts to bring some modern, non dated epistemologyto the question [4] have point out that the underdetermination problem is realand something we might have to learn to accept. Underdetermination happenswhen the predictions of two different models are identical. As a consequence,the probability ratio between the two models can not be altered by observingany data. And the initial, often subjective prior guesses remain. Under thesecircumstances, attempts to determine how to choose between underdeterminedtheories have caused deep disagreements between supporters and critics of StringTheory [5]. In particular, the idea that a theory could be “confirmed” purely bytheoretical arguments [6] is a claim met with strong resistance. There is goodreason for that. Bayesian methods can be obtained from logical considerations.And they have way to change probabilities from theoretical considerations.In this paper, I will show why inductive methods are a necessity and which2onclusions we can draw from them. I will explain why human cognition makesnormative methods to choose theories a real necessity [7]. There are too manytraps in our brains, we just can’t trust our feelings to pick the best ideas. I willbriefly explain that crisis in other areas, including the use of statistical tools,can also be traced to harmful interactions between our natural reasoning andour current tools [7]. We must use inductive methods. And we must understandthem well and reason as close as possible to their demands.Bayesian methods have some important consequences. Theories can nolonger be discarded in the majority of cases. They can and must be rankedprobabilistically. To further understand the consequences of inductive methods,I will see what else they can tell us about the status of string theory. I will showthat both sides of the quarrel have captured important aspects of the problem.Instead of using Bayesian induction to the ranking of theories, I will use anequivalent framework known as Solomonoff induction [8]. This will provide usa new way to look at the theoretical considerations issue. While we must keepall underdetermined theories as possibilities, we will see that the theoreticalmethods of “confirmation” do not provide ways to update probabilities. Butthey can carry important information to determine a priori assessments as wellas carry some heuristical value for limited beings.
When physicists try to predict the path of a newly found asteroid, they do notjust look at the data and provide an educated guess. We all know better than totrust our brains when our equations do a much better job at it. And there aregood reasons for that. Our brains did not evolve to track nor predict gravita-tional trajectories. And they also did not evolve to choose between competingtheories or even ideas. Quite the opposite. Recent evidence shows that ourreasoning skills might not have evolved to help us look for truth. Instead, theymight exist to help us win arguments, regardless of truth [9]. When thinkingabout identity-defining ideas, we use our analytical skills to defend our positions[10]. And the better our skills are, the better we are at reasoning to find waysto defend ideas. Even when they are wrong.We also have a tendency to be overconfident about our reasoning abilities[11]. But our confidence also plays strange tricks on us. As we get more dataabout an issue, we reach a point where our accuracy in making predictions usingthat data stops improving. At the same time, we keep getting more and moreconfident [12], despite no real improvement. Unfortunately, overconfidence is notlimited to the issues we are not specialists. Quite the opposite. It is often thecase that experts tend to consider themselves far better than they actually are[13]. We can be trained to know how good our estimates are, but that involvesa lot of feedback about when we get things right. It usually also requires bettermodels to support our estimates. Such an improvement in self calibration has3appened, for example, in meteorological prediction [14].In general, actuarial judgments, where simple statistical models replace thehuman decision process, often provide a significant improvement to rate of suc-cessful evaluations [15]. Our illusions about our own skills extend even to thepoint where we consider that random events, like the throw of a die, are morelikely to benefit us if we make the throws ourselves [16]. We believe we have farmore control over the world and our own reasoning than we actually do.The examples I just presented form a very small sample of a huge literature inhow our natural reasoning fails. Our reasoning fails and, still, we trust our brainsand our own evaluations when we should not. This means that, when we have abelief, we tend to be too confident about it. And, since argumentation might bemore about convincing than getting the best answer, we are expected to defendour beliefs even when evidence is against us. Add to this the fact that logic andmathematics can prove nothing without initial axioms. No belief about the realworld can be proven true. Deduction requires previous knowledge and inductivemethods, while immensely helpful, do not provide certainty. Combining thosefacts, we can see that, when we believe, we harm our ability to look for truth(or best answers). We should actually hold no beliefs and, wherever possible,replace our own analysis for mathematical models [17]. That is exactly whatwe need to undestand the arguments in the String Theory debate better.
An outdated but still influential attempt on using logic to the problem of the-ory choice was the Popperian notion of falseability [18]. Popper wanted to solvethe demarcation problem, to find a way to divide theories in scientific and non-scientific ones. He proposed that scientific theories must include the possibilitythat they could be proved false by an experiment. Unfortunately, from a de-ductive point of view, no theory can be proved true or false. If an experimentfails to produce a predicted outcome, we know something went wrong. Onepossibility is, indeed, that the theory is false. But it could also be a problem inthe experiment itself. Or the calculations that made the prediction. But thereare other fundamental problems as well. When we make any calculations, weassume not only the theory to be tested. We also have to use many other auxil-iary hypothesis about the world [19, 20]. When Newtonian Mechanics failed topredict the correct orbit of Uranus, the theory was not the one to blame. Theactual error was the incomplete model of the Solar System. The initial calcula-tions did not include the existence of Neptune. Current attempts to reconcilethe movement of galaxies with their observed mass by adding dark matter termsdo the same thing. Dark matter terms are proposed as an auxiliary hypothesis.Without them, the predictions for the behavior of galaxies would just be wrong.Maybe dark matter exists, maybe the theory is wrong. Experiments alwaysleave something underdetermined. As a matter of fact, one could go on adding4ew hypothesis as needed to rescue basically any model. The new theory mightbecome a convoluted model, but we might find ways to allow any theory to berescued and still fit observation. The Ptolemaic description of the Solar Systemdid something like that.Popper acknowledged that there is no deductive solid way to say a theory iscorrect. But there is also no way to determine a theory is wrong. We are leftwith induction for this task and the uncertainties that come with it. Luckily,while inductive methods can not tell us whether an idea is tue or not, theymay still provide estimates of how plausible an idea is [21]. And it turns outthat we can obtain clear rules for making inductions. Based on a few simpleand reasonable assumptions, it can be shown that assigning plausibilities canbe done by following the rules of probability and changing our estimates whenpresented with new data by using Bayes formula [3]. Induction, from a logicalpoint of view, is the same as probability and Bayesian statistics.This realization leads us to interesting consequences. And some of thoseconsequences are well known but poorly understood. For example, we mightbe capable of making incredibly accurate guesses, but we can not really know.Some descriptions of the world will become extremely improbable, but that isnot the same as false. Ideas can not be really rejected by induction [7]. Despitethe deceptive name of some statistical tools, we only at best say some ideasare improbable. Indeed, the current replication crisis in areas as medicine andpsychology have an important component of publication bias [22], but it is alsocaused by the widespread use of p -values [23, 24]. Statistics should indeed changeits misleading terminology and stop talking about rejection [7]. What we cando, instead, is to rank ideas from more probable ideas to extremely improbableones.A second consequence of using Bayesian methods is not so clear at first.Bayesian methods allow one to compare how the probability ratio of two theoriesplus their full set of auxiliary hypothesis changes based on data. While thereare questions about prior choice, the method is straightforward. But that onlyapplies to the very specific question of which set of theory plus hypothesis is morelikely. It says nothing about the probability ratio of the theories themselves.And the final answer, while in probability form, is not really a probability. Thatis because all other possibilities, necessary to the full assessment, were excluded.In principle, to really obtain probabilities, Bayesian methods require that weuse every possible theory and their variations [25]. It actually requires a kindof theoretical omniscience [26] where all theories are known beforehand. Thesimplicity of Bayes formula is misleading. Using it correctly involves far morethan the easy formula suggests.And this misleading simplicity makes reasoning based on Bayesian methodsharder than it should be. Instead of following this path, I will base my analysishere on an inductive method that is equivalent to Bayesian methods, that is,Solomonoff induction [8]. In it, theories are replaced by algorithms and proba-bilities become sums over successful algorithms. But the methods produce thesame results. The only difference is that, while choosing Bayesian initial prob-abilities, the priors, is not a simple task and still a matter of a lot of research,5olomonoff induction has no open problems. It comes with what can be seen aspriors already chosen by the method itself. While a Bayesian analysis of theory acceptance can teach us a lot, a few impor-tant details are not easy to notice in the Bayesian framework. It can be usefulto use a different but equivalent induction framework, Solomonoff induction [8].Solomonoff induction basically provides the recipe for an algorithmic generalmethod for induction that is equivalent to Bayes rule with a built-in choice ofprior distributions [27, 28]. It is also impossible to implement as it requires infi-nite computational resources. As a matter of fact, the same is true for Bayesianmethods. The infinities are just hidden in the Bayesian case by the fact we canestimate odds ratios of two cases. And we can also calculate probabilities if weassume a very limited number of scenarios are all possible theories. But thatassumption, while useful, is wrong. The ease that one can obtain incompleteand wrong analysis using Bayesian techniques is behind some of the criticismBayesian methods have encountered.Issues such as the problem of the old evidence [29] would not be real if wewere not limited beings. The old evidence problem is a doubt about how to useold data when a scientist creates a new theory. And the only way to solve itis to remake all calculations you had previously done without the new theorynow including it. This is usually not possible to do, but it is the correct way.After all, the order one creates theories should not matter, we must get the sameresult we would get if all theories were indeed known from the start . Solomonoffinduction requires in a much clearer way that all theories and ideas should beknown beforehand.What we get is that complete solutions to the problem of induction are in-computable [30]. We can obtain estimates from partially applying each method.But there is no way to estimate the error we might be making by disregardingparts of the complete calculation. There is no known way to assign an error toeither the probabilities we get from partial Bayesian methods nor the predictionswe get from partial Solomonoff induction.So, how does Solomonoff induction works? In simplified terms, assume wehave a string X that represents what we know about the world (or a givenproblem). For each program size m , we must generate all programs that havethis size. Some of these programs will have as output the string of data X ,followed by what we call the predictions of that program. If the output of agiven program does not give us X , ignore that program. If it does give us X ,keep its prediction about the “future” observations. Each prediction we obtainfrom this Solomonoff machine is the weighted average of the predictions of everyprogram that generates X . The weights are a function of program size m , givenby 2 − m . Repeat the procedure for every possible size m and you have solvedthe problem of induction.This recipe is both perfect and impossible to follow. Given all the practicalimpossibilities involved in applying his set of rules, Solomonoff inductions is6asically a gold standard. It tells us how we should do induction and it servesto enlighten us about the path to improve our estimates as well as point outwhat we might be doing wrong. Given our natural reasoning shortcomings, theexistence of a standard, despite impossible, can be a useful tool. It can help usunderstand better how we should evaluate and compare theories.In this case, of course, some effort to translate concepts is required. Solomonoffalgorithms do not correspond to a theory. And they are not even equivalent to atheory and all the auxiliary hypothesis required to perform calculations. Whenwe make a computational implementation of any theory, we sometimes havea theory that has probabilistic components. In this case, by simply changingthe seed of the random number generator or the generator we can get differ-ent strings. But different random generator seeds still correspond to the sametheory and auxiliary hypothesis. That is true even for completely deterministictheories. That happens because all measurement is subject to errors. If wetry to simulate the experiment that generated string X , we must estimate theuncertainty of the measurement process. And, just as with random numbergeneration, other calculations that we implement tend to have more than oneway to be implemented. The case of a single theory and all needed auxiliaryhypothesis we use to get a prediction still corresponds to a plethora of differentalgorithms.This huge set is actually needed, since, most of the time, the random gener-ator will not provide draws that match the exact observed X . That will happeneven if the theory was actually right. Here, precision will also play an importantrole. If the theory provides a very narrow distribution around the values actu-ally in X , it is possible that several different realizations will return X . Widedistributions or distributions that are not centered around X , on the other hand,will more rarely turn our observed string, if ever. As a result, the theories thatare indeed better will have more programs associated with them. Their predic-tions will count more in the final average. Certainty is not achieved. But ourpredictions might be based with far more weight associated to one theory thanits competitors.It is worth noticing that, while a theory will correspond to many differentprograms, there is no reason to expect that every program could be translatedinto a “reasonable” theory. And some algorithms might, in principle, correspondto more than one theory. If we just implement the final equations of two the-ories that predict the exact same behavior, a single algorithm may correspondto two or more theories. However, that does not mean that underdeterminedtheories will have the same set of algorithms. Implementation of the final dy-namical equations might be identical, but there will be implementations wherethe consequences are calculated by the algorithm from the basic axioms. Andthose will be different.In the end, this whole process is equivalent to making a prediction usingBayesian methods. Larger weights translate to larger posterior probabilities.And the predictions come from a weighted average of the theories.7 .2 Pursue every theory One thing should have become clear now. By averaging over every algorithmthat produces the observed X , we are actually including each and every theorythat is compatible with the observations. Even theories that are a little off willgenerate X from algorithms with a specific choice of seed for a potent randomgenerator. More serious disagreement can cause this to be so rare that theactual effect on the final average is close to null. But we still keep and inspectevery possibility.Some theories will survive in many different algorithms. Other theories willbecome quite rare. In that sense, Solomonoff induction is compatible with anassessment that theory A is a much better description than theory B , muchmore probable. Those are equivalent ways to say the same thing. But both A and B survive. That means that our standard for correct induction uses thewhole space of possible theories. Therefore we must study the set of all possibletheories as completely as we can. Once we do that, we will have theories thatare far more probable given our data. And some that are so improbable that,aside a small technical abuse of language, we can call them false. It is likelywe would end this process with more than one surviving theory that has areasonable probability associated to it. But, since we are limited, we must startour analysis with a small set of theories. To correct that, Solomonoff inductionsuggest that we should continuously look for other theories that are compatiblewith our data. And keep each and everyone of them that are already compatible,for as long as they remain so. An important consequence of a complete induction standard should be clear bynow. If a new theory provides the same predictions as an older one, none ofthem can be discarded, if those predictions are compatible with the experiments.Instead, underdetermination means we will have to live with both versions.Excluding one has no solid justification. Despite our best hopes, we mightnever have certainty about the one theory of the universe. And it is perfectlyreasonable to be forced to live with alternatives, if all alternatives describe ourobservations well. To desire one theory is a characteristic of our fallible minds.We want one true to believe and defend, so that we can stop thinking. Butsolid reasoning offers no such easy answers. The quest for unified theories can,of course, proceed. But we must understand that the final answer might be oneunified theory as well as it might turn out to be many theories.Once we accept that, we can proceed. And there are still questions to answer.String Theory survives as a possibility we can not disregard, not without exper-iments that are, for now, impossible. Penrose metaphor of a scientist walkingaround an empty city and trying to find the one real beautiful place by lookingfor signs of aesthetically pleasing neighborhoods [31] still provides interestingbut probably unplanned illustrations of the problem. In the original version,8here is only one building that we want to reach. In reality, there might bemany places in the city that are worth visiting. Even if one is better than allthe others, the only way to determine that is visiting them all and comparing.One question that remains is how much attention String Theory deserves.Or, in Bayesian terms, how probable it is that String Theory is the best alterna-tive we have? An important argument that defends that it should be consideredat least probable is based on the theoretical characteristics of the theory [6, 4].But Bayesian methods only allow for probability change when we observe newdata. Theoretical characteristics are not new data. And yet, the idea that atheory might be made more or less probable not only by data but also because ofthose theoretical characteristics has defenders. It would be easy to identify sucha defense with our desire to win arguments rather than seek for the truth. Butit is worth asking if there might not be something hidden in those arguments.Taken at face value, the concept that anything other than new data canalter our evaluation is normatively wrong. While true, this statement is aboutthe idealized version of the reasoning process. Limited humans might neglectedto evaluate some relevant aspect of the problem. If later, we discover we havemissed something, it might make sense to correct our estimates to account forthe neglected facts.Indeed, some of the theoretical considerations proposed by Dawid [6] can beunderstood as such a correction, while at least one of the arguments is wrong.The theoretical consideration that there is no other choice is false. First ofall, there might be unconceived alternatives [33] we have yet to find. And wealready have other proposals for doing quantum gravity. Loop quantum gravityis one example[32]. More than that, if one looks only at the string of all data X we have today, quantum field theory and general relativity seem to describeit all well, despite their incompatibilities. An algorithm where one of the twotheories is used depending on the problem with a sensible rule to choose is,indeed, compatible with current X . While we might feel that is inelegant andwrong, we should know better than to use any feelings we might have. Normativerules exist exactly because our feelings often lead us astray. Of course, if therewas only one algorithm that generated X and we had access to every possiblealgorithm, we could say we had no choice. Realistically, if we knew of only onealgorithm that generated X , we could still say that is all we have for now. Butwe are not there.The second theoretical argument used to defend String Theory is internalcoherence. This argument goes like this. While String Theory was planned asa way to circumvent problems with infinities in quantum field theories, it hasachieved much more. Gravity can be obtained as a consequence. String Theorycan also help us understand better the concept of supersymmetry as well as blackholes entropies. Indeed, as string theory seems capable of explaining more andmore about the structure and laws of the Universe, it does start to sound morelike a solid theory. But our normative rules don’t include the possibility ofchanging probabilities in this case.It is here that Solomonoff induction can provide a good illustration aboutwhat is going on. Matching the string X with a given algorithm is a simple yes or9o verification. The output either matches and the predictions of the algorithmare recorded with the proper weight, or the output does not match and thealgorithm is discarded. The process of obtaining new data means adding to thestring X . When we do that, more algorithms stop matching and are discarded,increasing the importance of those that remain. No considerations about thecharacteristics of the algorithms are relevant at this point. Not when more datais obtained.But that is not true about the whole process. Indeed, one central part of theprocess is to assign weights to different algorithms. The longer they are, the lessweight they have. One short algorithm may have a much larger weight or, inBayesian terms, a larger prior probability. When we are debating theories aboutparticles, forces, and gravity, our string X includes all we know about each ofthose issues. If we need different theories to address each case and rules aboutwhen to use which one, that makes for a larger algorithm. On the other hand,if we can get X from a small set of principles, this new set of principles cancorrespond to a smaller algorithm. A smaller algorithm means a larger weightin predictions. And that corresponds to a larger prior probability.From a normative point of view, this is no change in probability. In thatsense, the criticism that we must not use theoretical considerations to updateprobabilities is technically correct. But we should have already included thetheoretical considerations beforehand. Theoretical considerations should haveinfluenced the prior distributions. Only then, we should move on and startlearning from an expanding X , actually doing what we call updating probabili-ties. Unfortunately, as finite beings, this is not possible. We are always creatingnew theories. When that happens, we must repeat all the evaluation as if wehad always known the theory from the beginning. This might not be feasible.Bayesian epistemologists call this issue the problem of the old evidence. It isquite clear what we should do in principle. It is not so clear how to actually solvethe problem without the daunting task of processing every previous observationagain. One practical suggestion might be that, as we remake our estimates,theoretical considerations can and should be taken into account. And, insteadof entering in the priors, as theoretical considerations should, we may have toaccount for them later. As such, they will have an impact in the posterior es-timates. That does not happen because theoretical characteristics can be usedto update probabilities. It happens because we were unable to include thosecharacteristics when we should have done so.The third argument used to support string theory is that its developmentreflects the successful development of the standard model. Both theories startedfrom an attempt to solve technical problems with the previous descriptions. Andit is clear that the standard model was quite successful at it, as its predictionshave been confirmed by experiments. But there was a time in the past when weonly had the predictions and no confirmation. The community was still workingon the technology to build large enough accelerators to check the model claims.The standard model was considered a valid approach then, and only later itwas confirmed by experiments. Therefore, looking for a way to solve technicaldifficulties was considered then a valid procedure. The history of the standard10odel seem to suggest that, if you just find one solution to a technical problem,that solution might be correct.It may sound fair to ask for a similar treatment as the one that the standardmodel received. But that is no solid argument about the reliability of thetheory in itself. It could simply mean the standard model was accepted tooeasily. In particular, normative solutions to the problem of induction include nocomparison to human past behavior, even the behavior of scientists, and withgood reason. So, it seems this argument should be properly ignored. It reallydoes not fit our scheme of how a perfect induction must be performed. But, inreality, performing a perfect induction is an impossible task.We know we are unable to do the complete, perfect case. What this thirdargument does is to pose an interesting question. We do not have the full setof theories. And our resources to create new theories are limited. It can bea reasonable strategy to look for clues on which theoretical paths are moreworth pursuing. While a heuristics about theoretical paths have no place in theidealized induction solution, it can still be very useful in the real life. Inductionwill tell us nothing here. Our normative solutions assume we know all theories.In this case, learning from successful cases in the past can be a good strategy.The third argument is basically a theory on which theories are more likely to besuccessful. The argument does not make string theory more probable. But itcan make it a decent bet on where we should spend our efforts. As any heuristic,this one might fail. But if a heuristic is the best we have, it makes sense to useit and allow the string theory program to go on. Human reasoning is flawed. We need mathematical and logical rules to be ableto say with some degree of certainty that an idea might be true. Our tendencyto defend the ideas we like instead of making impartial estimates means we neednormative rules to replace our judgments in theory choice. Luckily, we have nowsolutions to the problem of induction. Bayesian statistics and the Solomonoffinduction are those solutions. They are also equivalent. The only difference isthe fact that Solomonoff induction comes with rules that, translated into theBayesian method, correspond to a pre-choice of priors. What both accountsimmediately show is that the question of whether a theory is scientific or notmakes no sense. All theories are acceptable and should have their consequencescalculated. From there we can rank them as more or less probable and thatis it. Some theories will describe the data so badly that their probabilitieswill be ridiculously close to zero. But that still does not make those theoriesnot scientific. It just makes them bad descriptions of the world. Not onlystring theory, but non-testable conjectures such as the existence of a multiverseor inflationary theories [34] are valid scientific ideas. That is a very differentstatement from claiming any of those ideas has decent chances of being thebest theory. For that, we do require better confirmatory evidence such as data.Some among us must learn to accept that untestable ideas such as the multiverse11nd inflation are valid theories. Others must accept that without experimentalconfirmation, they are not likely to be true. Unless they come as predictionsfrom the majority of theories that are compatible with the data. Sadly, the spaceof all theories is an infinite space we are not capable of exploring fully. We areleft with doubt. When underdeterminacy happens, competing theories surviveand we can not discard any of them. Once we have some data, if there is stillunderdetermination, the corresponding theories survive with the same initialrelative weight. To get rid of this prior, we can only keep on working until wefind differences between the theories and ways to measure them. If that takes along time and we feel frustrated, that says nothing about the theories. It is onlyabout our desire to know the truth. That desire can be actually quite harmfulto our ability to make sound judgments [17].A question that needed an answer was if only data can provide confirmation.String theory seemed to get its support from theoretical arguments, instead ofdata. I have shown Solomonoff induction can help us understand better thesequestions. Dawid’s arguments [6] may sound just plainly wrong from a purelyBayesian point of view. But by understanding how they stand in terms ofSolomonoff induction, we were able to see their real weaknesses and strengths.Questions about the simplicity and power of a theory actually have an effect onthe weight these theories have. In Bayesian terms, the characteristics of a theorycan influence its prior probability. More powerful theories that can be describedas smaller algorithms from where everything can be computed correspond tolarger Solomonoff weights. They have larger priors. It makes sense to considerString Theory somehow validated by its theoretical characteristics. This shouldhave happened before we start collecting any data. As theories appear afterdata is collected, we can only correct our analysis later.Finally, the idea that a strategy that provided us verifiable and good theoriesin the past might do the same again poses an interesting question. Our modelsof normative induction have nothing to say about this problem, as they assumeall theories are known before any data is collected. As we are limited, it makessense to use this kind of heuristic to evaluate which theoretical paths are morelikely to produce good theories. While this heuristic might not make stringtheory more probable, it does suggest it is worth understanding it better.
Acknowledgments
The author would like to thank the Funda¸c˜ao de Amparo `a Pesquisa do Es-tado de S˜ao Paulo (FAPESP) for partial support to this research under grant2014/00551-0.
References [1] Eugene Wigner. The unreasonable effectiveness of mathematics in the nat-ural sciences.
Communications in Pure and Applied Mathematics , 13(1),12960.[2] George Ellis and Joe Silk. Scientific method: Defend the integrity ofphysics. 516:321–323, 2014.[3] E.T. Jaynes.
Probability Theory: The Logic of Science . Cambridge, Cam-bridge University Press, 2003.[4] Richard Dawid.
String Theory and the Scientific Method . Cambridge Uni-versity Press, 2013.[5] Davide Castelvecchi. Feuding physicists turn to philosophy for help.528:446–447, 2015.[6] Richard Dawid. Underdetermination and theory succession from the per-spective of string theory. 73(3):298–322, 2006.[7] Andr´e C. R. Martins. Stop the tests: Opinion bias and statistical tests.Arxiv preprint: 1611.06545, 2016.[8] R. J. Solomonoff. A formal theory of inductive inference. part i.
Informationand Control , 7(1):1–22, 1964.[9] Hugo Mercier and Dan Sperber. Why do humans reason? arguments foran argumentative theory.
Behavioral and Brain Sciences , 34:57–111, 2011.[10] Dan M. Kahan. Ideology, motivated reasoning, and cognitive reflection.
Judgment and Decision Making , 8:407–424, 2013.[11] Stuart Oskamp. Overconfidence in case-study judgments.
Journal of Con-sulting Psychology , 29(3):261–265, 1965.[12] Claire I. Tsai, Joshua Klayman, and Reid Hastie. Effects of amount ofinformation on judgment accuracy and confidence.
Organizational Behaviorand Human Decision Processes , 107:97105, 2008.[13] Jay J. Christensen-Szalanski and James B. Bushyhead. Physicians’ use ofprobabilistic information in a real clinical setting.
Journal of ExperimentalPsychology: Human Perception and Performance , 7(4):928–935, 1981.[14] Allan H. Murphy and Robert L. Winkler. Probability forecasting in mete-orology.
Journal of the American Statistical Association , 79(387):489–500,1984.[15] Robyn M. Dawes, David Faust, and Paul E. Meehl. Clinical versus actuarialjudgment.
Science , 243:1668–1674, 1989.[16] Emily Pronin, Daniel M. Wegner, Kimberly McCarthy, and Sylvia Ro-driguez. Everyday magical powers: The role of apparent mental causationin the overestimation of personal influence.
Journal of Personality andSocial Psychology , 91(2):218–231, 2006.1317] Andr´e C. R. Martins. Thou shalt not take sides: Cognition, logic and theneed for changing how we believe.
Frontiers in Physics , 4(7), 2016.[18] K. Popper.
The Logic of Scientific Discovery . London, Hutchinson, 1959.[19] Pierre Duhem.
La th´eorie physique: Son Objet et sa Structure . 1989.[20] W.V. Quine. Epistemology naturalized. In
Ontological Relativity and OtherEssays . New York, Columbia University Press, 1969.[21] R. T. Cox.
The Algebra of Probable Inference . John Hopkins UniversityPress, 1961.[22] J. P. A. Ioannidis. Contradicted and initially stronger effects in highly citedclinical research.
Journal of the American Medical Association , 294:218–228, 2005.[23] Gerd Gigerenzer and Julian N. Marewski. Surrogate science: The idolof a universal method for scientific inference.
Journal of Management ,41(2):421–440, 2015.[24] David Trafimow and Michael Marks. Editorial.
Basic and Applied SocialPsychology , 37:1–2, 2015.[25] B. Fitelson and N. Thomason. Bayesians sometimes cannot ignore evenvery implausible theories (even ones that have not yet been thought of).
Australasian Journal of Logic , 6:25–36, 2008.[26] Andr´e C. R. Martins. Theoretical omniscience: Old evidence or new the-ory. PhilSci Preprint at http://philsci-archive.pitt.edu/archive/00002458/,2005.[27] Marcus Hutter. On universal prediction and bayesian confirmation. 384:33–48, 2007.[28] Samuel Rathmanner and Marcus Hutter. A philosophical treatise of uni-versal induction.
Entropy , 13(6):1076, 2011.[29] D. Garber. Old evidence and logical omniscience in bayesian confirmationtheory. In J. Earman, editor,
Testing Scientific Theories , volume X of
Minnesota Studies in the Philosophy of Science . University of MinneapolisPress, 1983.[30] Ray Solomonoff. Does algorithmic probability solve the problem of induc-tion.
Information, Statistics and Induction in Science , pages 7–8, 1996.[31] Roger Penrose.
The Road to Reality . Vintage Books, 2005.[32] Carlo Rovelli and Lee Smolin. Loop space representation of quantum gen-eral relativity.