Stephen T. Ziliak

Signifying nothing: reply to Hoover and Siegler

After William Gosset (1876–1937), the ‘Student’ of Students t, the best statisticians have distinguished economic (or agronomic or psychological or medical) significance from merely statistical ‘significance’ at conventional levels. A singular exception among the best was Ronald A. Fisher, who argued in the 1920s that statistical significance at the 0.05 level is a necessary and sufficient condition for establishing a scientific result. After Fisher many economists and some others – but rarely physicists, chemists, and geologists, who seldom use Fisher‐significance – have mixed up the two kinds of significance. We have been writing on the matter for some decades, with other critics in medicine, sociology, psychology, and the like. Hoover and Siegler, despite a disdainful rhetoric, agree with the logic of our case. Fisherian ‘significance,’ they agree, is neither necessary nor sufficient for scientific significance. But they claim that economists already know this and that Fisherian tests can still be used for specification searches. Neither claim seems to be true. Our massive evidence that economists get it wrong appears to hold up. And if rhetorical standards are needed to decide the importance of a coefficient in the scientific conversation, so are they needed when searching for an equation to fit. Fisherian ‘significance’ signifies nearly nothing, and empirical economics as actually practiced is in crisis.

Visualizing Uncertainty: On Soyer's and Hogarth's 'The Illusion of Predictability: How Regression Statistics Mislead Experts'

This comment was prepared for the International Journal of Forecasting mini-symposium on the Soyer-Hogarth experiment. The experiment evaluates the ability of expert econometricians to make predictions based on commonly provided regression output. Visual displays of quantitative information, including simple plots of data, outperformed predictions based on R-squared, t-statistics, and other common diagnostics. Reliance on graphing - on the visualization of uncertainty - was suggested more than a century ago by Karl Pearson, a founding father of English language statistics. The results of the Soyer and Hogarth experiment, when combined with evidence produced by Ziliak and McCloskey (2008) and others, suggests that graphing and visualization should receive more attention and tests of statistical significance, less.

W.S. Gosset and Some Neglected Concepts in Experimental Statistics: Guinnessometrics II

Students exacting theory of errors, both random and real, marked a significant advance over ambiguous reports of plant life and fermentation asserted by chemists from Priestley and Lavoisier down to Pasteur and Johannsen, working at the Carlsberg Laboratory. One reason seems to be that William Sealy Gosset (1876–1937) aka “Student” – he of Students t -table and test of statistical significance – rejected artificial rules about sample size, experimental design, and the level of significance, and took instead an economic approach to the logic of decisions made under uncertainty. In his job as Apprentice Brewer, Head Experimental Brewer, and finally Head Brewer of Guinness, Student produced small samples of experimental barley, malt, and hops, seeking guidance for industrial quality control and maximum expected profit at the large scale brewery. In the process Student invented or inspired half of modern statistics. This article draws on original archival evidence, shedding light on several core yet neglected aspects of Students methods, that is, Guinnessometrics, not discussed by Ronald A. Fisher (1890–1962). The focus is on Students small sample, economic approach to real error minimization, particularly in field and laboratory experiments he conducted on barley and malt, 1904 to 1937. Balanced designs of experiments, he found, are more efficient than random and have higher power to detect large and real treatment differences in a series of repeated and independent experiments. Students world-class achievement poses a challenge to every science. Should statistical methods – such as the choice of sample size, experimental design, and level of significance – follow the purpose of the experiment, rather than the other way around? (JEL classification codes: C10, C90, C93, L66)

Pauper Fiction in Economic Science: "Paupers in Almshouses" and the Odd Fit of Oliver Twist

The almshouse dominated the thinking about poverty and the poor during Americas period of industrialization and its greatest economic downturns. Yet economists had surprisingly little to say about the facts of almshouse demography, and what they have written has been a rather bad fiction when seen in contrast with American novels. The main object of the paper is to delineate typical characters and characteristics of almshouses in America, and to examine the plausibility of various literary characterizations in light of the facts. The data certainly suggest new stories about paupers in American history: economists, and even the new social historians, have gotten it wrong. Between the Civil War and the Great Depression, the typical pauper living in an almshouse was not Oliver Twist (as many believe). He was not the Shiftless Man of the classical imagination (as Malthusians and Benthamites believe). The typical pauper of an American almshouse was plural. Instructive examples in American literature include Lennie, of Steinbecks Of Mice and Men ; Denver, of Toni Morrisons Beloved ; Mrs. Thomson, of Edward Egglestons The Hoosier School-Master ; and Forrest Gump, of Winston Grooms Forrest Gump .

The Cult of Significance: Hardcover: 322+xxiv pages Publisher: The University of Michigan Press (First edition, Feb. 2008) Language: English ISBN-13: 978-0472050079

51 This book, written by economists Stephen Ziliak and Deirdre McCloskey, has a theme bound to attract both Bayesians and all those puzzled by the absolute and automatic faith in significance tests exhibited in many applied papers. The authors’ main argument is indeed that an overwhelming majority of papers involved in data analysis stop at rejecting variables (“coeficients”) on the sole and unsupported basis of nonsignificance at the 5% level, hence the subtitle, “How the standard error costs us jobs, justice, and lives.” Th is is an argument I completely agree with; however, the aggressive style of the book ended up putting me off as early as the fi rst chapter. Obviously, I could have let both the matter and the book go, but I think the book may, in the end, do a disservice to a valid issue and I thus endeavour to explain why.

Probability, Econometrics, and Truth: The Methodology of Econometrics. By Hugo A. Keuzenkamp. Cambridge: Cambridge University Press, 2000. Pp. ix, 312.

Richard von Mises (1883–1953), the younger brother of Ludwig, had an Apollonian genius. He was a leading probabilist and statistician of the world, a leading engineer and designer of airplanes for the Austro-Hungarian Empire, and a leading philosopher of “positivism.†Von Mises was rational and measured, a man of observations and proportions. Though too modest to say so, he was a hero in his own story of scientific philosophy. A Harvard professor, von Mises was also learned in the humanities: his Positivism: A Study in Human Understanding (Cambridge, MA: Harvard University Press, 1939 [1951]) is painted with images from Nietzsche, Schopenhauer, and Goethe, and in equal proportions with the positivist icons of Mach and Comte and Kant (pp. 401–04). Von Mises owned and loved the worlds largest collection of Rilke poetry before bequeathing the collection to Harvards Houghton library. But von Mises believed his Dionysian self to be rhetorically separable from the positivist-scientist self, and lower. He tried to peel rhetoric away from science and gaze at its logical and empirical clarity. Abraham Wald was his student. His theory of collective probability influenced the econometrics of Trygve Haavelmo. Yet econometricians and philosophers younger than Arnold Zellner have never heard of Richard von Mises.