Marcel Boumans

Measurement in Economics

The Representational Theory of Measurement conceives measurement as establishing homomorphisms from empirical relational structures into numerical relation structures, called models. There are two different approaches to deal with the justification of a model: an axiomatic and an empirical approach. The axiomatic approach verifies whether a given relational structure satisfies certain axioms to secure homomorphic mapping. The empirical approach conceives models to function as measuring instruments by transferring observations of an economic system into quantitative facts about that system. These facts are evaluated by their accuracy and precision. Precision is achieved by least squares methods and accuracy by calibration. For calibration standards are needed. Then two strategies can be distinguished. One aims at estimating the invariant (structural) equations of the system. The other strategy is to use known stable facts about the system to adjust the model parameters. For this strategy, the requirement of models as homomorphic mappings has been dropped.

'Ceteris paribus' conditions: materiality and the application of economic theories

The apparent gulf between the pronouncements of the methodology of economics literature and the work of economists in their applied and empirical domains has created a position in which economic methodologists can blithely use the phrase: ‘testing theories by confronting them with data’ and then give little attention to the detailed problems which economists face in making claims about the practical applicability of their theories. In saying this, of course we set up a straw methodologist, for particularly those concerned with econometrics have thought long and deep about this in relation to econometric research. Yet, in our experience, there is still little attention paid to two rapidly growing forms of research, namely laboratory experiments and simulations, and even less literature which analyses the methodological diierences between these newer methods and the older research methods used to apply theories, and compares the potential benefits and pitfalls of one approach versus another. We argue here that one useful direction for methodological work in the next years would be to provide an analytical comparison of the standard research activities of current economics. Comparative analysis has the power to provide insights and prevent oversights, and thus deepen our understanding of how the specific methodological assumptions and preconceptions used in each research activity work out in practise. We choose to argue this point by considering the functioning in this context of a well-known, but perhaps relatively underappreciated, rule: the ceteris paribus clause, in four different current environments: namely in mathematical modelling, econometrics, simulations and experiments.

Science outside the laboratory : measurement in field science and economics

The conduct of most of social science occurs outside the laboratory. Such studies in field science explore phenomena that cannot for practical, technical, or ethical reasons be explored under controlled conditions. These phenomena cannot be fully isolated from their environment or investigated by manipulation or intervention. Yet measurement, including rigorous or clinical measurement, does provide analysts with a sound basis for discerning what occurs under field conditions, and why. Science Outside the Laboratory explores the state of measurement theory, its reliability, and the role expert judgment plays in field investigations from the perspective of the philosophy of science. Its discussion of the problems of passive observation, the calculus of observation, the two-model problem, and model-based consensus uses illustrations drawn primarily from economics. The treatment clarifies the extent to which measurement provides valid information about objects and events in field sciences, but also has implications for measurement in the laboratory. Available in OSO:

Measurement Outside the Laboratory

The kinds of models discussed in this paper function as measuring instruments. We will concentrate on two necessary steps for measurement: (1) one should search for a mathematical representation of the phenomenon; (2) this representation about the phenomenon should cover an invariant relationship between properties of the phenomenon to be measured and observable associated attributes of a measuring instrument. Therefore, the measuring instrument should function as a nomological machine. However, invariant relationships are not necessarily ceteris paribus regularities, but could also occur when the influence of the environment is negligible. Then we are able to achieve accurate measurements outside the laboratory.

How to Design Galilean Fall Experiments in Economics

In the social sciences we hardly can create laboratory conditions, we only can try to find out which kinds of experiments Nature has carried out. Knowledge about Nature’s designs can be used to infer conditions for reliable predictions. This problem was explicitly dealt with in Haavelmo’s (1944) discussion of autonomous relationships, Friedman’s (1953) as‐if methodology, and Simon’s (1961) discussions of nearly‐decomposable systems. All three accounts take Marshallian partitioning as starting point, however not with a sharp ceteris paribus razor but with the blunt knife of negligibility assumptions. As will be shown, in each account reflection on which influences are negligible, for what phenomena and for how long, played a central role.

Battle in the planning office: biased experts versus normative statisticians

Generally, rational decision‐making is conceived as arriving at a decision by a correct application of the rules of logic and statistics. If not, the conclusions are called biased. After an impressive series of experiments and tests carried out in the last few decades, the view arose that rationality is tough for all, skilled field experts not excluded. A new type of planner’s counsellor is called for: the normative statistician, the expert in reasoning with uncertainty par excellence. To unravel this view, the paper explores a specific practice of clinical decision‐making, namely Evidence‐Based Medicine. This practice is chosen, because it is very explicit about how to rationalize practice. The paper shows that whether a decision‐making process is rational cannot be assessed without taking into account the environment in which the decisions have to be taken. To be more specific, the decision to call for new evidence should be rational too. This decision and the way in which this evidence is obtained are crucial to validate the base rates. Rationality should be model‐based, which means that not only the isolated decision‐making process should take a Bayesian updating process as its norm, but should also model the acquisition of evidence (priors and tests results) as a rational process.

The reliability of an instrument

Salviati… in our time it has pleased God to concede to human ingenuity an invention so wonderful as to have the power of increasing vision four, six, ten, twenty, thirty, and forty times, and an infinite number of objects which were invisible, either because of distance or extreme minuteness, have become visible by means of the telescope. Simplicio Everything that Salviati is presently setting forth is truly new to me. Frankly, I had no interest in reading those books, nor up till now have I put any faith in the newly introduced optical device. Instead, following in the footsteps of other Peripatetic philosophers of my group, I have considered as fallacies and deceptions of the lenses those things which other people have admired as stupendous achievements. (Galileo Galilei, Dialogue Concerning the Two Chief World Systems—Ptolemaic & Copernican, 335–6)

The Problem of Passive Observation

Trygve Haavelmos methodological manifesto “The Probability Approach in Econometrics” not only laid down the paradigm for the research pursued at the Cowles Commission, but also sets out a strategy for measurement outside the laboratory. His conceptualization of “passive observation” is still very useful for understanding measurements where intervention is not possible. Haavelmos classic is very rich: it provided the framework for introducing probabilistic methods in econometrics and a profound discussion on invariance (autonomy). These subjects are well treated by various historians of econometrics. This does not, however, apply to the problem of passive observation. It is only mentioned, if it is mentioned at all, in relation to the discussion of autonomy, but that is it. This essay will give a reconstruction of his discussion of “factual” and “potential” influences, which provided Haavelmo the framework to discuss “Natures experiments” and will allow us to discuss measurement outside the laboratory more generally.

Calibration of Models in Experiments.

The assessment of models in an experiment depends on their material nature and their function in the experiment. Models that are used to make the phenomenon under investigation visible - sensors - are assessed by calibration. However, calibration strategies assume material intervention. The experiment discssed in this paper is an experiment in economics to measure the influence of technology shocks on business cycles. It uses immaterial, mathematical instruments. It appears that calibration did not work for these kinds of models, it did not provide reliable evidence for the facts of the business cycle.

Experts and consensus in social science

When we evaluate the outcomes of investigative actions as justified or unjustified, good or bad, rational or irrational, we make, in a broad sense of the term, evaluative judgements about them. We look at operational accuracy as a desirable and evaluable quality of the outcomes and explore how the concepts of accuracy and precision, on the basis of insights borrowed from pragmatics and measurement theory, can be seen to do useful work in epistemology. Operational accuracy (but not metaphysical accuracy!) focuses on how a statement fits an explicit or implicit standard set by participants involved in a shared project. While truth can remain a thin semantic property of propositions, operational accuracy, as a quality of an outcome of inquiry and typically attached to a statement, a model, a diagram or a representation is an evaluation based on the the non-epistemic goals set by the goal of inquiry (which every inquiry has), and a substantial evaluative notion. The goals, often made explicit by relevant questions in a context of inquiry, act as a filter, with truths a reliable epistemic method has access to functioning as the input, and accurate representations as its output. Responsible inquiry seeks pragmatic equilibrium between what reliable knowledge on the one hand and degrees of accuracy required by the goal of inquiry.