Charles Twardy

Expert status and performance.

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.

Probabilistic Coherence Weighting for Optimizing Expert Forecasts

Methods for eliciting and aggregating expert judgment are necessary when decision-relevant data are scarce. Such methods have been used for aggregating the judgments of a large, heterogeneous group of forecasters, as well as the multiple judgments produced from an individual forecaster. This paper addresses how multiple related individual forecasts can be used to improve aggregation of probabilities for a binary event across a set of forecasters. We extend previous efforts that use probabilistic incoherence of an individual forecasters subjective probability judgments to weight and aggregate the judgments of multiple forecasters for the goal of increasing the accuracy of forecasts. With data from two studies, we describe an approach for eliciting extra probability judgments to (i) adjust the judgments of each individual forecaster, and (ii) assign weights to the judgments to aggregate over the entire set of forecasters. We show improvement of up to 30% over the established benchmark of a simple equal-weighted averaging of forecasts. We also describe how this method can be used to remedy the “fifty--fifty blip” that occurs when forecasters use the probability value of 0.5 to represent epistemic uncertainty.

A criterion of probabilistic causation

The investigation of probabilistic causality has been plagued by a variety of misconceptions and misunderstandings. One has been the thought that the aim of the probabilistic account of causality is the reduction of causal claims to probabilistic claims. Nancy Cartwright (1979) has clearly rebutted that idea. Another ill‐conceived idea continues to haunt the debate, namely the idea that contextual unanimity can do the work of objective homogeneity. It cannot. We argue that only objective homogeneity in combination with a causal interpretation of Bayesian networks can provide the desired criterion of probabilistic causality.

Automated trading in prediction markets

This research presents the ongoing results of trading experiments that have been performed on the DAGGRE prediction market. DAGGRE is a research project that aims to improve the forecasting methods of world events using prediction markets, crowdsourcing and Delphi groups. The DAGGRE prediction market aggregates estimates from hundreds of participants to forecast the outcome of these events. On the prediction market that involves a few thousand human traders, during a time period of a year and a half, we introduced 3 trading algorithms that have been trading live on the market, based on different rules and trading policies. While all the Autotraders improved the overall market participation and activity and outperform most of the human traders, one of them is adaptive to the new information that continuously comes from the market. This paper presents the comparative analysis of the forecasting accuracy and market performance of these 3 Autotraders and discusses the preliminary results of these experiments.

Actual Causation by Probabilistic Active Paths

Recently Halpern and Pearl and Hitchcock have presented influential accounts of actual (token) causation using Bayesian networks. These accounts have been deterministic. Here we present a probabilistic extension to these active path analyses of actual causation. The extension uses “soft” interventions to set distributions rather than just single values. The resulting account can handle at least as wide a range of examples as the original accounts, without assuming determinism.

Local complexity adaptable trajectory partitioning via minimum message length

We present a minimum message length (MML) framework for trajectory partitioning by point selection, and use it to automatically select the tolerance parameter є for Douglas-Peucker partitioning, adapting to local trajectory complexity. By examining a range of є for synthetic and real trajectories, it is easy to see that the best є does vary by trajectory, and that the MML encoding makes sensible choices and is robust against Gaussian noise. We use it to explore the identification of micro-activities within a longer trajectory. This MML metric is comparable to the TRACLUS metric - and shares the constraint of abstracting only by omission of points - but is a true lossless encoding. Such encoding has several theoretical advantages - particularly with very small segments (high frame rates) - but actual performance interacts strongly with the search algorithm. Both differ from unconstrained piecewise linear approximations, including other MML formulations.