Stephan B. Bruns

Is There Really Granger Causality Between Energy Use and Output

We carry out a meta-analysis of the very large literature on Granger causality tests between energy use and economic output to determine if there is a genuine effect in this literature or whether the large number of apparently significant results is due to publication and misspecification bias. Our model extends the standard meta-regression model for detecting genuine effects using the statistical power trace in the presence of publication biases by controlling for the tendency to over-fit vector auto regression models in small samples. These over-fitted models have inflated type 1 errors. We find that models that include energy prices as a control variable find a genuine effect from output to energy use in the long-run. A genuine causal effect also seems apparent from energy to output when employment is controlled for and the Johansen procedure is used.

What If Energy Time Series are Not Independent? Implications for Energy-GDP Causality Analysis

Time series of electricity, petroleum products, and renewables are found to be highly correlated with total energy consumption. Applying this insight to the huge literature on energy-GDP causality explains that the results of energy-GDP causality tests frequently coincide with the results of energy type-GDP tests. Using the test by Toda-Yamamoto in combination with a cointegration-based testing approach, we detect such cases of concordance for 92 per cent of the countries in our sample of 65 countries. As a consequence, it is difficult to draw specific economic conclusions regarding single types of energy from bivariate causality analysis.

p-Curve and p-Hacking in Observational Research

The p-curve, the distribution of statistically significant p-values of published studies, has been used to make inferences on the proportion of true effects and on the presence of p-hacking in the published literature. We analyze the p-curve for observational research in the presence of p-hacking. We show by means of simulations that even with minimal omitted-variable bias (e.g., unaccounted confounding) p-curves based on true effects and p-curves based on null-effects with p-hacking cannot be reliably distinguished. We also demonstrate this problem using as practical example the evaluation of the effect of malaria prevalence on economic growth between 1960 and 1996. These findings call recent studies into question that use the p-curve to infer that most published research findings are based on true effects in the medical literature and in a wide range of disciplines. p-values in observational research may need to be empirically calibrated to be interpretable with respect to the commonly used significance threshold of 0.05. Violations of randomization in experimental studies may also result in situations where the use of p-curves is similarly unreliable.

Is there really Granger causality between energy use and output

We carry out a meta-analysis of the very large literature on testing for Granger causality between energy use and economic output to determine if there is a genuine effect in this literature or whether the large number of apparently significant results is due to publication or misspecification bias. Our model extends the standard meta-regression model for detecting genuine effects in the presence of publication biases using the statistical power trace by controlling for the tendency to over-fit vector autoregression models in small samples. Granger causality tests in these over-fitted models have inflated type I errors. We cannot find a genuine causal effect in the literature as a whole. However, there is a robust genuine effect from output to energy use when energy prices are controlled for.

Research assessment using early citation information

Peer-review based research assessment, as implemented in Australia, the United Kingdom, and some other countries, is a very costly exercise. We show that university rankings in economics based on long-run citation counts can be easily predicted using early citations. This would allow a research assessment to predict the relative long-run impact of articles published by a university immediately at the end of the evaluation period. We compare these citation-based university rankings with the rankings of the 2010 Excellence in Research assessment in Australia and the 2008 Research Assessment Exercise in the United Kingdom. Rank correlations are quite strong, but there are some differences between rankings. However, if assessors are willing to consider citation analysis to assess some disciplines, as is the case for the natural sciences and psychology in Australia, it seems reasonable to consider also including economics in that set.

Meta-Granger Causality Testing

Understanding the (causal) mechanisms at work is important for formulating evidence-based policy. But evidence from observational studies is often inconclusive with many studies finding conflicting results. In small to moderately sized samples, the outcome of Granger causality testing heavily depends on the lag length chosen for the underlying vector autoregressive (VAR) model. Using the Akaike Information Criterion, there is a tendency to overfit the VAR model and these overfitted models show an increased rate of false-positive findings of Granger causality, leaving empirical economists with substantial uncertainty about the validity of inferences. We propose a meta-regression model that explicitly controls for this overfitting bias and we show by means of simulations that, even if the primary literature is dominated by false-positive findings of Granger causality, the meta-regression model correctly identifies the absence of genuine Granger causality. We apply the suggested model to the large literature that tests for Granger causality between energy consumption and economic output. We do not find evidence for a genuine relation in the selected sample, although excess significance is present. Instead, we find evidence that this excess significance is explained by overfitting bias.