Hristos Doucouliagos

THE AID EFFECTIVENESS LITERATURE: THE SAD RESULTS OF 40 YEARS OF RESEARCH

The aid effectiveness literature (AEL) consists of empirical macroeconomic estimates of the effects of development aid. By the end of 2004, it comprised 97 econometric studies of three families of related effects. Each family has been analyzed in a separate meta-analysis. The AEL is an ideal subject for meta-analysis as it uses only a few formally similar models to estimate the same underlying effects. It is also an area with strong beliefs, often generated by altruism. When this whole literature is examined, a clear pattern emerges. After 40 years of development aid, the preponderance of the evidence indicates that aid has not been effective. We show that the distribution of results is significantly asymmetric reflecting the reluctance of the research community to publish negative results. The Dutch disease effect on exchange rates provides a plausible explanation for the observed aid ineffectiveness.

Democracy and Economic Growth: A Meta-Analysis

Despite a sizeable theoretical and empirical literature, no firm conclusions have been drawn regarding the impact of political democracy on economic growth. This paper challenges the consensus of an inconclusive relationship through a quantitative assessment of the democracy-growth literature. It applies meta-regression analysis to the population of 483 estimates derived from 84 studies on democracy and growth. Using traditional meta-analysis estimators, the bootstrap, and Fixed and Random Effects meta-regression models, it derives several robust conclusions. Taking all the available published evidence together, it concludes that democracy does not have a direct impact on economic growth. However, democracy has robust, significant, and positive indirect effects through higher human capital, lower inflation, lower political instability, and higher levels of economic freedom. Democracies may also be associated with larger governments and less free international trade. There also appear to be country- and region-specific democracy-growth effects. Overall, democracys net effect on the economy does not seem to be detrimental.

Meta-regression approximations to reduce publication selection bias

Publication selection bias is a serious challenge to the integrity of all empirical sciences. We derive meta-regression approximations to reduce this bias. Our approach employs Taylor polynomial approximations to the conditional mean of a truncated distribution. A quadratic approximation without a linear term, precision-effect estimate with standard error (PEESE), is shown to have the smallest bias and mean squared error in most cases and to outperform conventional meta-analysis estimators, often by a great deal. Monte Carlo simulations also demonstrate how a new hybrid estimator that conditionally combines PEESE and the Egger regression intercept can provide a practical solution to publication selection bias. PEESE is easily expanded to accommodate systematic heterogeneity along with complex and differential publication selection bias that is related to moderator variables. By providing an intuitive reason for these approximations, we can also explain why the Egger regression works so well and when it does not. These meta-regression methods are applied to several policy-relevant areas of research including antidepressant effectiveness, the value of a statistical life, the minimum wage, and nicotine replacement therapy.

Meta-Analysis Of Economics Research Reporting Guidelines

Meta‐regression analysis (MRA) can provide objective and comprehensive summaries of economics research. Their use has grown rapidly over the last few decades. To improve transparency and to raise the quality of MRA, the meta‐analysis of economics research‐network (MAER‐Net) has created the below reporting guidelines. Future meta‐analyses in economics will be expected to follow these guidelines or give valid reasons why a meta‐analysis must deviate from them.

Aid Effectiveness on Accumulation. A Meta Study

The AEL (aid effectiveness literature) studies the macroeconomic effects of development aid using cross-country or panel data econometrics. It contains 97 papers of which 43 study whether development aid leads to increasing accumulation. The aggregate results of the 43 studies are that aid increases investment with about 25% of the aid, while most of the remaining 75% of the effect is crowded out by a fall in savings. However, these aggregate results are so variable that it is dubious if accumulation rises.

PICTURE THIS: A SIMPLE GRAPH THAT REVEALS MUCH ADO ABOUT RESEARCH

Funnel graphs provide a simple, yet highly effective, means to identify key features of an empirical literature. This paper illustrates the use of funnel graphs to detect publication selection bias, identify the existence of genuine empirical effects and discover potential moderator variables that can help to explain the wide variation routinely found among reported research findings. Applications include union–productivity effects, water price elasticities, common currency-trade effects, minimum-wage employment effects, efficiency wages and the price elasticity of prescription drugs.

Unions and Profits: A Meta-Regression Analysis

The effect of unions on profits continues to be an unresolved theoretical and empirical issue. In this paper, clustered data analysis and hierarchical linear meta-regression models are applied to the population of forty-five econometric studies that report 532 estimates of the direct effect of unions on profits. Unions have a significant negative effect on profits in the United States, and this effect is larger when market-based measures of profits are used. Separate meta-regression analyses are used to identify the effects of market power and long-lived assets on profits, as well as the sources of union-profit effects. The accumulated evidence rejects market power as a source of union-profit effects. While the case is not yet proven, there is some evidence in support of the appropriation of quasi-rent hypothesis. There is a clear need for further American and non-American primary research in this area.

Neither fixed nor random: weighted least squares meta‐analysis

This study challenges two core conventional meta-analysis methods: fixed effect and random effects. We show how and explain why an unrestricted weighted least squares estimator is superior to conventional random-effects meta-analysis when there is publication (or small-sample) bias and better than a fixed-effect weighted average if there is heterogeneity. Statistical theory and simulations of effect sizes, log odds ratios and regression coefficients demonstrate that this unrestricted weighted least squares estimator provides satisfactory estimates and confidence intervals that are comparable to random effects when there is no publication (or small-sample) bias and identical to fixed-effect meta-analysis when there is no heterogeneity. When there is publication selection bias, the unrestricted weighted least squares approach dominates random effects; when there is excess heterogeneity, it is clearly superior to fixed-effect meta-analysis. In practical applications, an unrestricted weighted least squares weighted average will often provide superior estimates to both conventional fixed and random effects.

Could It Be Better to Discard 90% of the Data? A Statistical Paradox

Conventional practice is to draw inferences from all available data and research results. When a scientific literature is plagued by publication selection bias, a simple discarding of the vast majority of empirical results can actually improve statistical inference and estimation. Simulations demonstrate that, if statistical significance is used as a criterion for reporting or publishing estimates, discarding 90% of the published findings greatly reduces publication selection bias and is often more efficient than conventional summary statistics. Improving statistical estimation and inference through removing so much data goes against statistical theory and practice; hence, it is paradoxical. We investigate a very simple method to reduce the effects of publication bias and to improve the efficiency of summary estimates of accumulated empirical research results that averages the most precise 10% of the reported estimates (i.e., ‘Top10’). In the process, the critical importance of precision (the inverse of an estimate’s standard error) as a measure of a study’s quality is brought to light. Reviewers and journal editors should use precision, when possible, as one objective measure of a study’s quality.

Aid Effectiveness on Growth

The AEL (aid effectiveness literature) is econometric studies of the macroeconomic effects of development aid. It contains about 100 papers of which 68 are reduced form estimates of the effect of aid on growth in the recipient country. The raw data show that growth is unconnected to aid, but the AEL has put so much structure on the data that all results possible have emerged. The present meta study considers both the best-set of the 68 papers and the all-set of 543 regressions published. Both sets have a positive average aid-growth elasticity, but it is small and insignificant: The AEL has not established that aid works. Using meta regression analysis it is shown that about 20 factors influence the results. Much of the variation between studies is an artifact and can be attributed to publication outlet, institutional affiliation, and specification differences. However, some of the difference between studies is real. In particular, the aid-growth association is stronger for Asian countries, and the aid-growth association is shown to have been weaker in the 1970s.