Dominic Coey

Why marketplace experimentation is harder than it seems: the role of test-control interference

Classical statistical inference of experimental data assumes that the treatment affects the test group but not the control group. This assumption will typically be violated when experimenting in marketplaces because of general equilibrium effects: changing test demand affects the supply available to the control group. We illustrate this with an email marketing campaign performed by eBay. Ignoring test-control interference leads to estimates of the campaigns effectiveness which are too large by a factor of around two. We present the simple economics of this bias in a supply and demand framework, showing that the bias is larger in magnitude where there is more inelastic supply, and is positive if demand is elastic.

The Effect of Medicaid on Health Care Consumption of Young Adults

All states provide Medicaid until the age of 19 years. After 19 years, young adults may become ineligible for Medicaid. Using the Medical Expenditure Panel Survey, we find that the resulting loss of Medicaid coverage causes substantial changes to the level and composition of health care use. The total number of visits to health care providers falls by over 60%, two-thirds of which is due to a decline in office visits. Expenditures, in particular inpatient expenditures, also appear to fall sharply.