Mainak Mazumdar

Addressing Challenges with Big Data for Media Measurement

The digital media and TV - which is increasingly digitized, have amassed and generating enormous amount of data. While extremely useful, the big data generated by these platforms poses unique challenges for Data Scientists working on developing measurement framework and metrics. Most practitioners optimize speed and scale at the expense of accuracy, which is critical for any measurement. And, the trade-off between bias and variance is not in consideration. In this paper, we will demonstrate how Nielsen is combining proprietary ground truth data and methodologies with Big Data to address the accuracy and bias/variance challenges. We argue that high quality ground truth or training set is pre-requisite to deploying Big Data for high quality media measurement. To illustrate the point, we will share how Nielsen is combining its proprietary high quality panels with Set Top Box for TV measurement in the U.S.