Johannes Bjelland

Mapping poverty using mobile phone and satellite data

Poverty is one of the most important determinants of adverse health outcomes globally, a major cause of societal instability and one of the largest causes of lost human potential. Traditional approaches to measuring and targeting poverty rely heavily on census data, which in most low- and middle-income countries (LMICs) are unavailable or out-of-date. Alternate measures are needed to complement and update estimates between censuses. This study demonstrates how public and private data sources that are commonly available for LMICs can be used to provide novel insight into the spatial distribution of poverty. We evaluate the relative value of modelling three traditional poverty measures using aggregate data from mobile operators and widely available geospatial data. Taken together, models combining these data sources provide the best predictive power (highest r2 = 0.78) and lowest error, but generally models employing mobile data only yield comparable results, offering the potential to measure poverty more frequently and at finer granularity. Stratifying models into urban and rural areas highlights the advantage of using mobile data in urban areas and different data in different contexts. The findings indicate the possibility to estimate and continually monitor poverty rates at high spatial resolution in countries with limited capacity to support traditional methods of data collection.

Big Data-Driven Marketing: How machine learning outperforms marketers' gut-feeling

This paper shows how big data can be experimentally used at large scale for marketing purposes at a mobile network operator. We present results from a large-scale experiment in a MNO in Asia where we use machine learning to segment customers for text-based marketing. This leads to conversion rates far superior to the current best marketing practices within MNOs. Using metadata and social network analysis, we created new metrics to identify customers that are the most likely to convert into mobile internet users. These metrics falls into three categories: discretionary income, timing, and social learning. Using historical data, a machine learning prediction model is then trained, validated, and used to select a treatment group. Experimental results with 250 000 customers show a 13 times better conversion-rate compared to the control group. The control group is selected using the current best practice marketing. The model also shows very good properties in the longer term, as 98% of the converted customers in the treatment group renew their mobile internet packages after the campaign, compared to 37% in the control group. These results show that data-driven marketing can significantly improve conversion rates over current best-practice marketing strategies.

Small Circles: Mobile Telephony and the Cultivation of the Private Sphere

This article examines how we use mobile telephony to maintain our physically and socially closest social circle. The analysis is based on traffic data gathered from Norway using approximately 24 million calls and texts made by private individuals. Previous research has shown that our temporal and spatial movement is highly predictable and that the majority of calls and text messages are sent to only four to six different persons. This article extends this research by examining both tie strength and the distance between the interlocutors in urban and rural settings. The findings show that even as information and communication technologies (ICTs) potentially put the world at our fingertips, the mobile phone is an instrument of a more limited geographical and social sphere. Approximately two-thirds of our calls/texts go to strong ties that are within a 25-km radius.

Product Adoption Networks and Their Growth in a Large Mobile Phone Network

To understand the diffusive spreading of a product in a telecom network, whether the product is a service, handset, or subscription, it can be very useful to study the structure of the underlying social network. By combining mobile traffic data and product adoption history from one of Telenor’s markets, we can define and measure an adoption network—roughly, the social network of adopters. By studying the time evolution of adoption networks, we can observe how different products diffuses through the network, and measure potential social influence. This paper presents an empirical and comparative study of three adoption networks evolving over time in a large telecom network. We believe that the strongest spreading of adoption takes place in the dense core of the underlying network, and gives rise to a dominant largest connected component (LCC) in the adoption network, which we call “the social network monster”. We believe that the size of the monster is a good indicator for whether or not a product is taking off. We show that the evolution of the LCC, and the size distribution of the other components, vary strongly with different products. The products studied in this article illustrate three distinct cases: that the social network monsters can grow or break down over time, or fail to occur at all. Some of the reasons a product takes off are intrinsic to the product; there are also aspects of the broader social context that can play in. Tentative explanations are offered for these phenomena. Also, we present two statistical tests which give an indication of the strength of the spreading over the social network. We find evidence that the spreading is dependent on the underlying social network, in particular for the early adopters.

Eigenvectors of directed graphs and importance scores: dominance, T-Rank, and sink remedies

We study the properties of the principal eigenvector for the adjacency matrix (and related matrices) for a general directed graph. In particular—motivated by the use of the eigenvector for estimating the “importance” of the nodes in the graph—we focus on the distribution of positive weight in this eigenvector, and give a coherent picture which builds upon and unites earlier results. We also propose a simple method—“T-Rank”—for generating importance scores. T-Rank generates authority scores via a one-level, non-normalized matrix, and is thus distinct from known methods such as PageRank (normalized), HITS (two-level), and SALSA (two-level and normalized). We show, using our understanding of the principal eigenvector, that T-Rank has a much less severe “sink problem” than does PageRank. Also, we offer numerical results which quantify the “tightly-knit community” or TKC effect. We find that T-Rank has a stronger TKC effect than PageRank, and we offer a novel interpolation method which allows for continuous tuning of the strength of this TKC effect. Finally, we propose two new “sink remedies”, i.e., methods for ensuring that the principal eigenvector is positive everywhere. One of our sink remedies (source pumping) is unique among sink remedies, in that it gives a positive eigenvector without rendering the graph strongly connected. We offer a preliminary evaluation of the effects and possible applications of these new sink remedies.

Small and Even Smaller Circles: The Size of Mobile Phone-Based Core Social Networks in Scandinavia and South Asia

Previous research in developed countries has shown that mobile phone users call and text to a relatively small circle of people. Research from the Global South indicates that core network size is often larger than in the developed world since the logistics of daily life require extended informal logistics. This suggests that the core social network, as seen in the use of mobile voice and texting, will be larger in developing countries than in developed countries. This is tested using mobile phone log data from Norway, Malaysia, Thailand and Pakistan. A total of 4000 subscribers and their “one hop” social networks (approx. 80,000 links) were examined. The results show that the core mobile phone-based networks are not larger in developing countries. This indicates that cost, literacy and other cultural issues are significant when considering the question of core network size as seen in the use of mobile telephony.

Improving official statistics in emerging markets using machine learning and mobile phone data

Mobile phones are one of the fastest growing technologies in the developing world with global penetration rates reaching 90%. Mobile phone data, also called CDR, are generated everytime phones are used and recorded by carriers at scale. CDR have generated groundbreaking insights in public health, official statistics, and logistics. However, the fact that most phones in developing countries are prepaid means that the data lacks key information about the user, including gender and other demographic variables. This precludes numerous uses of this data in social science and development economic research. It furthermore severely prevents the development of humanitarian applications such as the use of mobile phone data to target aid towards the most vulnerable groups during crisis. We developed a framework to extract more than 1400 features from standard mobile phone data and used them to predict useful individual characteristics and group estimates. We here present a systematic cross-country study of the applicability of machine learning for dataset augmentation at low cost. We validate our framework by showing how it can be used to reliably predict gender and other information for more than half a million people in two countries. We show how standard machine learning algorithms trained on only 10,000 users are sufficient to predict individual’s gender with an accuracy ranging from 74.3 to 88.4% in a developed country and from 74.5 to 79.7% in a developing country using only metadata. This is significantly higher than previous approaches and, once calibrated, gives highly accurate estimates of gender balance in groups. Performance suffers only marginally if we reduce the training size to 5,000, but significantly decreases in a smaller training set. We finally show that our indicators capture a large range of behavioral traits using factor analysis and that the framework can be used to predict other indicators of vulnerability such as age or socio-economic status. Mobile phone data has a great potential for good and our framework allows this data to be augmented with vulnerability and other information at a fraction of the cost.

Comparing and Visualizing the Social Spreading of Products on a Large Social Network

By combining mobile traffic data and product adoption history from one of the markets of the telecom provider Telenor, we define and measure an adoption network—roughly, the social network among adopters. We study and compare the evolution of this adoption network over time for several products—the iPhone handset, the Doro handset, the iPad 3G and videotelephony. We show how the structure of the adoption network changes over time, and how it can be used to study the social effects of product diffusion. Specifically, we show that the evolution of the Largest Connected Component (LCC) and the size distribution of the other components vary strongly with different products. We also introduce simple tests for quantifying the social spreading effect by comparing actual product diffusion on the network to random based spreading models. As videotelephony is adopted pairwise, we suggest two types of tests: transactional- and node based adoption test. These tests indicate strong social network dependencies in adoption for all products except the Doro handset. People who talk together, are also likely to adopt together. Supporting this, we also find that adoption probability increases with the number of adopting friends for all the products in this study. We believe that the strongest spreading of adoption takes place in the dense core of the underlying network, and gives rise to a dominant LCC in the adoption network, which we call “the social network monster”. This is supported by measuring the eigenvector centrality of the adopters. We believe that the size of the monster is a good indicator for whether or not a product is going to “take off”.

A Handset-centric View of Smartphone Application Use

Studying the use of applications on smart phones is important for developers, handset designers and network operators. We conducted a study on Android devices by installing an instrumentation application, Device Analyzer, on participants’ handsets. Over a 4 month period we collected 10.9 billion records from 674 different users. In this paper we describe how to use the research study features of Device Analyzer to control participant selection and to access information (with consent) that is withheld for privacy reasons from the main dataset. We describe our data processing architecture and the steps required to preformat and analyse the data. Our data contains 3329 distinct applications (from the Google Play store) but despite this, on average, a user makes use of only 8 unique applications in a week. Almost 100% of our users make use of some email application on their phone. Fewer users (85%) made use of the Facebook application but 4–5 times more frequently than for email with sessions lasting almost twice as long. We also investigated whether different applications have correlated usage using a network analysis and a principal component analysis. We see that application usage tends to correlate by vendor more than by activity. This is potentially due to vendors integrating or cross-promoting services between applications.

Use of mobile phone data for analysis of number of train travellers

Abstract Several studies have pointed to the difficulties of obtaining good data on train ridership. There are at least two challenges regarding these data. First, train operators consider such data confidential business information, especially in high resolution. Second, the data that actually are available vary in quality and coverage. This paper studies mobile phone data as an alternative measure to obtain data about train ridership. Handset counts were obtained from one telecom operator for selected mobile phone base stations and compared with timetable data and APC. The selected base stations are located so that it is likely that a large share of the mobile phone traffic is generated by train passengers. The number of units connected to a base station is found to correspond relatively well with the trains that pass close to the base stations. A ratio between the handset count and APC data appear as promising in utilizing handset count to calculate train ridership, with ratios around one in the rush hours. We discuss preliminary results as well as methodological and technical challenges. To make sure that we do not violate privacy concerns, the data used in the study have been approved by personal privacy representatives.