Aneesh Sharma

GraphJet: real-time content recommendations at twitter

This paper presents GraphJet, a new graph-based system for generating content recommendations at Twitter. As motivation, we trace the evolution of our formulation and approach to the graph recommendation problem, embodied in successive generations of systems. Two trends can be identified: supplementing batch with real-time processing and a broadening of the scope of recommendations from users to content. Both of these trends come together in Graph-Jet, an in-memory graph processing engine that maintains a real-time bipartite interaction graph between users and tweets. The storage engine implements a simple API, but one that is sufficiently expressive to support a range of recommendation algorithms based on random walks that we have refined over the years. Similar to Cassovary, a previous graph recommendation engine developed at Twitter, GraphJet assumes that the entire graph can be held in memory on a single server. The system organizes the interaction graph into temporally-partitioned index segments that hold adjacency lists. GraphJet is able to support rapid ingestion of edges while concurrently serving lookup queries through a combination of compact edge encoding and a dynamic memory allocation scheme that exploits power-law characteristics of the graph. Each GraphJet server ingests up to one million graph edges per second, and in steady state, computes up to 500 recommendations per second, which translates into several million edge read operations per second.

The Who-To-Follow System at Twitter: Strategy, Algorithms, and Revenue Impact

The who-to-follow system at Twitter is an algorithmic data product that recommends accounts for Twitter users to follow. Building the system involved algorithmic, analytics, operational, and experimental challenges; operations research and analytics techniques played a key role in resolving these challenges. This product has had significant direct impact on Twitters growth and the quality of its user engagement, and has also been a major driver of revenue. More than one-eighth of all new connections on the Twitter network are a direct result of this system, and a substantial majority of Twitters revenue comes from its promoted products, for which this system was a foundation. To place this contribution into perspective, Twitter is now a publicly traded company with a market capitalization of more than

WTF: the who to follow service at Twitter

30 billion, projected annual revenue of close to

Information network or social network?: the structure of the twitter follow graph

1 billion, and more than 240 million active users.