Gunnar Kreitz

Spotify -- Large Scale, Low Latency, P2P Music-on-Demand Streaming

Spotify is a music streaming service offering low-latency access to a library of over 8 million music tracks. Streaming is performed by a combination of client-server access and a peer-to-peer protocol. In this paper, we give an overview of the protocol and peer-to-peer architecture used and provide measurements of service performance and user behavior. The service currently has a user base of over 7 million and has been available in six European countries since October 2008. Data collected indicates that the combination of the client-server and peer-to-peer paradigms can be applied to music streaming with good results. In particular, 8.8% of music data played comes from Spotifys servers while the median playback latency is only 265 ms (including cached tracks). We also discuss the user access patterns observed and how the peer-to-peer network affects the access patterns as they reach the server.

Understanding user behavior in Spotify

Spotify is a peer-assisted music streaming service that has gained worldwide popularity in the past few years. Until now, little has been published about user behavior in such services. In this paper, we study the user behavior in Spotify by analyzing a massive dataset collected between 2010 and 2011. Firstly, we investigate the system dynamics including session arrival patterns, playback arrival patterns, and daily variation of session length. Secondly, we analyze individual user behavior on both multiple and single devices. Our analysis reveals the favorite times of day for Spotify users. We also show the correlations between both the length and the downtime of successive user sessions on single devices. In particular, we conduct the first analysis of the device-switching behavior of a massive user base.

Measurements on the spotify peer-assisted music-on-demand streaming system

Spotify is a streaming service offering low-latency access to a large library of music. Streaming is performed by a combination of client-server access and a peer-to-peer protocol. The service currently has a user base of over 10 million and is available in seven European countries. We provide a background on the Spotify protocol with emphasis on the formation of the peer-to-peer overlay. Using measurement data collected over a week by instrumenting Spotify clients, we analyze general network properties such as the correspondence between individual user accounts and the number of IP addresses they connect from and the prevalence of Network Address Translation devices (NATs). We also discuss the performance of one of the two peer discovery mechanisms used by Spotify.

Access control in decentralized online social networks: Applying a policy-hiding cryptographic scheme and evaluating its performance

Privacy concerns in online social networking services have prompted a number of proposals for decentralized online social networks (DOSN) that remove the central provider and aim at giving the users control over their data and who can access it. This is usually done by cryptographic means. Existing DOSNs use cryptographic primitives that hide the data but reveal the access policies. At the same time, there are privacy-preserving variants of these cryptographic primitives that do not reveal access policies. They are, however, not suitable for usage in the DOSN context because of performance or storage constraints. A DOSN needs to achieve both privacy and performance to be useful. We analyze predicate encryption (PE) and adapt it to the DOSN context. We propose a univariate polynomial construction for access policies in PE that drastically increases performance of the scheme but leaks some part of the access policy to users with access rights. We utilize Bloom filters as a means of decreasing decryption time and indicate objects that can be decrypted by a particular user. We evaluate the performance of the adapted scheme in the concrete scenario of a news feed. Our PE scheme is best suited for encrypting for groups or small sets of separate identities.

Cost-Effective Resource Allocation for Deploying Pub/Sub on Cloud

Publish/subscribe (pub/sub) is a popular communication paradigm in the design of large-scale distributed systems. A fundamental challenge in deploying pub/sub systems on a data center or a cloud infrastructure is efficient and cost-effective resource allocation that would allow delivery of notifications to all subscribers. In this paper, we provide answers to the following three fundamental questions: Given a pub/sub workload, (1) what is the minimum amount of resources needed to satisfy all the subscribers, (2) what is a cost-effective way to allocate resources for the given workload, and (3) what is the cost of hosting it on a public Infrastructure-as-a-Service (IaaS) provider like Amazon EC2. To answer these questions, we formulate a problem coined Minimum Cost Subscriber Satisfaction (MCSS). We prove MCSS to be NP-hard and provide an efficient heuristic solution based on a combination of optimizations. We evaluate the solution experimentally using real traces from Spotify and Twitter along with a pricing model from Amazon. We show the impact of each optimization using a naive solution as the baseline. Using a variety of practical scenarios for each dataset, we also show that our solution scales well for millions of subscribers and runs fast.

Lower bounds for subset cover based broadcast encryption

In this paper, we prove lower bounds for a large class of Subset Cover schemes (including all existing schemes based on pseudorandom sequence generators). In particular, we show that - For small r, bandwidth is Ω(r) - For some r, bandwidth is Ω(n/log(s)) - For large r, bandwidth is n - r where n is the number of users, r is the number of revoked users, and s is the space required per user. These bounds are all tight in the sense that they match known constructions up to small constants.

Maximizing the Number of Satisfied Subscribers in Pub/Sub Systems Under Capacity Constraints

Publish/subscribe (pub/sub) is a popular communication paradigm in the design of large-scale distributed systems. A provider of a pub/sub service (whether centralized, peer-assisted, or based on a federated organization of cooperatively managed servers) commonly faces a fundamental challenge: given limited resources, how to maximize the satisfaction of subscribers? We provide, to the best of our knowledge, the first formal treatment of this problem by introducing two metrics that capture subscriber satisfaction in the presence of limited resources. This allows us to formulate matters as two new flavors of maximum coverage optimization problems. Unfortunately, both variants of the problem prove to be NP-hard. By subsequently providing formal approximation bounds and heuristics, we show, however, that efficient approximations can be attained. We validate our approach using real-world traces from Spotify and show that our solutions can be executed periodically in real-time in order to adapt to workload variations.

Game-Theoretic Mechanisms to Increase Data Availability in Decentralized Storage Systems

In a decentralized storage system, agents replicate each other’s data to increase availability. Compared to organizationally centralized solutions, such as cloud storage, a decentralized storage system requires less trust in the provider and may result in smaller monetary costs. Our system is based on reciprocal storage contracts that allow the agents to adopt to changes in their replication partners’ availability (by dropping inefficient contracts and forming new contracts with other partners). The data availability provided by the system is a function of the participating agents’ availability. However, a straightforward system in which agents’ matching is decentralized uses the given agent availability inefficiently. As agents are autonomous, the highly available agents form cliques replicating data between each other, which makes the system too hostile for the weakly available newcomers. In contrast, a centralized, equitable matching is not incentive compatible: it does not reward users for keeping their software running. We solve this dilemma by a mixed solution: an “adoption” mechanism in which highly available agents donate some replication space, which in turn is used to help the worst-off agents. We show that the adoption motivates agents to increase their availability (is incentive-compatible), but also that it is sufficient for acceptable data availability for weakly-available agents.

Passwords in peer-to-peer

One of the differences between typical peer-to-peer (P2P) and client-server systems is the existence of user accounts. While many P2P applications, like public file sharing, are anonymous, more complex services such as decentralized online social networks require user authentication. In these, the common approach to P2P authentication builds on the possession of cryptographic keys. A drawback with that approach is usability when users access the system from multiple devices, an increasingly common scenario. In this work, we present a scheme to support logins based on users knowing a username-password pair. We use passwords, as they are the most common authentication mechanism in services on the Internet today, ensuring strong user familiarity. In addition to password logins, we also present supporting protocols to provide functionality related to password logins, such as resetting a forgotten password via e-mail or security questions. Together, these allow P2P systems to emulate centralized password logins. The results of our performance evaluation indicate that incurred delays are well within acceptable bounds.

A zero-one law for secure multi-party computation with ternary outputs

There are protocols to privately evaluate any function in the passive (honest-but-curious) setting assuming that the honest nodes are in majority. For some specific functions, protocols are known which remain secure even without an honest majority. The seminal work by Chor and Kushilevitz [7] gave a complete characterization of Boolean functions, showing that each Boolean function either requires an honest majority, or is such that it can be privately evaluated regardless of the number of colluding nodes. The problem of discovering the threshold for secure evaluation of more general functions remains an open problem. Towards a resolution, we provide a complete characterization of the security threshold for functions with three different outputs. Surprisingly, the zero-one law for Boolean functions extends to Z3, meaning that each function with range Z3 either requires honest majority or tolerates up to n colluding nodes.