Niels Zeilemaker

Tribler: P2P media search and sharing

Tribler is an open-source software project facilitating search, streaming and sharing content using P2P technology. Over 800,000 people have used Tribler since the project started in 2005. The Tribler P2P core supports BitTorrent-compatible downloading, video on demand and live streaming. Aside from a regular desktop GUI that runs on multiple OSes, it can be installed as a browser plug-in, currently used by Wikipedia. Aditionally, it runs on a 450~MHz processor, showcasing future TV support. We continuously work on extensions and test out novel research ideas within our user base, resulting in sub-second content search, a reputation system for rewarding upload, and channels for content publishing and spam prevention. Presently, 1200 channels have been created, enabling rich multimedia communities without requiring any server.

Open source column: Tribler: P2P search, share and stream

Six years ago, we created a new open source P2P file sharing program called Tribler. During this time over one million users have used it, and three generations of Ph.D. students tested their algorithms in the real world.

Large-scale message synchronization in challenged networks

In this paper we introduce Dispersy, a message synchronization platform capable of running inside a challenged network. Dispersy uses Bloom filters to let peers advertise their local state and receive missing messages. However, in contrast to previous work, the efficiency and effectiveness of our design allows for deployment in networks of unprecedented scale. We show, through extensive experimental evidence that peers have no difficulties in synchronizing over 100,000 messages. We integrate in Dispersy a NAT traversal technique able to puncture 77% of NAT-firewalls. Not puncturing these firewalls would prevent up to 64% of peers from receiving any synchronization requests. Implementing a NAT traversal technique proved essential when we used Dispersy to extend the functionalities of a BitTorrent client. To date, over 350,000 users used our modified BitTorrent client which included Dispersy, synchronizing in overlays consisting of more than 500,000 messages. We emulate an overlay consisting of 1000 unmodified Dispersy peers in order to show the propagation and synchronization speed, bandwidth requirements, churn resilience, and overall throughput of Dispersy. Dispersy is able to synchronize a single message to all peers within 19 synchronization steps, withstand an average session-time of 30 seconds, and achieve an average throughput of 10,000 messages/s.

Building a privacy-preserving semantic overlay for Peer-to-Peer networks

Searching a Peer-to-Peer (P2P) network without using a central index has been widely investigated but proved to be very difficult. Various strategies have been proposed, however no practical solution to date also addresses privacy concerns.

Tribler: Search and stream

Tribler is an open-source software project facilitating searching, streaming and sharing content using P2P technology that has been used by over 800 000 people. The Tribler P2P core supports BitTorrent-compatible downloading, video on demand and live streaming. We continuously work on extensions and test out novel research ideas within our user base, resulting in sub-second content search, a reputation system for rewarding upload, and channels for content publishing and spam prevention.

4P: Performant private peer-to-peer file sharing

In recent years fully decentralized file sharing systems were developed aimed at improving anonymity among their users. These systems provide typical file sharing features such as searching for and downloading files. However, elaborate schemes originally aimed at improving anonymity cause partial keyword matching to be virtually impossible, or introduce a substantial bandwidth overhead. In this paper we introduce 4P, a system that provides users with anonymous search on top of a semantic overlay. The semantic overlay allows users to efficiently locate files using partial keyword matching, without having to resort to an expensive flooding operation. Included into 4P are a number of privacy enhancing features such as probabilistic query forwarding, path uncertainty, caching, and encrypted links. Moreover, we integrate a content retrieval channel into our protocol allowing users to start downloading a file from multiple sources immediately without requiring all intermediate nodes to cache a complete copy. Using a trace-based dataset, we mimic a real-world query workload and show the cost and performance of search using six overlay configurations, comparing random, semantic, Gnutella, RetroShare, and OneSwarm to 4P. The state-of-the-art flooding based alternatives required approximately 10,000 messages to be sent per query, in contrast 4P only required 313. Showing that while flooding can achieve a high recall (more than 85% in our experiments) it is prohibitively expensive. With 4P we achieve a recall of 76% at a considerable reduction in messages sent.

100 Million DHT replies

Crawling a DHT allows researchers to monitor the behaviour of peers, determine their geographic location, etc. However, it has always been an error-prone process as it is not feasible to capture a full snapshot of the Mainline DHT in a timely manner. Therefore, researchers have developed approximation methods which can run on subsets of the DHT and extrapolate those in order to reason on the size and structure of the complete DHT. However, in this paper we introduce a new method of collecting information on peers connected to a DHT. We exploit the caches present at bootstrap servers to collect information on peers which are currently connected. Originally added to the bootstrap servers in order to be able to withstand more than 20,000 requests for peers each second, we now use the same mechanism as peers bootstrapping into the DHT to discover more than 20 Million peers in less than 2 hours. Using the bootstrap servers, we discover more than twice as many peers as BitMon, which crawls a subset of the DHT and then extrapolates. Moreover, in contrast to related work which often require highly customized/tuned BitTorrent clients, our script consists of 90 lines of Python code and runs on a simple laptop.