Joseph S. Kong

Experience versus talent shapes the structure of the Web

We use sequential large-scale crawl data to empirically investigate and validate the dynamics that underlie the evolution of the structure of the web. We find that the overall structure of the web is defined by an intricate interplay between experience or entitlement of the pages (as measured by the number of inbound hyperlinks a page already has), inherent talent or fitness of the pages (as measured by the likelihood that someone visiting the page would give a hyperlink to it), and the continual high rates of birth and death of pages on the web. We find that the web is conservative in judging talent and the overall fitness distribution is exponential, showing low variability. The small variance in talent, however, is enough to lead to experience distributions with high variance: The preferential attachment mechanism amplifies these small biases and leads to heavy-tailed power-law (PL) inbound degree distributions over all pages, as well as over pages that are of the same age. The balancing act between experience and talent on the web allows newly introduced pages with novel and interesting content to grow quickly and surpass older pages. In this regard, it is much like what we observe in high-mobility and meritocratic societies: People with entitlement continue to have access to the best resources, but there is just enough screening for fitness that allows for talented winners to emerge and join the ranks of the leaders. Finally, we show that the fitness estimates have potential practical applications in ranking query results.

A General Framework for Scalability and Performance Analysis of DHT Routing Systems

In recent years, many DHT-based P2P systems have been proposed, analyzed, and certain deployments have reached a global scale with nearly one million nodes. One is thus faced with the question of which particular DHT system to choose, and whether some are inherently more robust and scalable. Toward developing such a comparative framework, we present the reachable component method (RCM) for analyzing the performance of different DHT routing systems subject to random failures. We apply RCM to five DHT systems and obtain analytical expressions that characterize their routability as a continuous function of system size and node failure probability. An important consequence is that in the large-network limit, the routability of certain DHT systems go to zero for any non-zero probability of node failure. These DHT routing algorithms are therefore unscalable, while some others, including Kademlia, which powers the popular eDonkey P2P system, are found to be scalable

Resilience of structured P2P systems under churn: The reachable component method

Users in a peer-to-peer (P2P) system join and leave the network in a continuous manner. Understanding the resilience properties of P2P systems under high rate of node churn becomes important. In this work, we first find that a lifetime-based dynamic churn model for a P2P network that has reached stationarity is reducible to a uniform node failure model. This is a simple yet powerful result that bridges the gap between the complex dynamic churn models and the more tractable uniform failure model. We further develop the reachable component method and derive the routing performance of a wide-range of structured P2P systems under varying rates of churn. We find that the de Bruijn graph based routing systems offer excellent resilience under extremely high rate of node turnovers, followed by a group of routing systems that include CAN, Kademlia, Chord and randomized-Chord. We show that our theoretical predictions agree well with large-scale simulation results. We finish by suggesting methods to further improve the routing performance of dynamic P2P systems in the presence of churn and failures.

Preferential survival in models of complex ad hoc networks

There has been a rich interplay in recent years between (i) empirical investigations of real-world dynamic networks, (ii) analytical modeling of the microscopic mechanisms that drive the emergence of such networks, and (iii) harnessing of these mechanisms to either manipulate existing networks, or engineer new networks for specific tasks. We continue in this vein, and study the deletion phenomenon in the web by the following two different sets of websites (each comprising more than 150,000 pages) over a one-year period. Empirical data show that there is a significant deletion component in the underlying web networks, but the deletion process is not uniform. This motivates us to introduce a new mechanism of preferential survival (PS), where nodes are removed according to the degree-dependent deletion kernel, D(k)∝k−α, with α≥0. We use the mean-field rate equation approach to study a general dynamic model driven by Preferential Attachment (PA), Double PA (DPA), and a tunable PS (i.e., with any α>0), where c nodes (c<1) are deleted per node added to the network, and verify our predictions via large-scale simulations. One of our results shows that, unlike in the case of uniform deletion (i.e., where α=0), the PS kernel when coupled with the standard PA mechanism, can lead to heavy-tailed power-law networks even in the presence of extreme turnover in the network. Moreover, a weak DPA mechanism, coupled with PS, can help to make the network even more heavy-tailed, especially in the limit when deletion and insertion rates are almost equal, and the overall network growth is minimal. The dynamics reported in this work can be used to design and engineer stable ad hoc networks and explain the stability of the power-law exponents observed in real-world networks.

Price of Structured Routing and Its Mitigation in P2P Systems under Churn

We study the resilience of structured peer-to-peer (P2P) networks in the presence of churn. Using the lifetime-based failure assumptions, we first show that a realistic churn model has an equivalent uniform failure model in the steady state. We then determine via percolation analysis and simulation the gap between the size of the connected component and the reachable component of a randomly picked node for symphony and chord. This gap represents the price of structured routing: the size of the set of nodes that are reachable by any unstructured routing method (e.g. broadcast on the unstructured overlay) from a randomly picked surviving node, but are not reachable using structured routing on the structured overlay. As an illustration, for 24- minute average node lifetime with 1-minute average node-search delay, the gap is around 12 thousand nodes or 1.2% in a chord network of around 1 million nodes. We finish by discussing potential techniques to mitigate the price of structured routing.An important security issue in DHT-based structured overlay networks is to provide anonymity to the storage nodes. Compromised routing tables in those DHTs leak information about other nodes in the system and therefore compromise privacy. In this paper, we use information theory to build a model to quantify the information leak from compromised routing tables for a given DHT with certain routing geometry and route table size. Based on this model, we have analyzed and compared how existing DHTs perform in face of anonymity attacks. We found that ring-based routing geometry (Chord) performs the best among the studied DHTs with the same routing complexity when no routing optimizations are used. The analysis of the interaction between routing geometries and recipient anonymity will help improve the design of future DHTs which can achieve a balance between routing efficiency and robustness against information leak.