Nima Sarshar

Percolation search in power law networks: making unstructured peer-to-peer networks scalable

We introduce a scalable searching protocol for locating and retrieving content in random networks with power-law (PL) and heavy-tailed degree distributions. The proposed algorithm is capable of finding any content in the network with probability one in time O(logN), with a total traffic that provably scales sub-linearly with the network size, N. Unlike other proposed solutions, there is no need to assume that the network has multiple copies of contents; the protocol finds all contents reliably, even if every node in the network starts with a unique content. The scaling behavior of the size of the giant connected component of a random graph with heavy tailed degree distributions under bond percolation is at the heart of our results. The percolation search algorithm can be directly applied to make unstructured peer-to-peer (P2P) networks, such as Gnutella, Limewire and other file-sharing systems (which naturally display heavy-tailed degree distributions and scale-free network structures), scalable. For example, simulations of the protocol on the limewire crawl number 5 network, consisting of over 65,000 links and 10,000 nodes, shows that even for this snapshot network, the traffic can be reduced by a factor of at least 100, and yet achieve a hit-rate greater than 90%.

Scale-free and stable structures in complex ad hoc networks

Unlike the well-studied models of growing networks, where the dominant dynamics consist of insertions of new nodes and connections and rewiring of existing links, we study ad hoc networks, where one also has to contend with rapid and random deletions of existing nodes (and, hence, the associated links). We first show that dynamics based only on the well-known preferential attachments of new nodes do not lead to a sufficiently heavy-tailed degree distribution in ad hoc networks. In particular, the magnitude of the power-law exponent increases rapidly (from 3) with the deletion rate, becoming infinity in the limit of equal insertion and deletion rates. We then introduce a local and universal compensatory rewiring dynamic, and show that even in the limit of equal insertion and deletion rates true scale-free structures emerge, where the degree distributions obey a power law with a tunable exponent, which can be made arbitrarily close to 2. The dynamics reported in this paper can be used to craft protocols for designing highly dynamic peer-to-peer networks and also to account for the power-law exponents observed in existing popular services.

Experience versus talent shapes the structure of the Web

We use sequential large-scale crawl data to empirically investigate and validate the dynamics that underlie the evolution of the structure of the web. We find that the overall structure of the web is defined by an intricate interplay between experience or entitlement of the pages (as measured by the number of inbound hyperlinks a page already has), inherent talent or fitness of the pages (as measured by the likelihood that someone visiting the page would give a hyperlink to it), and the continual high rates of birth and death of pages on the web. We find that the web is conservative in judging talent and the overall fitness distribution is exponential, showing low variability. The small variance in talent, however, is enough to lead to experience distributions with high variance: The preferential attachment mechanism amplifies these small biases and leads to heavy-tailed power-law (PL) inbound degree distributions over all pages, as well as over pages that are of the same age. The balancing act between experience and talent on the web allows newly introduced pages with novel and interesting content to grow quickly and surpass older pages. In this regard, it is much like what we observe in high-mobility and meritocratic societies: People with entitlement continue to have access to the best resources, but there is just enough screening for fitness that allows for talented winners to emerge and join the ranks of the leaders. Finally, we show that the fitness estimates have potential practical applications in ranking query results.

Rainbow Network Flow of Multiple Description Codes

This paper is an enquiry into the interaction between multiple description coding (MDC) and network routing. We are mainly concerned with rate-distortion optimized network flow of a multiple description (MD) source from multiple servers to multiple sinks. We aim at maximizing a collective metric of the quality of source reconstruction at all sinks, by optimally routing the MD source streams from the server nodes to the sinks. This problem turns out to be very different from conventional maximum network flow. The objective function involves not only the flow volume but also the diversity of the flow contents (i.e., distinction of descriptions), hence, the term rainbow network flow (RNF). For a general network topology, a general fidelity function, and an arbitrary distribution of MDC descriptions on the servers, we prove the RNF problem to be Max-SNP-hard. However, the problem becomes tractable in many practical scenarios, such as when MDC is balanced with descriptions of the same length and importance, when all source nodes have the complete set of MDC descriptions, and when the network topology is a tree or has only one sink. Polynomial-time RNF algorithms are developed for these cases.

Multiple power-law structures in heterogeneous complex networks.

This paper develops a framework for analyzing and designing dynamic networks comprising different classes of nodes that coexist and interact in one shared environment. We consider {\em ad hoc} (i.e., nodes can leave the network unannounced, and no node has any global knowledge about the class identities of other nodes) {\em preferentially grown networks}, where different classes of nodes are characterized by different sets of local parameters used in the stochastic dynamics that all nodes in the network execute. We show that multiple scale-free structures, one within each class of nodes, and with tunable power-law exponents (as determined by the sets of parameters characterizing each class) emerge naturally in our model. Moreover, the coexistence of the scale-free structures of the different classes of nodes can be captured by succinct phase diagrams, which show a rich set of structures, including stable regions where different classes coexist in heavy-tailed and light-tailed states, and sharp phase transitions. Finally, we show how the dynamics formulated in this paper will serve as an essential part of {\em ad-hoc networking protocols}, which can lead to the formation of robust and efficiently searchable networks (including, the well-known Peer-To-Peer (P2P) networks) even under very dynamic conditions.

Scalable percolation search on complex networks

We introduce a scalable searching protocol for locating and retrieving content in random networks with heavy-tailed and in particular power-law (PL) degree distributions. The proposed algorithm is capable of finding any content in the network with probability one in time O(log N), with a total traffic that provably scales sub-linearly with the network size, N. Unlike other proposed solutions, there is no need to assume that the network has multiple copies of contents; the protocol finds all contents reliably, even if every node in the network starts with a unique content. The scaling behavior of the size of the giant connected component of a random graph with heavy-tailed degree distributions under bond percolation is at the heart of our results. The percolation search algorithm can be directly applied to make unstructured peer-to-peer (P2P) networks, such as Gnutella, Limewire and other file-sharing systems (which naturally display heavy-tailed degree distributions and approximate scale-free network structures), scalable. For example, simulations of the protocol on the limewire crawl number 5 network [Ripeanu et al., Mapping the Gnutella network: properties of large-scale peer-to-peer systems and implications for system design, IEEE Internet Comput. J. 6 (1) (2002)], consisting of over 65,000 links and 10,000 nodes, shows that even for this snapshot network, the traffic can be reduced by a factor of at least 100, and yet achieve a hit-rate greater than 90%.

On fair and optimal multi-source IP-multicast

We investigate the problem of maximizing multicast throughput under a fairness constraint. Multiple server nodes wish to communicate to their intended set of client nodes over a shared network infrastructure. Our goal is to devise distributed algorithms to construct multicast sessions, one for each server node, such that (a) the network infrastructure is optimally utilized and (b) the network resources are fairly distributed between multicast sessions, i.e., no individual session claims more than a prescribed share of the network bandwidth resources. We are particularly interested in multi-tree multicast strategies in which every multicast session may contain many multicast trees. We show how the use of multiple trees increases network throughput and the load distribution in the network. We propose a class of round-robin algorithms that are based on successive selection of multicast trees for each multicast session, in a loosely cooperative, yet distributed fashion. Our best algorithm, the Cooperative Shortest Path Tree Packing (CSPTP) algorithm, performs well in a variety of scenarios, ranging from very sparse to dense applications. Through extensive simulations on random networks, we compare the performance of our algorithms with those commonly used in IP-multicast as well as theoretical upper bounds derived from network coding formulations. We show that the CSPTP can improve the throughput, and often achieves about 90% of the theoretical upper bound.

Rate-distortion optimized multimedia communication in networks

Large-scale, widespread distribution of high definition multimedia contents using IP networks is extremely resource intensive. Service providers have to employ an expensive network of servers, routers, link infrastructures and set-top boxes to accommodate the generated traffic. The goal in this paper is to develop network-aware media communication solutions that help service providers to efficiently utilize their deployed network infrastructures for media delivery. In particular, we investigate the following fundamental problem: given a fixed network infrastructure, what is the best strategy to multicast multiple multimedia contents from a set of server nodes to a set of clients, to realize the best reconstruction quality at the client nodes? We use rate-distortion to formalize the notion of media quality and to formulate the corresponding optimization problem. We show that current approaches in which multimedia compression and network delivery mechanisms are designed separately are inherently suboptimal. Thus, better utilization of network resources requires a joint consideration of media compression and network delivery. We develop one such approach based on optimized delivery of balanced Multiple Description Codes (MDC), in which the MDC itself is also optimized with respect to the optimized delivery strategy. Simulation results are reported, verifying that our solution can significantly outperform existing, layered, solutions. As a byproduct, our solutions introduces a fundamentally different use of MDC. Up until now, MDC has been adopted to combat losses, mostly in packet lossy networks. We show that MDC is an efficient tool for network communication, even in error-free networks. In particular MDC, when properly duplicated at routers, can exploit the rich topological structures in networks to maximize the utilization of the network resources, beyond conventional coding techniques.

Video Processing Techniques for Assisted CCTV Inspection and Condition Rating of Sewers

This chapter describes the development of a software system to semi-automatically extract historical condition data information from archived sewer inspection …

Low latency wireless ad hoc networking: power and bandwidth challenges and a solution

This paper is concerned with the scaling of the number of relay nodes (i.e., hops) individual messages have to transit through in a large-scale wireless ad hoc network (WANET); we call this hop-count as network latency (NL). A large network latency affects all aspects of data communication in a WANET, including an increase in delay, packet loss, and the power needed to process and store messages in nodes lying on the relay path. We consider network management and data routing challenges in WANETs with scalable network latency, e.g., when NL increases only polylogarithmically in the network size. On the physical side, reducing network latency imposes a significantly higher power and bandwidth demand on nodes, and are captured in a set of new bounds derived in this paper. On the protocol front, designing distributed routing protocols that can guarantee the delivery of data packets within a scalable number of hops is a challenging task. To solve this, we introduce multi-resolution randomized hierarchy (MRRH), a novel power and bandwidth efficient WANET protocol with scalable network latency. MRRH uses a randomized algorithm for building and maintaining a random hierarchical network topology, which together with the proposed routing algorithm, can guarantee efficient delivery of data packets in the wireless network. For a network of size N, MRRH can provide an average latency of only O(log3 N). The power consumption and bandwidth requirements of MRRH are shown to be nearly optimal for the latency it provides. Therefore, MRRH is a provably efficient candidate for truly large-scale wireless ad hoc networking.