Shah Asaduzzaman

CliqueStream: An Efficient and Fault-Resilient Live Streaming Network on a Clustered Peer-to-Peer Overlay

Several overlay-based live multimedia streaming platforms have been proposed in the recent peer-to-peer streaming literature. In most of the cases, the overlay neighbors are chosen randomly for robustness of the overlay. However, this causes nodes that are distant in terms of proximity in the underlying physical network to become neighbors, and thus data travels unnecessary distances before reaching the destination. For efficiency of bulk data transmission like multimedia streaming, the overlay neighborhood should resemble the proximity in the underlying network. In this paper, we exploit the proximity and redundancy properties of a recently proposed clique-based clustered overlay network, named eQuus, to build efficient as well as robust overlays for multimedia stream dissemination. To combine the efficiency of content pushing over tree structured overlays and the robustness of data-driven mesh overlays, higher capacity stable nodes are organized in tree structure to carry the long haul traffic and less stable nodes with intermittent presence are organized in localized meshes. The overlay construction and fault-recovery procedures are explained in details. Simulation study demonstrates the good locality properties of the platform. The outage time and control overhead induced by the failure recovery mechanism are minimal as demonstrated by the analysis.

Utilizing unreliable public resources for higher profit and better SLA compliance in computing utilities

Computing utilities are emerging as an important part of the infrastructure for outsourcing computer services. Fundamental to outsourcing is the notion of quality of service, which is defined by service level agreements (SLAs) between the computing utilities and clients. One of the major objectives of computing utilities is to maximize their net profit while maintaining customer loyalty. To achieve this objective, the computing utilities should meet or exceed their SLA constraints most of the time. Defining the SLAs conservatively might be one way of easily achieving these goals. However, by tuning the SLA parameters conservatively the computing utility might under utilize its resources with a resultant loss of revenue. Therefore, we can see two main issues with SLA management: designing SLAs competitively so that expected revenue for the computing utility is maximized and maintaining the operating conditions such that SLAs are satisfied with very high probability. In this paper, we show that inducting unreliable public resources into a computing utility enables more competitive SLAs while maintaining higher levels of run time compliances as well as maximizing profit. Our scheduling algorithms assume that idle cycles from public resources are available in plenty, therefore, the performance gains do not incur any additional financial cost. However, there is communication overhead when public resources from a wide area network is included. This overhead is kept to the minimum by enabling the scheduler work without any monitoring on the public resources.

Proposal for st-Routing Protocol

A routing protocol chooses one of the several paths (routes) from a source node to a destination node in the computer network, to send a packet of information. In this paper, we propose a new routing protocol, which we call st-routing protocol, based on st-numbering of a graph. The protocol fits well in noisy environments where robustness of routing using alternative paths is a major issue. The proposed routing protocol provides a systematic way to retry alternative paths without generating any duplicate packets. The protocol works for only those networks that can be represented by biconnected graphs.

Strategies to create platforms for differentiated services from dedicated and opportunistic resources

This paper is proposing a new platform for implementing services in future service oriented architectures. The basic premise of our proposal is that by combining the large volume of uncontracted resources with small clusters of dedicated resources, we can dramatically reduce the amount of dedicated resources while the goodput provided by the overall system remains at a high level. This paper presents particular strategies for implementing this idea for a particular class of applications. We performed very detailed simulations on synthetic and real traces to evaluate the performance of the proposed strategies. Our findings on compute-intensive applications show that preemptive reallocation of resources is necessary for assured services. The proposed preemption-based scheduling heuristic can significantly improve utilization of the dedicated resources by opportunistically offloading the peak loads on uncontracted resources, while keeping the service quality virtually unaffected.

CliqueStream: Creating an efficient and resilient transport overlay for peer-to-peer live streaming using a clustered DHT

Several overlay-based live multimedia streaming platforms have been proposed in the recent peer-to-peer streaming literature. In most of the cases, the overlay neighbors are chosen randomly for robustness of the overlay. However, this causes nodes that are distant in terms of proximity in the underlying physical network to become neighbors, and thus data travels unnecessary distances before reaching the destination. For efficiency of bulk data transmission like multimedia streaming, the overlay neighborhood should resemble the proximity in the underlying network. In this paper, we exploit the proximity and redundancy properties of a recently proposed clique-based clustered overlay network, named eQuus, to build efficient as well as robust transport overlays for multimedia streaming. To combine the efficiency of content pushing over tree structured overlays and the robustness of data-driven mesh overlays, higher capacity stable nodes are organized in tree structure to carry the long haul traffic and less stable nodes with intermittent presence are organized in localized meshes. The overlay construction and fault-recovery procedures are explained in details. Simulation study demonstrates the good locality properties of the platform. The outage time and control overhead induced by the failure recovery mechanism are minimal as demonstrated by the analysis.

Towards a quality of service aware public computing utility

This work describes a design for a quality of service aware public computing utility (PCU). The goal of the PCU is to utilize the idle capacity of the shared public resources and augment the capacity with dedicated resources as necessary, to provide high quality of service to the clients at the least cost. Our PCU design combines peer-to-peer (P2P) and grid computing ideas in a novel manner to construct a utility-based computing environment. In This work, we present the overall architecture and describe two major components: a P2P overlay substrate for connecting the resources in a global network and a community-based decentralized resource management system.

Decentralized management of bi-modal network resources in a distributed stream processing platform

This paper presents resource management techniques for allocating communication and computational resources in a distributed stream processing platform. The platform is designed to exploit the synergy of two classes of network connections -- dedicated and opportunistic. Previous studies we conducted have demonstrated the benefits of such bi-modal resource organization that combines small pools of dedicated computers with a very large pool of opportunistic computing capacities of idle computers to serve high throughput computing applications. This paper extends the idea of bi-modal resource organization into the management of communication resources. Since distributed stream processing applications demand large volume of data transmission between processing sites at a consistent rate, adequate control over the network resources is important to assure a steady flow of processing. The system model used in this paper is a platform where stream processing servers at distributed sites are interconnected with a combination of dedicated and opportunistic communication links. Two pertinent resource allocation problems are analyzed in details and solved using decentralized algorithms. One is the mapping of the stream processing tasks on the processing and the communication resources. The other is the adaptive re-allocation of the opportunistic communication links due to the variations in their capacities. Overall optimization goal is higher task throughput and better utilization of the expensive dedicated links. The evaluation demonstrates that the algorithms are able to exploit the synergy of bi-modal communication links towards achieving the optimization goals.

A locality preserving routing overlay using geographic coordinates

This paper presents a design for a general locality preserving routing overlay network based on geographic coordinates of nodes. For efficient use of network resources, it is important to follow some locality principles while routing bulk multimedia content. The proposed overlay network creates the interconnection among large number of highly dynamic nodes in a completely decentralized manner, based on adaptive hierarchical partitioning of the geographical space. A major benefit of using coarse grain geographic coordinates is that they are available at off-the-shelf databases and thus do not incur additional measurement overhead. The performance of the proposed geography based routing overlay, in terms of two locality properties - stretch and route-convergence, is evaluated by simulation studies. Real topology data of 65 ISPs with presence in 534 cities across the world, collected by the RocketFuel project, is used as the basis of the simulated network. The performance results are compared with corresponding results obtained from a simulated Pastry overlay constructed on the same underlying network. The performance results show that proposed overlay has significantly better stretch and route-convergence characteristics compared to a regular Pastry network. The locality properties of the proposed routing overlay are almost equivalent to those of an optimal Pastry network where all nodes use the most proximal neighbor for each routing table row. Thus the proposed geographical routing network provides good locality properties without the overhead of proximity neighbor selection. Moreover, the proposed scheme facilitates certain geographical search applications that are difficult to realize in a Pastry overlay.

Towards a Decentralized Algorithm for Mapping Network and Computational Resources for Distributed Data-Flow Computations

Several high-throughput distributed data-processing applications require multi-hop processing of streams of data. These applications include continual processing on data streams originating from a network of sensors, composing a multimedia stream through embedding several component streams originating from different locations, etc. These data-flow computing applications require multiple processing nodes interconnected according to the data-flow topology of the application, for on-stream processing of the data. Since the applications usually sustain for a long period, it is important to optimally map the component computations and communications on the nodes and links in the network, fulfilling the capacity constraints and optimizing some quality metric such as end-to-end latency. The mapping problem is unfortunately NP-complete and heuristics have been previously proposed to compute the approximate solution in a centralized way. However, because of the dynamicity of the network, it is practically impossible to aggregate the correct state of the whole network in a single node. In this paper, we present a distributed algorithm for optimal mapping of the components of the data flow applications. We propose several heuristics to minimize the message complexity of the algorithm while maintaining the quality of the solution.

Using Dedicated and Opportunistic Networks in Synergy for a Cost-Effective Distributed Stream Processing Platform

This paper presents a case for exploiting the synergy of dedicated and opportunistic network resources in a distributed hosting platform for data stream processing applications. Our previous studies have demonstrated the benefits of combining dedicated reliable resources with opportunistic resources in case of high-throughput computing applications, where timely allocation of the processing units is the primary concern. Since distributed stream processing applications demand large volume of data transmission between the processing sites at a consistent rate, adequate control over the network resources is important here to assure a steady flow of processing. In this paper, we propose a system model for the hybrid hosting platform where stream processing servers installed at distributed sites are interconnected with a combination of dedicated links and public Internet. Decentralized algorithms have been developed for allocation of the two classes of network resources among the competing tasks with an objective towards higher task throughput and better utilization of expensive dedicated resources. Results from extensive simulation study show that with proper management, systems exploiting the synergy of dedicated and opportunistic resources yield considerably higher task throughput and thus, higher return on investment over the systems solely using expensive dedicated resources.