Zhongtang Cai

SmartPointers: Personalized Scientific Data Portals In Your Hand

The SmartPointer system provides a paradigm for utilizing multiple light-weight client endpoints in a real-time scientific visualization infrastructure. Together, the client and server infrastructure form a new type of data portal for scientific computing. The clients can be used to personalize data for the needs of the individual scientist. This personalization of a shared dataset is designed to allow multiple scientists, each with their laptops or iPaqs to explore the dataset from different angles and with different personalized filters. As an example, iPaq clients can display 2D derived data functions which can be used to dynamically update and annotate the shared data space, which might be visualized separately on a large immersive display such as a CAVE. Measurements are presented for such a system, built upon the ECho middleware system developed at Georgia Tech.

Implementing Diverse Messaging Models with Self-Managing Properties using IFLOW

Implementing self-management is hard, especially when building large scale distributed systems. Publish/subscribe middlewares, scientific visualization and collaboration tools and corporate operational information systems are examples of one class of systems, distributed information flow infrastructures, that could benefit from self management. This paper presents IFLOW, an autonomic middleware for implementing these different distributed systems in a self-managing way. IFLOW reduces different messaging models down to a common information flow abstraction, creates a self-managing implementation of that abstraction and then provides a substrate for building diverse information flow systems. We describe the design and implementation of IFLOW and describe case studies of implementing different messaging models as self-managing systems.

Middleware for enterprise scale data stream management using utility-driven self-adaptive information flows

Abstract We consider enterprise-wide information flows that are responsible for acquiring, processing and delivering operational information across the business units. Middleware that enables such aggregation of data-streams must not only support scalable and efficient self-management to deal with changes in the operating conditions, but should also have an embedded business-sense to appreciate the business critical nature of some updates. In this paper, we present a novel self-adaptation algorithm that has been designed to scale efficiently for thousands of streams and aims to maximize the overall business utility attained from running middleware-based applications. The outcome is that the middleware not only deals with changing network conditions or resource requirements, but also responds appropriately to changes in business policies. An important feature of the algorithm is a hierarchical node-partitioning scheme that decentralizes reconfiguration and suitably localizes its impact. Extensive simulation experiments and benchmarks attained with actual enterprise operational data corroborate this paper’s claims.

IQ-Paths: Predictably High Performance Data Streams Across Dynamic Network Overlays

Overlay networks are a key vehicle for delivering network and processing resources to high performance applications. For shared networks, however, to consistently deliver such resources at desired levels of performance, overlays must be managed at runtime, based on the continuous assessment and prediction of available distributed resources. Data-intensive applications, for example, must assess, predict, and judiciously use available network paths, and dynamically choose alternate or exploit concurrent paths. Otherwise, they cannot sustain the consistent levels of performance required by tasks like remote data visualization, online program steering, and remote access to high end devices. The multiplicity of data streams occurring in complex scientific workflows or in large-scale distributed collaborations exacerbate this problem, particularly when different streams have different performance requirements. This paper presents IQ-Paths, a set of techniques and their middleware realization that implement self-regulating overlay streams for data-intensive distributed applications. Self-regulation is based on (1) the dynamic and continuous assessment of the quality of each overlay path, (2) the use of online network monitoring and statistical analyses that provide probabilistic guarantees about available path bandwidth, loss rate, and RTT, and (3) self-management, via an efficient packet routing and scheduling algorithm that dynamically schedules data packets to different overlay paths in accordance with their available bandwidths. IQ-Paths offers probabilistic guarantees for application-level specifications of stream utility, based on statistical predictions of available network bandwidth. This affords applications with the ability, for instance, to send control or steering data across overlay paths that offer strong guarantees for future bandwidth vs. across less guaranteed paths. Experimental results presented in this paper use IQ-Paths to better handle the different kinds of data produced by two high performance applications and one multimedia application: (1) a data-driven interactive high performance code with user-defined utility requirements, (2) an adaptive overlay version of the popular Grid-FTP application, and (3) a MPEG-4 Fine-Grained Scalable layered video streaming.

IQ-Services: network-aware middleware for interactive large-data applications

IQ‐Services are application‐specific, resource‐aware code modules executed by data transport middleware. They constitute a ‘thin’ layer between application components and the underlying computational and communication resources. This layer implements the data manipulations necessary to permit wide‐area collaborations to proceed smoothly in the presence of dynamic resource variations. IQ‐Services interact with the application and resource layers via dynamic performance attributes, and end‐to‐end implementations of such attributes also permit clients to interact with data providers. The joint middleware/resource and provider/consumer interactions implemented with performance attributes may be used to realize effective methods for managing the data flows in the large‐data, distributed Grid applications targeted by our research. Experimental results in this paper demonstrate substantial performance improvements. These are attained by coordinating network‐level with service‐level adaptations of the data being transported and by permitting end users to dynamically deploy and use application‐specific services for manipulating data in ways suitable for their current needs. Copyright

IQ-Paths: Predictably High Performance Data Streams across Dynamic Network Overlays

Overlay networks are a key vehicle for delivering network and processing resources to high performance applications. For shared networks, however, to consistently deliver such resources at desired levels of performance, overlays must be managed at runtime, based on the continuous assessment and prediction of available distributed resources. Data-intensive applications, for example, must assess, predict, and judiciously use available network paths, and dynamically choose alternate or exploit concurrent paths. Otherwise, they cannot sustain the consistent levels of performance required by tasks like remote data visualization, online program steering, and remote access to high end devices. The multiplicity of data streams occurring in complex scientific workflows or in large-scale distributed collaborations exacerbate this problem, particularly when different streams have different performance requirements. This paper presents IQ-Paths, a set of techniques and their middleware realization that implement self-regulating overlay streams for data-intensive distributed applications. Self-regulation is based on (1) the dynamic and continuous assessment of the quality of each overlay path, (2) the use of online network monitoring and statistical analyses that provide probabilistic guarantees about available path bandwidth, loss rate, and RTT, and (3) self-management, via an efficient packet routing and scheduling algorithm that dynamically schedules data packets to different overlay paths in accordance with their available bandwidths. IQ-Paths offer probabilistic guarantees for application-level specifications of stream utility, based on statistical predictions of available network bandwidth. This affords applications with the ability, for instance, to send control or steering data across overlay paths that offer strong guarantees for future bandwidth vs. across less guaranteed paths. Experimental results presented in this paper use IQ-Paths to better handle the different kinds of data produced by two high performance applications: (1) a data-driven or interactive high performance code with user-defined utility requirements and (2) an adaptive overlay version of the popular Grid-FTP application