Munehiro Fukuda

Distributed computing using autonomous objects

Most existing distributed systems are structured as statically compiled processes communicating with each other via messages. The systems intelligence is embodied in the processes, while the messages contain simple, passive pieces of information. This is referred to as the communicating objects paradigm. In the autonomous objects paradigm, a message has its own identity and behavior. It decides at runtime where it wants to propagate and what tasks to perform there; the nodes become simply generic interpreters that enable messages to navigate and compute. In this scenario, an applications intelligence is embodied in and carried by messages as they propagate through the network. The autonomous objects paradigm is more flexible than the communicating objects paradigm because it allows developers to change the programs behavior after it has started to run. We based our system, MESSENGERS, on autonomous objects, and intended it for the composition and coordination of concurrent activities in a distributed environment. It combines powerful navigational capabilities found in other autonomous objects based systems with efficient dynamic linking mechanisms supported by some new programming languages, like Java.

A mobile-agent-based PC grid

This paper proposes a mobile-agent-based middleware that benefits remote computer users who wish to mutually offer their desktop computing resource to other Internet group members while their computers are not being used. Key to this resource exchange grid is the use of mobile agents. Each agent represents a client user, carries his/her job requests, searches for resources available for the request, executes the job at suitable computers, and migrates it to others when the current ones have become unavailable for use. All the features of job migration will be encapsulated in a user program wrapper that is implemented on Java layer between a mobile agent and the corresponding user program. The wrapper maintains the complete execution state of the user program, is carried by the mobile agent upon a job migration, and restores its user program its destination. For this purpose, a user program is preprocessed with JavaCC and ANTLR to include check-pointing functions before its execution. These functions periodically save the execution state of a user program into its corresponding program wrapper, which can thus be carried by an agent smoothly.

Messages versus Messengers in Distributed Programming

Messengers are autonomous objects, each capable of navigating through the underlying network and performing various tasks at each node. Messengers applications are written usingnavigationalcommands rather than thesend/receiveprimitives of conventional message-passing approaches. In this paper we contrast the two programming styles. From a software engineering viewpoint, the navigational style generally results in a smaller semantic gap between abstract algorithm descriptions and their actual implementations, which makes programs easier to construct, understand, and maintain. In terms of performance, Messengers programs are highly competitive with message-passing. We demonstrate these advantages using two concrete applications programmed using Messengers and PVM.

Automatic State Capture of Self-Migrating Computations in MESSENGERS

With self-migrating computations, the main challenge is the extraction and subsequent restoration of the computation’s state during migration. This is very difficult when the navigational statement may be placed anywhere in the code and hence many systems place the burden of state capture on the application programmer. We describe an intermediate approach, where the use of navigational statements is restricted to the top level of the self-migrating computation. This permits an efficient implementation of a fully transparent state capture and restoration. We demonstrate that this approach is applicable not only to interpreted mobile code but also to compiled self-migrating computations executing entirely in native mode.

AgentTeamwork: Coordinating grid-computing jobs with mobile agents

AgentTeamwork is a grid-computing middleware system that dispatches a collection of mobile agents to coordinate a user job over remote computing nodes in a decentralized manner. Its utmost focus is to maintain high availability and dynamic balancing of distributed computing resources to a parallel-computing job. For this purpose, a mobile agent is assigned to each process engaged in the same job, monitors its execution at a different machine, takes its periodical execution snapshot, moves it to a lighter-loaded machine, and resumes it from the latest snapshot upon an accidental crash. The system also restores broken inter-process communication involved in the same job using its error-recoverable socket and mpiJava libraries in collaboration among mobile agents.We have implemented the first version of our middleware including a mobile agent execution platform, error-recoverable socket and mpiJava API libraries, a job wrapper program, and several types of mobile agents such as commander, resource, sentinel, and bookkeeper agents, each orchestrating, allocating resources to, monitoring and maintaining snapshots of a user process respectively. This paper presents AgentTeamwork’s execution model, its implementation techniques, and our performance evaluation using the Java Grande benchmark test programs.

Distributed coordination with MESSENGERS

Abstract messengers is a paradigm for the programming of distributed systems. It is based on the principles of autonomous messages, called Messengers, which carry their own behavior in the form of a program. This enables them to navigate freely in the underlying computational network, communicate with one another, and invoke compiled node-resident C functions in the nodes they visit. Hence, a distributed application is viewed as a collection of C functions whose invocation and interoperation is orchestrated by Messengers. This provides for a clear separation between computations, as expressed by the individual node functions, and coordination, which is the order of function invocations and the transport of information among them as prescribed by Messengers. This separation allows each layer to be designed and implemented separately. It also supports the reuse of the coordination structures and the interactive and incremental development and use of distributed applications.

A Design of Flexible Data Channels for Sensor-Cloud Integration

The recent popularity of sensor networks and cloud computing has brought new opportunities of sensor-cloud integration that facilitates users to immediately forecast the future status of what they are observing. The main challenge is how to establish a series of elastic data channels from sensors to mobile users through data-analyzing jobs running in the cloud. The variation of network protocols, the dynamism of cloud-computing resources, and even user mobility result in the following three design problems. Data-analyzing programs must: (1) deal with the major sensor and cloud communication protocols, (2) include hard-coded sensor manipulation and data retrieval operations, and (3) handle job/user migration in dynamic cloud environments, which needs the recovery of broken data connections. To address these problems, we are developing the Connector software tool kit that provides cloud jobs with uniform data channels, allows them to exchange standard input/output as well as graphics with remote users, and re-establishes their data channels upon a job or user migration. This paper presents Connectors system overview, implementation, and performance consideration.

A Parallel Multi-agent Spatial Simulation Environment for Cluster Systems

For more than the last 20 decades, multi-agent simulations have been highlighted to model mega-scale social or biological agents and to simulate their emergent collective behavior that may be difficult only with mathematical and macroscopic approaches. A successful key for simulating mega-scale agents is to speed up the execution with parallelization. Although many parallelization attempts have been made to multi-agent simulations, most work has been done on shared-memory programming environments such as OpenMP, CUDA, and Global Array, or still has left several programming problems specific to distributed-memory systems, such as machine unawareness, ghost space management, and cross-processor agent management (including migration, propagation, and termination). To address these parallelization challenges, we have been developing MASS, a new parallel-computing library for multi-agent and spatial simulation over a cluster of computing nodes. MASS composes a user application of distributed arrays and multi-agents, each representing an individual simulation place or an active entity. All computation is enclosed in each array element or agent, all communication is scheduled as periodic data exchanges among those entities, using machine-independent identifiers, and agents migrate to a remote array element for rendezvousing with each other. This paper presents the programming model, implementation, and evaluation of the MASS library.

Intra- and Inter-Object Coordination with MESSENGERS

MESSENGERS is a paradigm for the programming of distributed systems. It is based on the principles of autonomous messages, called Messengers, which carry their own behavior in the form of a program. This enables them to navigate freely in the underlying computational network, communicate with one another, and invoke pre-compiled node-resident functions. Coordination is facilitated at two distinct levels of abstraction: first, Messengers coordinate the invocation and the exchange of data among the various functions distributed throughout the network in both time and space (intra-object coordination); second, Messengers, each representing a high-level entity, can coordinate their behaviors among themselves (inter-object coordination). These principles, where an application is composed of autonomous, mobile entities whose behaviors may change dynamically and which can coordinate their actions among themselves, offer great flexibility in interacting with and manipulating the application at run time, as well as improved performance. This is illustrated using two concrete examples—a Toxicology simulation from medicine and a study of collective fish behavior from biology.

Design and performance of Maestro cluster network

Most clusters so far have used WAN or LAN-based network products for communication due to their market availability. However they do not always match communication patterns in clusters, thus incurring extra overhead. Based on our investigation for such overhead, we have optimized cluster communication at link layer. Partitioning each message in 16-byte packets, our optimization uses two techniques: (1) transferring in burst as many packets as the receiving buffer accepts at once, and (2) having each hardware component pass one packet to another in a pipelined manner. We have realized those two techniques in a link control hardware chip, referred to as MLC(Maestro Link Controller), and have constructed the Maestro cluster network using MLCs. This paper describes the feature of the Maestro cluster network and demonstrates the efficiency of our optimization techniques through performance experiments over this network.