Felix Freitag

Catallaxy-based Grid markets

Grid computing has recently become an important paradigm for managing computationally demanding applications, composed of a collection of services. The dynamic discovery of services, and the selection of a particular service instance providing the best value out of the discovered alternatives, poses a complex multi-attribute n:m allocation decision problem, which is often solved using a centralized resource broker. To manage complexity, this article proposes a two-layer architecture for service discovery in such Application Layer Networks (ALN). The first layer consists of a service market in which complex services are translated to a set of basic services, which are distinguished by price and availability. The second layer provides an allocation of services to appropriate resources in order to enact the specified services. This framework comprises the foundations for a later comparison of centralized and decentralized market mechanisms for allocation of services and resources in ALNs and Grids.

The catallaxy approach for decentralized economic-based allocation in Grid resource and service markets

Efficient resource allocation in dynamic large-scale environments is one of the challenges of Grids. In centralized economic-based allocation approaches, the user requests can be matched to the fastest, cheapest or most available resource. This approach, however, shows limitations in scalability and in dynamic environments. In this paper, we explore a decentralized economic approach for resource allocation in Grid markets based on the Catallaxy paradigm. Catallactic agents discover selling nodes in the resource and service Grid markets, and negotiate with each other maximizing their utility by following a strategy. By means of simulations, we evaluate the behavior of the approach, its resource allocation efficiency and its performance with different demand loads in a number of Grid density and dynamic environments. Our results indicate that while the decentralized economic approach based on Catallaxy applied to Grid markets shows similar efficiency to a centralized system, its decentralized operation provides greater advantages: scalability to demand and offer, and robustness in dynamic environments.

Decentralized resource allocation in application layer networks

Application-layer networks (ALN) are software architectures that allow the provisioning of services requiring a huge amount of resources by connecting large numbers of individual computers. The ALN simulation project CATNET evaluates a decentralized mechanism for resource allocation in ALN, which is based on the economic paradigm of the Catallaxy, against a centralized mechanism using an arbitrator object. In both versions, software agents buy and sell network services and resources to and from each other. The economic model is based on self-interested maximization of utility and self-interested cooperation between agents. This article describes the design of money and message flows for centralized and decentralized coordination in both versions and shows preliminary results.

A dynamic periodicity detector: application to speedup computation

We propose a dynamic periodicity detector (DPD) for the estimation of periodicities in data series obtained from the execution of applications. We analyze the algorithm used by the periodicity detector and its performance on a number of data streams. It is shown how the periodicity detector is used for the segmentation and prediction of data streams. In an application case we describe how the periodicity detector is applied to the dynamic detection of iterations in parallel applications, where the detected segments are evaluated by a speedup computation tool. We test the performance of the periodicity detector on a number of parallelized benchmarks. The periodicity detector correctly identifies the iterations of parallel structures also in the case where the application has nested parallelism. In our implementation we measure only a negligible overhead produced by the periodicity detector. We find the DPD to be useful and suitable for the incorporation in dynamic optimization tools.

Decentralized vs. centralized economic coordination of resource allocation in grids

Application layer networks are software architectures that allow the provisioning of services requiring a huge amount of resources by connecting large numbers of individual computers, like in Grid or Peer-to-Peer computing. Controlling the resource allocation in those networks is nearly impossible using a centralized arbitrator. The network simulation project CATNET will evaluate a decentralized mechanism for resource allocation, which is based on the economic paradigm of the Catallaxy, against a centralized mechanism using an arbitrator object. In both versions, software agents buy and sell network services and resources to and from each other. The economic model is based on self-interested maximization of utility and self-interested cooperation between agents. This article describes the setup of money and message flows both for centralized and decentralized coordination in comparison.

A catallactic market for data mining services

We describe a Grid market for exchanging data mining services based on the catallactic market mechanism proposed by von Hayek. This market mechanism allows selection between multiple instances of services based on operations required in a data mining task (such as data migration, data pre-processing and subsequently data analysis). Catallaxy is a decentralized approach, based on a free market mechanism, and is particularly useful when the number of market participants is large or when conditions within the market often change. It is therefore particularly suitable in Grid and peer-2-peer systems. The approach assumes that the service provider and user are not co-located, and require multiple message exchanges to carry out a data mining task. A market of J48-based decision tree algorithm instances, each implemented as a Web service, is used to demonstrate our approach. We have validated the feasibility of building catallactic data mining grid applications, and implemented a proof-of-concept application (Cat-COVITE) mapped to a catallactic Grid middleware.

Improving the service time of web clients using server redirection

This paper describes and evaluates experimentally a web server infrastructure, which consists of a small number of servers that redirect client requests based on the estimated client service time. The web servers have replicated content, are located in geographically different regions, and redirect clients between servers. The web servers use metrics obtained from server logs to estimate the service time of a client. Based on the estimated service time the server redirects the web client. The implementation of the measurement and redirection mechanism is done in the web servers and is independent of the clients. Using server logs the measuring mechanism does not introduce traffic into the network. We have experimentally evaluated the proposed web server infrastructure. In our experiments the client service time improved from 4 to 40 % when using the proposed mechanism. The web server infrastructure could be applied to improve the service time of selected clients, which frequently access a web server to retrieve a significant amount of data.

Multicast injection for application network deployment

The introduction of new services on the Internet is a laborious, time-consuming task. Application networks, applications being serviced through multiple interconnected service nodes disseminated across the Internet for better performance, fault tolerance and availability, as well are costly to set up. In order to provide a network-enabled application service, a number of surrogate servers have to be provisioned. We propose a mechanisms to dynamically deploy an application network: multicast injection. Currently employed dynamic deployment mechanisms, SNMP per-node configuration, is a centralized model that can not scale or be as fault tolerant as more distributed mechanisms such as multicast injection. We perform simulations to compare its efficiency in terms of deployment request success ratio, unused allocation percentage and traffic vs. deployment resource allocation requests. We show that multicast injection has a higher success ratio with lower bandwidth consumption at the deployers locations.

An architecture for incorporating decentralized economic models in application layer networks

Efficient resource discovery and allocation is one of the challenges of any large scale Application Layer Network (ALN) such as computational Grids, Content Distribution Networks and P2P applications. In centralized approaches, the user requests can easily be matched to the most convenient resource. These approaches, however, present scalability limits in the highly dynamic and complex ALN environments. This paper, explores an architecture for incorporating fully decentralized economic mechanisms for resource allocation. These mechanisms are implemented by a set of trading agents that operate on behalf of the clients and service providers, interacting over an overlay network and interfacing with the underlying resources of the platform. A prototype of the proposed architecture is presented and the practical implications of its implementation in a grid scenario are discussed.

A trace-scaling agent for parallel application tracing

Tracing and performance analysis tools are an important component in the development of high performance applications. Tracing parallel programs with current tracing tools, however, easily leads to large trace files with hundreds of Megabytes. The storage, visualization, and analysis of such trace files is often difficult. We propose a trace-scaling agent for tracing parallel applications, which learns the application behavior in runtime and achieves a small, easy to handle trace. The agent dynamically identifies the amount of information needed to capture the application behavior. This knowledge acquired at runtime allows recording only the non-iterative trace information, which drastically reduces the size of the trace file.