Xuan Zhou

Adaptive Service Composition Based on Reinforcement Learning

The services on the Internet are evolving. The various properties of the services, such as their prices and performance, keep changing. To ensure user satisfaction in the long run, it is desirable that a service composition can automatically adapt to these changes. To this end, we propose a mechanism for adaptive service composition. The mechanism requires no prior knowledge about services’ quality, while being able to achieve the optimal composition solution by leveraging the technology of reinforcement learning. In addition, it allows a composite service to dynamically adjust itself to fit a varying environment, where the properties of the component services continue changing. We present the design of our mechanism, and demonstrate its effectiveness through an extensive experimental evaluation.

Web Service Selection with Incomplete or Inconsistent User Preferences

Web service selection enables a user to find the most desirable service based on his / her preferences. However, user preferences in real world can be either incomplete or inconsistent, such that service selection cannot be conducted properly. This paper presents a system to facilitate Web service selection in face of incomplete or inconsistent user preferences. The system utilizes the information of historical users to amend the active users preference, so as to improve the results of service selection. We present a detailed design of the system and verify its efficiency through extensive experiments.

Web Service Classification Using Support Vector Machine

Classification is a widely used mechanism for facilitating Web service discovery. Existing methods for automatic Web service classification only consider the case where the category set is small. When the category set is big, the conventional classification methods usually require a large sample collection, which is hardly available in real world settings. This paper presents a novel method to conduct service classification with a medium or big category set. It uses the descriptive information of categories in a large-scale taxonomy as sample data, so as to disengage from the dependence on sample service documents. A new feature selection method is introduced to enable efficient classification using this new type of sample data. We demonstrate the effectiveness of our classification method through extensive experiments.

End-to-End Service Support for Mashups

We propose a service-oriented approach to generate and manage mashups. The proposed approach is realized using the Mashup Services System (MSS), a novel platform to support users to create, use, and manage mashups with little or no programming effort. The proposed approach relieves users from programming-intensive, error-prone, and largely nonreusable output process for creating and maintaining mashups. We describe the overall design of MSS and discuss and evaluate its main enabling technologies.

Adaptive and Dynamic Service Composition Using Q-Learning

In a dynamic environment, some services may become unavailable, some new services may be published and the various properties of the services, such as their prices and performance, may change. Thus, to ensure user satisfaction in the long run, it is desirable that a service composition can automatically adapt to these changes. To this end, we leverage the technology of reinforcement learning and propose a mechanism for adaptive service composition. The mechanism requires no prior knowledge about services’ quality, while being able to achieve the optimal composition solution. In addition, it allows a composite service to dynamically adjust itself to fit a varying environment. We present the design of our mechanism, and demonstrate its effectiveness through an extensive experimental evaluation.

A Quantitative and Qualitative Approach for NFP-Aware Web Service Composition

Web service composition is a standard approach to create value-added services from existing ones. As the Web services on the Internet grows, there are more and more services providing identical functionalities while differing in their non-functional properties (NFPs). However, most of the existing techniques for NFP-aware service composition consider either only quantitative NFPs or only qualitative NFPs. In this paper, we present a service composition model considering both quantitative and qualitative NFPs. We propose two algorithms for conducting service composition. One combines global optimization with local selection into one mechanism. The other is a genetic algorithm based solution. We have conducted extensive experiments to evaluate the effectiveness of our proposals.

Preference recommendation for personalized search

Conditional Preference Networks (CP-nets) are widely used to express qualitative preferences. As users are sometimes reluctant or unable to specify complete CP-nets, it prohibits personalized search from being effectively conducted. In this article, we present an approach to perform personalized search using incomplete CP-nets. We propose a preference recommendation scheme for complementing a users CP-nets, so as to improve the accuracy of personalized search. We have conducted extensive simulation and user study to demonstrate the effectiveness of our approach.

Top-k retrieval using conditional preference networks

This paper considers top-k retrieval using Conditional Preference Network (CP-Net). As a model for expressing user preferences on multiple mutually correlated attributes, CP-Net is of great interest for decision support systems. However, little work has addressed how to conduct efficient data retrieval using CP-Nets. This paper presents an approach to efficiently retrieve the most preferred data items based on a users CP-Net. The proposed approach consists of a top-k algorithm and an indexing scheme. We conducted extensive experiments to compare our approach against a baseline top-k method - sequential scan. The results show that our approach outperform sequential scan in several circumstances.

Automatic Discovery and Transfer of MAXQ Hierarchies in a Complex System

Reinforcement learning has been an important category of machine learning approaches exhibiting self-learning and online learning characteristics. Using reinforcement learning, an agent can learn its behaviors through trial-and-error interactions with a dynamic environment and finally come up with an optimal strategy. Reinforcement learning suffers the curse of dimensionality, though there has been significant progress to overcome this issue in recent years. MAXQ is one of the most common approaches for reinforcement learning. To function properly, MAXQ requires a decomposition of the agents task into a task hierarchy. Previously, the decomposition can only be done manually. In this paper, we propose a mechanism for automatic subtask discovery. The mechanism applies clustering to automatically construct task hierarchy required by MAXQ, such that MAXQ can be fully automated. We present the design of our mechanism, and demonstrate its effectiveness through theoretical analysis and an extensive experimental evaluation.

A Multi-agent Reinforcement Learning Model for Service Composition

This paper describes a multi-agent reinforcement learning model for the optimization of Web service composition. Based on the model, we propose a multiagent Q-learning algorithm, where each agent would benefit from the advice of other agents in team. In contrast to single-agent reinforcement learning, our algorithm can speed up convergence to optimal policy. In addition, it allows composite service to dynamically adjust itself to fit the varying environment, where the properties of the component services continue changing. Our experiments demonstrate the efficiency of our algorithm.