Pierre-Henri Wuillemin

Learning the structure of Factored Markov Decision Processes in reinforcement learning problems

Recent decision-theoric planning algorithms are able to find optimal solutions in large problems, using Factored Markov Decision Processes (FMDPs). However, these algorithms need a perfect knowledge of the structure of the problem. In this paper, we propose SDYNA, a general framework for addressing large reinforcement learning problems by trial-and-error and with no initial knowledge of their structure. SDYNA integrates incremental planning algorithms based on FMDPs with supervised learning techniques building structured representations of the problem. We describe SPITI, an instantiation of SDYNA, that uses incremental decision tree induction to learn the structure of a problem combined with an incremental version of the Structured Value Iteration algorithm. We show that SPITI can build a factored representation of a reinforcement learning problem and may improve the policy faster than tabular reinforcement learning algorithms by exploiting the generalization property of decision tree induction algorithms.

Insulin resistance and inflammation predict kinetic body weight changes in response to dietary weight loss and maintenance in overweight and obese subjects by using a Bayesian network approach

BACKGROUND The ability to identify obese subjects who will lose weight in response to energy restriction is an important strategy in obesity treatment. OBJECTIVE We aimed to identify obese subjects who would lose weight and maintain weight loss through 6 wk of energy restriction and 6 wk of weight maintenance. DESIGN Fifty obese or overweight subjects underwent a 6-wk energy-restricted, high-protein diet followed by another 6 wk of weight maintenance. Network modeling by using combined biological, gut microbiota, and environmental factors was performed to identify predictors of weight trajectories. RESULTS On the basis of body weight trajectories, 3 subject clusters were identified. Clusters A and B lost more weight during energy restriction. During the stabilization phase, cluster A continued to lose weight, whereas cluster B remained stable. Cluster C lost less and rapidly regained weight during the stabilization period. At baseline, cluster C had the highest plasma insulin, interleukin (IL)-6, adipose tissue inflammation (HAM56+ cells), and Lactobacillus/Leuconostoc/Pediococcus numbers in fecal samples. Weight regain after energy restriction correlated positively with insulin resistance (homeostasis model assessment of insulin resistance: r = 0.5, P = 0.0002) and inflammatory markers (IL-6; r = 0.43, P = 0.002) at baseline. The Bayesian network identified plasma insulin, IL-6, leukocyte number, and adipose tissue (HAM56) at baseline as predictors that were sufficient to characterize the 3 clusters. The prediction accuracy reached 75.5%. CONCLUSION The resistance to weight loss and proneness to weight regain could be predicted by the combination of high plasma insulin and inflammatory markers before dietary intervention.

Structured probabilistic inference

Probabilistic inference is among the main topics with reasoning in uncertainty in AI. For this purpose, Bayesian Networks (BNs) is one of the most successful and efficient Probabilistic Graphical Model (PGM) so far. Since the mid-90s, a growing number of BNs extensions have been proposed. Object-oriented, entity-relationship and first-order logic are the main representation paradigms used to extend BNs. While entity-relationship and first-order models have been successfully used for machine learning in defining lifted probabilistic inference, object-oriented models have been mostly underused. Structured inference, which exploits the structural knowledge encoded in an object-oriented PGM, is a surprisingly unstudied technique. In this paper we propose a full object-oriented framework for PRM and propose two extensions of the state-of-the-art structured inference algorithm: SPI which removes the major flaws of existing algorithms and SPISBB which largely enhances SPI by using d-separation.

Bayesian network structure learning using cooperative coevolution

We propose a cooperative-coevolution - Parisian trend - algorithm, IMPEA (Independence Model based Parisian EA), to the problem of Bayesian networks structure estimation. It is based on an intermediate stage which consists of evaluating an independence model of the data to be modelled. The Parisian cooperative coevolution is particularly well suited to the structure of this intermediate problem, and allows to represent an independence model with help of a whole population, each individual being an independence statement, i.e. a component of the independence model. Once an independence model is estimated, a Bayesian network can be built. This two level resolution of the complex problem of Bayesian network structure estimation has the major advantage to avoid the difficult problem of direct acyclic graph representation within an evolutionary algorithm, which causes many troubles related to constraints handling and slows down algorithms. Comparative results with a deterministic algorithm, PC, on two test cases (including the Insurance BN benchmark), prove the efficiency of IMPEA, which provides better results than PC in a comparable computation time, and which is able to tackle more complex issues than PC.

Bayesian network structure learning from limited datasets through graph evolution

Bayesian networks are stochastic models, widely adopted to encode knowledge in several fields. One of the most interesting features of a Bayesian network is the possibility of learning its structure from a set of data, and subsequently use the resulting model to perform new predictions. Structure learning for such models is a NP-hard problem, for which the scientific community developed two main approaches: score-and-search metaheuristics, often evolutionary-based, and dependency-analysis deterministic algorithms, based on stochastic tests. State-of-the-art solutions have been presented in both domains, but all methodologies start from the assumption of having access to large sets of learning data available, often numbering thousands of samples. This is not the case for many real-world applications, especially in the food processing and research industry. This paper proposes an evolutionary approach to the Bayesian structure learning problem, specifically tailored for learning sets of limited size. Falling in the category of score-and-search techniques, the methodology exploits an evolutionary algorithm able to work directly on graph structures, previously used for assembly language generation, and a scoring function based on the Akaike Information Criterion, a well-studied metric of stochastic model performance. Experimental results show that the approach is able to outperform a state-of-the-art dependency-analysis algorithm, providing better models for small datasets.

A memetic approach to bayesian network structure learning

Bayesian networks are graphical statistical models that represent inference between data. For their effectiveness and versatility, they are widely adopted to represent knowledge in different domains. Several research lines address the NP-hard problem of Bayesian network structure learning starting from data: over the years, the machine learning community delivered effective heuristics, while different Evolutionary Algorithms have been devised to tackle this complex problem. This paper presents a Memetic Algorithm for Bayesian network structure learning, that combines the exploratory power of an Evolutionary Algorithm with the speed of local search. Experimental results show that the proposed approach is able to outperform state-of-the-art heuristics on two well-studied benchmarks.

A Dynamic Bayesian Network to Represent a Ripening Process of a Soft Mould Cheese

Available knowledge to describe food processes has been capitalized from different sources, is expressed under different forms and at different scales. To reconstruct the puzzle of knowledge by taking into account uncertainty, we need to combine, integrate different kinds of knowledge. Mathematical concepts such that expert systems, neural networks or mechanistic models reach operating limits. In all cases, we are faced with the limits of available data, mathematical formalism and the limits of human reasoning. Dynamical Bayesian Networks (DBNs) are practical probabilistic graphic models to represent dynamical complex systems tainted with uncertainty. This paper presents a simplified dynamic bayesian networks which allows to represent the dynamics of microorganisms in the ripening of a soft mould cheese (Camembert type) by means of an integrative sensory indicator. The aim is the understanding and modeling of the whole network of interacting entities taking place between the different levels of the process.

The MIDAS Cloud Platform for Testing SOA Applications

While Service Oriented Architectures (SOAs) are for many parts deployed online, and today often in a cloud, the testing of the systems still happens mostly locally. In this paper, we want to present the MIDAS Testing as a Service (TaaS), a cloud platform for the testing of SOAs. We focus on the testing of whole SOA orchestrations, a complex task due to the number of potential service interactions and the increasing complexity with each service that joins an orchestration. Since traditional testing does not scale well with such a complex setup, we employ a Model-based Testing (MBT) approach based on the Unified Modeling Language (UML) and the UML Testing Profile (UTP) within MIDAS. Through this, we provide methods for functional testing, security testing, and usage-based testing of service orchestrations. Through harnessing the computational power of the cloud, MIDAS is able to generate and execute complex test scenarios which would be infeasible to run in a local environment.

A kd-tree algorithm to discover the boundary of a black box hypervolume

In the framework of Decision Support Systems, mathematical viability theory can be used to classify the states and the trajectories of a dynamical system evolving in a set of desirable states. Since obtaining this viability theory output is a complex and computationally intensive task, we propose in this article to consider a compact representation of this set and its approximations using kd-trees. Given a subset of Rn

Automation and intelligent scheduling of distributed system functional testing

\mathbb {R}^{n}