Spiridon A. Reveliotis

Polynomial-complexity deadlock avoidance policies for sequential resource allocation systems

The development of efficient deadlock avoidance policies (DAPs) for sequential resource allocation systems (RASs) is a problem of increasing interest in the scientific community, largely because of its relevance to the design of large-scale flexibly automated manufacturing systems. Much of the work on this problem existing in the literature is focused on the so-called single-unit RAS model, which is the simplest model in the considered class of RASs. Furthermore, due to a well-established result stating that, even for single-unit RASs, the computation of the maximally permissive DAP is intractable (NP-hard), many researchers (including our group) have focused on obtaining good suboptimal policies which are computationally tractable (scalable) and provably correct. In the first part of the paper, it is shown, however, that for a large subset (in fact, a majority) of single-unit RASs, the optimal DAP can be obtained in real-time with a computational cost which is a polynomial function of the system size (i.e., the number of resource types and the distinct route stages of the processes running through the system). The implications of this result for the entire class of single-unit RASs are also explored. With a result on the design of optimal DAPs for single-unit RASs, the second part of the paper concentrates on the development of scalable and provably correct DAPs for the more general case of conjunctive RASs.

Design Guidelines for Deadlock-Handling Strategies in Flexible Manufacturing Systems

Deadlock-free operation of flexible manufacturing systems (FMSs) is an important goal of manufacturing systems control research. In this work, we develop the criteria that real-time FMS deadlock-handling strategies must satisfy. These criteria are based on a digraph representation of the FMS state space. Control policies for deadlock-free operation are characterized as partitioning cuts on this digraph. We call these structural control policies (SCPs) because, to avoid deadlock, they must guarantee certain structural properties of the subdigraph containing the empty state; namely, that it is strongly connected. A policy providing this guarantee is referred to as correct. Furthermore, an SCP must be configurable and scalable; that is, its correctness must not depend on configuration-specific system characteristics and it must remain computationally tractable as the FMS grows in size. Finally, an SCP must be efficient; that is, it must not overly constrain FMS operation. We formally develop and define these criteria, formulate guidelines for developing policies satisfying these criteria, and then provide an example SCP development using these guidelines. Finally, we present an SCP that guarantees deadlock-free buffer space allocation for FMSs with no route restrictions.

A practical approach to the design of maximally permissive liveness-enforcing supervisors for complex resource allocation systems

The problem of designing and deploying liveness-enforcing supervisors (LES) for sequential resource allocation systems is well-documented and extensively researched in the current literature. Acknowledging the fact that the computation of the maximally permissive LES is an NP-hard problem, most of the present solutions tend to trade off maximal permissiveness for computational tractability and ease of the policy design and implementation. In this work, we demonstrate that the maximally permissive LES can be a viable solution for the resource allocation taking place in many practical applications, by (a) effectively differentiating between the off-line and on-line problem complexity, and (b) controlling the latter through the development of succinct and compact representations of the information that is necessary for the characterization of the maximal permissive LES.

Reinforcement learning: Architectures and algorithms

This article is related to the research effort of constructing an intelligent agent, i.e., a computer system that is able to sense its environment (world), reason utilizing its internal knowledge and execute actions upon the world (act). the specific part of this effor presented in this article is reinforcement learning, i.e., the process of acquiring new knowledge based upon an evaluative feedback, called reinforcement, received by tht agent through interactions with the world. This article has two objectives: (1) to give a compact overview of reinforcement learning, and (2) to show that the evolution of the reinforcement learning paradigm has been driven by the need for more efficient learning through the addition of more structure to the learning agent. Therefore, both main ideas of reinforcement learning are introduced, and structural solutions to reinforcemen learning are reviewed. Several architectural enhancements of the RL paradigm are discussed. These include incorporation of state information in the learning process, architectural solutions to learning with delayed reinforcement, dealing with structurally changing worlds through utilization of multiple models of the world, and focusing attention of the learning agent through active perception. the paper closes with an overview of directions for applications and for future research in this area.

Optimal Node Visitation in Stochastic Digraphs

The optimal node visitation (ONV) problem addressed in this paper concerns the visitation of a subset of nodes in a stochastic graph a specified number of times, while minimizing the expected visits to another node in this graph. The presented results first provide a formulation of the ONV problem as a stochastic shortest path problem, and subsequently they develop a suboptimal policy that is computationally tractable and asymptotically optimal. In particular, it is established that the ratio of the expected performance of this policy to the expected performance of an optimal policy converges to one, as the underlying visitation requirements are scaled uniformly to infinity. Furthermore, it is shown that under some stronger assumptions, the divergence of the performance of this policy from the performance of the optimal policy remains uniformly bounded by a constant, as the visitation requirements are scaled to infinity. Finally, it is shown that, for certain problem structures, the considered policy admits a closed-form characterization of its performance, which subsequently enables its optimized parameterization and its efficient integration into adaptive control schemes of even higher efficiency.

Efficient implementations of Banker's algorithm for deadlock avoidance in flexible manufacturing systems

Manufacturing systems researchers have dismissed Bankers algorithm as being too conservative for deadlock avoidance in contemporary flexibly automated, discrete-part manufacturing systems. In this paper, we provide a modified Bankers logic for the FMS context, and show that the resulting implementations compare favorably in terms of operational flexibility with modern deadlock avoidance policies developed specifically for manufacturing. Furthermore, we establish interesting theoretical relationships between Bankers and these more recent policies, and discuss extensions of Bankers logic that can also accommodate the effects of the routing flexibility which is inherent in modern production systems.

Correctness Verification of Generalized Algebraic Deadlock Avoidance Policies through Mathematical Programming

Generalized algebraic deadlock avoidance policies (DAPs) for sequential resource allocation systems (RAS) have recently been proposed as an interesting extension of the class of algebraic DAPs, that maintains the analytical representation and computational simplicity of the latter, while it guarantees completeness with respect to the maximally permissive DAP. The original work of S. Reveliotis, et al., (2007) that introduced these policies also provided a design methodology for them, but this methodology is limited by the fact that it necessitates the deployment of the entire state space of the considered RAS. Hence, this paper seeks the development of alternative computational tools that can support the synthesis of correct generalized algebraic DAPs while controlling the underlying computational complexity. From a conceptual standpoint, the presented results are motivated by and extend similar past results for the synthesis of correct algebraic DAPs. However, when viewed from a more technical standpoint, the presented developments are complicated by the fact that generalized algebraic DAPs do not admit a convenient representation in the Petri net (PN) modeling framework, that has been the primary vehicle for the aforementioned past developments, and therefore, the relevant analysis must be pursued in an alternative, automaton-based representation of the RAS behavior and the applied policy logic. We believe that this translation of the past results in this new representational framework is a significant contribution in itself, since it enables a more profound understanding of the past developments, and at the same time, it renders them more accessible to the practitioner.

An analytical framework for evaluating and optimizing the performance of structurally controlled flexible manufacturing systems

Defining the capacity of the structurally controlled FMS is an extremely involved task. Most existing evaluation models fail to capture all these features that define the effective capacity of such a system. Recognizing that, under the assumption of exponential processing times, the structurally controlled FMS behaves like a continuous-time Markov chain, this paper proposes an analytical framework for evaluating and optimizing its attained performance. Due to its nonpolynomial computational cost, the approach is applicable only to small FMS configurations, but it seems to be the only known framework able to support the exact solution of the considered problem. Furthermore, it can provide the insight needed for the development of valid approximating techniques that are computationally tractable.

A framework for on-line learning of plant models and control policies for restructurable control

In this paper a learning framework to deal with restructurable control of a single-output dynamic plant is proposed. The central concept used to represent the restructurable behavior of the plant, and subsequently for the design of the framework, is the behavioral graph. The nodes of this graph correspond to possible local behaviors of the system while its edges model the switching scheme of the plant among its local behaviors. In the definition of this concept, general dynamical system theory is used. The framework is able to learn the dynamics (models) of a reconfigurable system, select appropriate models, and ultimately control the plant according to given specifications. The framework design borrows concepts and techniques from the active fields of adaptive and learning control. The underlying ideas and the software prototype implementing the framework design are tested through a series of simulated experiments. The simulations demonstrate the feasibility of the approach for controlling plants with unexpectedly and structurally changing behaviors in moderately noisy environments. They also identify a number of constraints that have to be satisfied for successful operation of the framework. This paper also discusses further validation of the approach, real-time application issues, and potential enhancements of the frameworks functionality. >

Integrating qualitative and quantitative methods for model validation and monitoring

The issue of monitoring a time-varying plant for qualitative (structural) changes in each model when the models equations are not known is addressed. Problems with multivariate continuous domains are examined, and an overview of qualitative physics and the problem of qualitative learning and monitoring is given. Qualitative models and the semantics of qualitative transitions are discussed. The learning and monitoring are discussed. Results of experiments with a system that learns how to select one of the models given access to quantitative variables characterizing the controlled plant are presented. The model selection system makes its decisions only when a qualitative change in the plants behavior takes place.<<ETX>>