John M. Usher

Application of reinforcement learning for agent-based production scheduling

Reinforcement learning (RL) has received some attention in recent years from agent-based researchers because it deals with the problem of how an autonomous agent can learn to select proper actions for achieving its goals through interacting with its environment. Although there have been several successful examples demonstrating the usefulness of RL, its application to manufacturing systems has not been fully explored yet. In this paper, Q-learning, a popular RL algorithm, is applied to a single machine dispatching rule selection problem. This paper investigates the application potential of Q-learning, a widely used RL algorithm to a dispatching rule selection problem on a single machine to determine if it can be used to enable a single machine agent to learn commonly accepted dispatching rules for three example cases in which the best dispatching rules have been previously defined. This study provided encouraging results that show the potential of RL for application to agent-based production scheduling.

The application of genetic algorithms to operation sequencing for use in computer-aided process planning

Operation sequencing has long been a difficult problem in process planning. As part complexity increases, the number of potential solutions increases exponentially. This paper presents an approach to operation sequence coding that permits the application of genetic algorithms for quickly determining optimal, or near-optimal, operation sequences for parts of varying complexity. This approach improves on existing techniques by utilizing common sequencing constraints to guide the coding process resulting in a further reduction in the size of the solution search space. These improvements permit the determination of near-optimal operation sequences for complex parts within a time frame necessary for real-time dynamic planning. Application of this strategy is illustrated using three parts of varying complexity as well as comparing the genetic algorithms performance using the improved constrained coding strategy with that of an unconstrained strategy.

Dynamic process planning — The static phase

Abstract Dynamic process planning is concerned with the integration of process planning with scheduling. This paper presents the computer-aided process planning (CAPP) system, PARIS, that is meant to be integrated with a scheduling system. The approach to dynamic process planning implemented divides the planning tasks into the two phases: static and dynamic planning. This paper discusses the overall approach and focusing on the details of the implementation of the static phase. An example is used to illustrate the concepts presented.

Negotiation-based routing in job shops via collaborative agents

In recent years, the application of agent-based systems in manufacturing has received a lot of attention in several areas, particularly scheduling and shop-floor control. This paper explores two methods of enhancing the negotiation process employed by an agent-based system to support performance improvements in real-time routing of job orders within job-shop environments. The first method takes advantage of an extended negotiation period to provide a more complete picture of the shops conditions in order to enhance the validity of the decisions made by individual agents. The second approach explores the possibility of process model data to increase the accuracy of time estimates used in the negotiation process. The resulting performance of the two methods is tested using a simulated job shop.

Speculations on the value of telepresence.

Synthetic environments (SE) feature computer-mediated interaction with an environment physically separate from the user. An SE allows human perceptual, cognitive, and psychomotor capabilities to be projected into distant, dangerous, or simulated environments. This article examines some aspects of application of immersive/telepresence interfaces and discusses how the new technology fits into a user-centered design approach to teleoperators and virtual environments. The central theme of an immersive/telepresence design approach is that the world may be displayed to a user as if that person were physically present in a computermediated world. However, the ability of SEs to re-create a computer-mediated world by using immersive displays does not annul the responsibility of designers to tailor interfaces to meet the task-dependant needs of users. Whether functioning in reality or a virtual reality, interfaces must satisfy user information requirements to optimize performance. It does not necessarily follow that the combination of immersive interfaces, strict reproduction of the remote world, and telepresence gives users the most efficient human-machine interface. Other aspects of human behavior, such as concentration and attentional resource allocation or situation awareness, which are not necessarily encompassed by the concept of telepresence, need to be considered in the interface design.

A STEP-based object-oriented product model for process planning

This paper presents an object-oriented approach to the definition of a product model for use in supporting the task of computer-aided process planning (CAPP). The product model is based on the STEP application protocol, AP 224. The object class hierarchies used in the implementation of the product model are presented, as well as, an example to demonstrate the application of the model.

Evaluating the impact of alternative plans on manufacturing performance

In recent years, process planners have become interested in the development of dynamic process planning systems that can interface to scheduling systems providing alternative process plans to increase flexibility in scheduling. However, deciding how many alternatives are needed has not been addressed in any previous studies. This paper presents the results of a simulation-based study aimed at characterizing the benefit provided from having alternative plans available for use in scheduling. This benefit is quantified in terms of the overall performance of a job-shop manufacturing environment. The results of this study indicate that the advantage gained by increasing the number of alternative process plans diminishes rapidly. In fact, under some conditions for the particular system studied, increasing the number of alternatives actually resulted in degraded system performance. Based on these results developers of process planning systems and methodologies need to evaluate carefully the benefit of expending time and resources on the generation of alternative plans or optimal plans.

Establishing information requirements for supervisory controllers in a flexible manufacturing system using GTA

In this article we consider the technological change that has occurred in complex manufacturing systems within the past two decades and the implications it has had on the role of human operators in manufacturing systems control. Our examination ranges from the traditional production line manned by skilled machinists to flexible manufacturing systems (FMS) under supervisory control. On the basis of this study, we raise the question as to whether new advanced manufacturing technology interfaces are supportive of human operators in their responsibilities to manufacturing systems. We address this problem by analyzing supervisory controller information requirements for intervening in complex process control tasks as part of FMS operation. This analysis was conducted using a cognitive engineering research methodology, which has not previously been applied, in the domain of manufacturing. The method of GTA was applied to supervisory control of an FMS and produced detailed information requirements, which facilitated the formulation of general design guidelines for FMS interface design. The guidelines are aimed at supporting human operator process strategy development and decision making.

An object-oriented application of tool selection in dynamic process planning

Tool selection is a vital function within any process planning system. This paper presents an approach to computer-aided tool selection that has the ability to generate and rank all alternative tool sets for a process plan. This approach has been used to develop the OATS (Object-oriented Application of Tool Selection) system. OATS is designed for use as a module within a dynamic process planning system and has been tested as a part of the dynamic planning system, PARIS (Process planning Architecture for Integration with Scheduling) (Usher and Fernandes 1996a, b). The necessary input data for this system are taken from a product model based on the ISO STEP application protocol, AP224. Two alternative plans for a single part are used as examples to illustrate the approach and demonstrate the ability of the system to generate and rank all the viable alternative tool sets.

Using evolution strategies and simulation to optimize a pull production system

Abstract Evolution strategies (ES) has proven to be a robust search technique for solving deterministic problems. An ES conducts its search by processing a population of solutions for an optimization problem based on principles from evolution. This paper describes the use of ES integrated with a simulation model, which includes stochastic processes of a manufacturing system, to solve the kanban sizing problem. The ES search heuristic determines the minimum number of kanbans and corresponding production trigger values required to meet demand. The procedure is illustrated with an applied problem from a leading appliance manufacturer consisting of 39 decision variables. Insights are provided on the effect of the population size (number of parents and offspring) on the fitness of the solutions. The solutions found by the ES search heuristic are compared to solutions obtained from using the Toyota kanban sizing equation. Results indicate that the ES search heuristic provides good solutions for large kanban sizing problems.