I.S.M. de Jong

Test Sequencing in Complex Manufacturing Systems

Testing complex manufacturing systems, such as an ASML lithographic machine, takes up to 45% of the total development time of a system. The problem of which tests must be executed in what sequence to ensure in the shortest possible test time that the system works, which is the test-sequencing problem, was already solved by Pattipati et al. for the diagnosis of systems during operation. Test-sequencing problems during the development and manufacturing phases of systems, however, require a different approach than the test-sequencing problems during operation. In this paper, the test problem description and algorithms developed by Pattipati et al. are extended to solve test-sequencing problems for the development and manufacturing of manufacturing systems. For a case study in the manufacturing process of an ASML lithographic machine, it is shown that solving a test-sequencing problem with this method can reduce the test time by 15% to 30% compared to experts that solve this problem manually.

Hierarchical Test Sequencing for Complex Systems

Testing complex systems, such as the ASML TWINSCAN lithographic machine, is expensive and time consuming. In a previous work, a test sequencing method to calculate time-optimal test sequences has been developed. Because complex systems are composed of several subsystems, which are again composed of several modules, there exists a need to hierarchically model test sequencing problems. Such a hierarchical test sequencing problem consists of a high-level model that describes a test sequencing problem at the system level, and one or more low-level models that describe the test sequencing problems at the subsystem or module level. The tests at the system level correspond to the solutions of low-level problems. This paper describes a hierarchical test sequencing model and proposes two algorithms to compute an optimal test sequence. The benefits of hierarchically modeling a problem are less computational effort and less modeling effort, because not all relations are needed. This is illustrated by a small example. The industrial relevance of this method is illustrated on a case study related to a manufacturing testing phase of a lithographic machine.

Integration and Test Sequencing for Complex Systems

The integration and test phase of complex manufacturing machines, like an ASML lithographic manufacturing system, is expensive and time consuming. The tests that can be performed at a certain point in time during the integration phase depend on the modules that are integrated and, therefore, on the preceding integration sequence. In this paper, we introduce a mathematical model to describe an overall integration and test sequencing problem, and we propose an algorithm to solve this problem. The method is a combination of integration sequencing and test sequencing. Furthermore, we introduce several strategies that determine when test phases should start. With a case study within the development of a software release that is used to control an ASML lithographic machine, we show that the described method and strategies can be used to solve real-life problems.

Risk-Based Stopping Criteria for Test Sequencing

Testing complex manufacturing systems, like ASML lithographic machines, can take up to 45% of the total development time. The decision of when to stop testing is often difficult to make because less testing may leave critical faults in the system, while more testing increases time-to-market. In this paper, we solve the problem of deciding when to stop testing by introducing a test-sequencing method that incorporates several stopping criteria. These stopping criteria consist of objectives and constraints on the test cost and the remaining risk cost. For a given problem, a suitable stopping criterion can be chosen. For example, with the risk-based stopping criterion, testing stops when the test time or cost exceeds the risk cost. Furthermore, we show that it also is possible to model reliability problems with this test-sequencing method. The method is demonstrated on ASML systems with two case studies. The first case study was conducted in the test phase during the development of the software that is used to control an ASML lithographic machine. The second case study was conducted on the reliability testing of a lithographic machine.