Rajesh Ganesan

Wavelet-based multiscale statistical process monitoring: A literature review

Data that represent complex and multivariate processes are well known to be multiscale due to the variety of changes that could occur in a process with different localizations in time and frequency. Examples of changes may include mean shift, spikes, drifts and variance shifts all of which could occur in a process at different times and at different frequencies. Acoustic emission signals arising from machining, images representing MRI scans and musical audio signals are some examples that contain these changes and are not suited for single scale analysis. The recent literature contains several wavelet-decomposition-based multiscale process monitoring approaches including many real life process monitoring applications. These approaches are shown to be effective in handling different data types and, in concept, are likely to perform better than existing single scale approaches. There also exists a vast literature on the theory of wavelet decomposition and other statistical elements of multiscale monitoring methods, such as principal components analysis, denoising and charting. To our knowledge, no comprehensive review of the work relevant to multiscale monitoring of both univariate and multivariate processes has been presented to the literature. In this paper, over 150 both published and unpublished papers are cited for this important subject, and some extensions of the current research are also discussed.

Wavelet-based identification of delamination defect in CMP (Cu-low k) using nonstationary acoustic emission signal

Wavelet-based multiscale analysis approaches have revolutionized the tasks of signal processing, such as image and data compression. However, the scope of wavelet-based methods in the fields of statistical applications, such as process monitoring, density estimation, and defect identification, are still in their early stages of evolution. Recent literature contains some applications of wavelet-based methods in monitoring, such as tool-life monitoring, bearing defect monitoring, and monitoring of ultra-precision processes. This paper presents a novel application of a wavelet-based multiscale method in a nanomachining process [chemical mechanical planarization (CMP)] of wafer fabrication. The application involves identification of delamination defect of low-k dielectric layers by analyzing the nonstationary acoustic emission (AE) signal and coefficient of friction (CoF) signal collected during copper damascene (Cu-low k) CMP process. An offline strategy and a moving window-based strategy for online implementation of the wavelet monitoring approach are developed. Both offline and moving window-based strategies are implemented on the data collected from two different sources. The results show that the wavelet-based approach using the AE signal offers an efficient means for real-time detection of delamination defects in CMP processes. Such an online strategy, in contrast to the existing offline approaches, offers a viable tool for CMP process control. The results also indicate that the CoF signal is insensitive to delamination defect.

Online end point detection in CMP using SPRT of wavelet decomposed sensor data

Efficient end point detection (EPD) in chemical mechanical planarization (CMP) is critical to quality and productivity of the wafer fabrication process. The cost of over and under polishing, and the cost of ownership of many expensive metrology-based EPD methods have motivated the researchers to seek cost effective and efficient alternatives. This paper presents a novel method for EPD, which uses a sequential probability ratio test (SPRT) on the wavelet decomposed coefficient of friction (CoF) data from the CMP process. The method is made suitable for online application by developing a moving block data processing strategy, which matches the rate of data acquisition. Tests on both oxide and copper metal CMP show that the developed methodology is uniquely capable of identifying the start and finish of the end point event.

Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning

An important component of the cyber-defense mechanism is the adequate staffing levels of its cybersecurity analyst workforce and their optimal assignment to sensors for investigating the dynamic alert traffic. The ever-increasing cybersecurity threats faced by today’s digital systems require a strong cyber-defense mechanism that is both reactive in its response to mitigate the known risk and proactive in being prepared for handling the unknown risks. In order to be proactive for handling the unknown risks, the above workforce must be scheduled dynamically so the system is adaptive to meet the day-to-day stochastic demands on its workforce (both size and expertise mix). The stochastic demands on the workforce stem from the varying alert generation and their significance rate, which causes an uncertainty for the cybersecurity analyst scheduler that is attempting to schedule analysts for work and allocate sensors to analysts. Sensor data are analyzed by automatic processing systems, and alerts are generated. A portion of these alerts is categorized to be significant, which requires thorough examination by a cybersecurity analyst. Risk, in this article, is defined as the percentage of significant alerts that are not thoroughly analyzed by analysts. In order to minimize risk, it is imperative that the cyber-defense system accurately estimates the future significant alert generation rate and dynamically schedules its workforce to meet the stochastic workload demand to analyze them. The article presents a reinforcement learning-based stochastic dynamic programming optimization model that incorporates the above estimates of future alert rates and responds by dynamically scheduling cybersecurity analysts to minimize risk (i.e., maximize significant alert coverage by analysts) and maintain the risk under a pre-determined upper bound. The article tests the dynamic optimization model and compares the results to an integer programming model that optimizes the static staffing needs based on a daily-average alert generation rate with no estimation of future alert rates (static workforce model). Results indicate that over a finite planning horizon, the learning-based optimization model, through a dynamic (on-call) workforce in addition to the static workforce, (a) is capable of balancing risk between days and reducing overall risk better than the static model, (b) is scalable and capable of identifying the quantity and the right mix of analyst expertise in an organization, and (c) is able to determine their dynamic (on-call) schedule and their sensor-to-analyst allocation in order to maintain risk below a given upper bound. Several meta-principles are presented, which are derived from the optimization model, and they further serve as guiding principles for hiring and scheduling cybersecurity analysts. Days-off scheduling was performed to determine analyst weekly work schedules that met the cybersecurity system’s workforce constraints and requirements.

Estimating Taxi-out times with a reinforcement learning algorithm

Flight delays have a significant impact on the nationpsilas economy. Taxi-out delays in particular constitute a significant portion of the block time of a flight. In the future, it can be expected that accurate predictions of dasiawheels-offpsila time may be used in determining whether an aircraft can meet its allocated slot time, thereby fitting into an en-route traffic flow. Without an accurate taxi-out time prediction for departures, there is no way to effectively manage fuel consumption, emissions, or cost. Dynamically changing operations at the airport makes it difficult to accurately predict taxi-out time. This paper describes a method for estimating average taxi-out times at the airport in 15 minute intervals of the day and at least 15 minutes in advance of aircraft scheduled gate push-back time. A probabilistic framework of stochastic dynamic programming with a learning-based solution strategy called Reinforcement Learning (RL) has been applied. Historic data from the Federal Aviation Administrationpsilas (FAA) Aviation System Performance Metrics (ASPM) database were used to train and test the algorithm. The algorithm was tested on John F. Kennedy International airport (JFK), one of the busiest, challenging, and difficult to predict airports in the United States that significantly influences operations across the entire National Airspace System (NAS). Due to the nature of departure operations at JFK the prediction accuracy of the algorithm for a given day was analyzed in two separate time periods (1) before 4:00 P.M and (2) after 4:00 P.M. On an average across 15 days, the predicted average taxi-out times matched the actual average taxi-out times within plusmn5 minutes for about 65 % of the time (for the period before 4:00 P.M) and 53 % of the time (for the period after 4:00 P.M). The prediction accuracy over the entire day within plusmn5 minutes range of accuracy was about 60 %. Further, application of the RL algorithm to estimate taxi-out times at airports with multi-dependent static surface surveillance data will likely improve the accuracy of prediction. The implications of these results for airline operations and network flow planning are discussed.

Airport Taxi-Out Prediction Using Approximate Dynamic Programming: Intelligence-Based Paradigm

Flight delay is one of the pressing problems that have far-reaching effects on society and the nations economy. A primary cause of flight delay in the National Airspace System is high taxi-out times (time between gate push-back and wheels-off) at major airports. Accurate prediction of taxi-out time is needed to make downstream schedule adjustments and for better departure planning, which could mitigate delays, emissions, and congestion on the ground. However, accurate prediction of taxi-out time is difficult because of uncertainties associated with the dynamically changing airport operation. A novel stochastic approximation scheme based on reinforcement learning (RL) is presented for predicting taxi-out times in the presence of weather and other departure-related uncertainties. The prediction problem is cast in a probabilistic framework of stochastic dynamic programming and solved by using approximate dynamic programming approaches (particularly RL). The strengths of the method are that it is nonparametric, unlike regression models with fixed parameters, it is highly adaptable to the dynamic airport environment since it is learning based, it is scalable, it is inexpensive since it does not need highly sophisticated surface management systems, and it can effectively handle uncertainties because of the probabilistic framework. Taxi-out prediction performance was tested on data obtained from the FAA Aviation System Performance Metrics database on Detroit International and Washington Reagan National Airports. Results show that the root-mean-square prediction error calculated 15 min before gate departure time is on average 2.9 min for about 80% of the predicted flights.

A Multiresolution Analysis-Assisted Reinforcement Learning Approach to Run-by-Run Control

In recent years, the run-by run (RbR) control mechanism has emerged as a useful tool for keeping complex semiconductor manufacturing processes on target during repeated short production runs. Many types of RbR controllers exist in the literature of which the exponentially weighted moving average (EWMA) controller is widely used in the industry. However, EWMA controllers are known to have several limitations. For example, in the presence of multiscale disturbances and lack of accurate process models, the performance of EWMA controller deteriorates and often fails to control the process. Also, the control of complex manufacturing processes requires sensing of multiple parameters that may be spatially distributed. New control strategies that can successfully use spatially distributed sensor data are required. This paper presents a new multiresolution analysis (wavelet) assisted reinforcement learning (RL)-based control strategy that can effectively deal with both multiscale disturbances in processes and the lack of process models. The novel idea of a wavelet-aided RL-based controller represents a paradigm shift in the control of large-scale stochastic dynamic systems of which the control problem is a subset. Henceforth, we refer our new control strategy as a WRL-RbR controller. The WRL-RbR controller is tested on a multiple-input-multiple-output chemical mechanical planarization process of wafer fabrication for which the process model is available. Results show that the RL controller outperforms EWMA-based controllers for low autocorrelation. The new controller also performs quite well for strongly autocorrelated processes for which the EWMA controllers are known to fail. Convergence analysis of the new breed of the WRL-RbR controller is presented. Further enhancement of the controller to deal with model-free processes and for inputs coming from spatially distributed environments are also discussed. Note to Practitioners-This work was motivated by the need to develop an intelligent and efficient RbR process controller, especially for the control of processes with short production runs as in the case of the semiconductor manufacturing industry. A novel controller that is presented here is capable of generating optimal control actions in the presence of multiple time-frequency disturbances, and allows the use of realistic (often complex) process models without sacrificing robustness and speed of execution. Performance measures, such as reduction of variability in process output and control recipe, minimization of initial bias, and ability to control processes with high autocorrelations are shown to be superior in comparison to the commercially available exponentially weighted moving average controllers. The WRL-RbR controller is very generic, and can be easily extended to processes with drifts and sudden shifts in the mean and variance. The viability of extending the controller to distributed input parameter sensing environments, including those for which process models are not available, is also discussed

Optimal Scheduling of Cybersecurity Analysts for Minimizing Risk

Cybersecurity threats are on the rise with evermore digitization of the information that many day-to-day systems depend upon. The demand for cybersecurity analysts outpaces supply, which calls for optimal management of the analyst resource. Therefore, a key component of the cybersecurity defense system is the optimal scheduling of its analysts. Sensor data is analyzed by automatic processing systems, and alerts are generated. A portion of these alerts is considered to be significant, which requires thorough examination by a cybersecurity analyst. Risk, in this article, is defined as the percentage of unanalyzed or not thoroughly analyzed alerts among the significant alerts by analysts. The article presents a generalized optimization model for scheduling cybersecurity analysts to minimize risk (a.k.a., maximize significant alert coverage by analysts) and maintain risk under a pre-determined upper bound. The article tests the optimization model and its scalability on a set of given sensors with varying analyst experiences, alert generation rates, system constraints, and system requirements. Results indicate that the optimization model is scalable and is capable of identifying both the right mix of analyst expertise in an organization and the sensor-to-analyst allocation in order to maintain risk below a given upper bound. Several meta-principles are presented, which are derived from the optimization model, and they further serve as guiding principles for hiring and scheduling cybersecurity analysts. The simulation studies (validation) of the optimization model outputs indicate that risk varies non-linearly with an analyst/sensor ratio, and for a given analyst/sensor ratio, the risk is independent of the number of sensors in the system.

Static sectorization approach to dynamic airspace configuration using approximate dynamic programming

The National Airspace System (NAS) is an important and a vast resource. Efficient management of airspace capacity is important to ensure safe and systematic operation of the NAS eventually resulting in maximum benefit to the stakeholders. Dynamic Airspace Configuration (DAC) is one of the NextGen Concept of Operations (ConOps) that aims at efficient allocation of airspace as a capacity management technique. This paper is a proof of concept for the Approximate Dynamic Programming (ADP) approach to Dynamic Airspace Configuration (DAC) by static sectorization. The objective of this paper is to address the issue of static sectorization by partitioning airspace based on controller workload i.e. airspace is partitioned such that the controller workload is balanced between adjacent sectors. Several algorithms exist that address the issue of static restructuring of the airspace to meet capacity requirements on a daily basis. The intent of this paper is to benchmark the results of our methodology with the state-of-the-art algorithms and lay a foundation for future work in dynamic resectorization.

A Multiscale Bayesian SPRT Approach for Online Process Monitoring

Online monitoring of complex processes, such as semiconductor manufacturing processes, often requires the need to analyze sensor data with multiple characteristics. Some of these characteristics include nonstationary behavior, non-Gaussian distribution, high frequency of data generation, and multiscale (multiple frequencies) noise that mask the true nature of the process. Furthermore, it is necessary to implement process monitoring schemes that take into consideration the cost associated with sampling and incorrect decision making without sacrificing sensitivity, robustness, and ease of implementation. In this paper, a novel multiscale Bayesian sequential probability ratio test (MBSPRT) is developed, which is shown to be efficient in monitoring processes with the above characteristics. The MBSPRT method is also made suitable for online application by developing a moving block data processing strategy, which can match the data processing speed with the rate of data acquisition. The efficacy of the MBSPRT method was tested via detection of the end point occurrence in a chemical-mechanical planarization (CMP) process of semiconductor manufacturing using coefficient of friction (CoF) data. The proposed methodology offers a cost effective alternative to the traditional end point method, which is based on expensive metrology. Test results from both oxide and copper metal CMP are presented which show that MBSPRT is uniquely capable of identifying the start and finish of the end point event.