Dustin Garvey

Development and Application of Fault Detectability Performance Metrics for Instrument Calibration Verification and Anomaly Detection

Traditionally, the calibration of safety critical nuclear instrumentation has been performed during each refueling outage. However, many nuclear plants have moved toward conditiondirected rather than time-directed calibration. This condition-directed calibration is accomplished through the use of on-line monitoring. On-line monitoring (OLM) commonly uses an autoassociative empirical modeling architecture to assess instrument channel performance. An autoassociative architecture predicts a group of correct sensor values when supplied a group of sensor values that is usually corrupted with process and instrument noise, and could also contain faults such as sensor drift or complete failure. This paper describes one such autoassociative model architecture, specifically autoassociative kernel regression (AAKR), and presents five metrics that may be used to evaluate performance. These metrics include the previously developed accuracy, auto sensitivity and cross sensitivity metrics along with a description of two new fault detectability performance metrics for application to instrument calibration verification (ICV) and anomaly detection. These parameters are calculated for an AAKR model of an operating nuclear power plant steam system and were used to describe the effects of model architecture on performance. It is shown that the ability of an empirical model to detect sensor faults in ICV systems is largely dependent on the model uncertainty and to a lesser degree its auto sensitivity. It is also shown that the ability of an empirical model to detect anomalies via the Sequential Probability Ratio Test (SPRT) is also related to uncertainty and the SPRT detectability is on the order of 50% smaller than the ICV detectability. These guidelines provide a framework for developing various models, in that models intended to be applied to ICV and anomaly detection tasks should focus on the minimization of uncertainty. Furthermore, the ICV and anomaly detection performance metrics are shown to be within the traditional +/-1% calibration tolerance and their performance under artificially faulted conditions are shown to be in direct agreement with their theoretical foundations.

Process and equipment monitoring methodologies applied to sensor calibration monitoring

Traditionally, the calibration of safety critical nuclear instrumentation has been performed at each refueling cycle. However, many nuclear plants have moved toward condition-directed rather than time-directed calibration. This condition-directed calibration is accomplished through the use of on-line monitoring. On-line monitoring (OLM) uses an autoassociative empirical modeling architecture to assess instrument channel performance. An autoassociative architecture predicts a group of correct sensor values when supplied a group of sensor values that is usually corrupted with process and instrument noise, and could also contain faults such as sensor drift or complete failure. The Process and Equipment Monitoring (PEM) Toolbox, which was developed at The University of Tennessee, is a set of MATLAB based tools, which have been developed to support the design of process and equipment condition monitoring systems. Its purpose is to provide the necessary functionality so that different empirical modeling and uncertainty estimation methods may be investigated and compared. This paper discusses the purpose and architecture of the PEM Toolbox, as well as presenting an example application to power plant sensor calibration monitoring using actual plant data. Copyright

Verification of helical tomotherapy delivery using autoassociative kernel regression.

Quality assurance (QA) is a topic of major concern in the field of intensity modulated radiation therapy (IMRT). The standard of practice for IMRT is to perform QA testing for individual patients to verify that the dose distribution will be delivered to the patient. The purpose of this study was to develop a new technique that could eventually be used to automatically evaluate helical tomotherapy treatments during delivery using exit detector data. This technique uses an autoassociative kernel regression (AAKR) model to detect errors in tomotherapy delivery. AAKR is a novel nonparametric model that is known to predict a group of correct sensor values when supplied a group of sensor values that is usually corrupted or contains faults such as machine failure. This modeling scheme is especially suited for the problem of monitoring the fluence values found in the exit detector data because it is able to learn the complex detector data relationships. This scheme still applies when detector data are summed over many frames with a low temporal resolution and a variable beam attenuation resulting from patient movement. Delivery sequences from three archived patients (prostate, lung, and head and neck) were used in this study. Each delivery sequence was modified by reducing the opening time for random individual multileaf collimator (MLC) leaves by random amounts. The error and error-free treatments were delivered with different phantoms in the path of the beam. Multiple autoassociative kernel regression (AAKR) models were developed and tested by the investigators using combinations of the stored exit detector data sets from each delivery. The models proved robust and were able to predict the correct or error-free values for a projection, which had a single MLC leaf decrease its opening time by less than 10msec. The model also was able to determine machine output errors. The average uncertainty value for the unfaulted projections ranged from 0.4% to 1.8% of the detector signal. The low model uncertainty indicates that the AAKR model is extremely accurate in its predictions and also suggests that the model may be able to detect errors that cause the fluence to change by less than 2%. However, additional evaluation of the AAKR technique is needed to determine the minimum detectable error threshold from the compressed helical tomotherapy detector data. Further research also needs to explore applying this technique to electronic portal imaging detector data.

An Integrated Fuzzy Inference-based Monitoring, Diagnostic, and Prognostic System for Intelligent Control and Maintenance

With the advent of modern computation, intelligent control and maintenance systems have become a viable option for complex engineering processes and systems. Such control and maintenance systems can be generically described as being composed of 5 analysis steps: (1) predict the expected system signals from their measured values, (2) use the residual of the measured and predicted value to determine if the system is operating in a nominal or a degraded mode, (3) if the system is operating in a degraded mode, diagnose the fault, (4) prognose the failure by estimating the remaining useful life (RUL) of the system, and (5) use the collected information to determine if an appropriate control or maintenance action should be performed to maintain the health and safety of the system performance. This chapter presents the development and adaptation of a single generic inference procedure, namely the nonparametric fuzzy inference system (NFIS), for monitoring, diagnostics, and prognostics. To illustrate the proposed methodologies, the embodiments of the NFIS are used to detect, diagnose, and prognose faults in the steering system of an automated oil drill. The embodiments of the NFIS were found to have similar performance to traditional algorithms, such as autoassociative kernel regression (AAKR) and k-nearest neighbor (kNN), for monitoring and diagnosis. The NFIS prognoser was also shown to estimate the remaining useful life of the steering system to within an hour of its actual time of failure.

SU‐GG‐AUD‐01: Exit Dosimetry Treatment Verification Using Auto‐Associative Kernel Regression

Purpose: The purpose of this work was to develop a novel technique for automatically evaluating exit dosimetry on tomotherapy systems using auto‐associative modeling that is robust and has the capability to learn complex detector data relationships, even with detector data with a low temporal resolution and beam attenuation from the patient. Method and Materials: Delivery sequences from 3 patients were used in this study. Each delivery sequence was modified by reducing the opening time for random individual MLC leaves by random amounts. The error and error‐free treatments were delivered with different phantoms in the path of the beam. Multiple auto‐associative kernel regression (AAKR) models were developed and tested by the investigators using combinations of the sinogramdata sets.AAKR is a non‐parametric model that is used to predict correct values when supplied a group of sensor values that is corrupted. Models were tested using the data containing errors. However, models were never developed with data which had the same object in the path of the beam as the dataset it was testing. This allowed the testing of the models error detection capabilities in the presence of attenuation. Results: The results show that the model correctly distinguished the MLC positional error from changes in attenuation. The model identified errors in compressed detector data that had been summed over 94 frames. Generally, errors greater than 7 milliseconds were visually discernable. Some smaller errors could be detected, but it depended on the position of the erroneous leaf in the projection and the actual projection shape. Conclusion: The results presented suggest that AAKR modeling could be used to monitor and eventually improve the reliability of radiation delivery. This method has the potential to play a noteworthy role in determining and possibly correcting for the types of machine‐related errors that occur during actual patient treatments.

TU‐C‐M100J‐10: Respiration Motion Prediction Using Time‐Delay Kernel Regression Modeling

Purpose: The purpose of this study was to develop a novel technique for dynamically predicting respiration motion and uncertainty up to 1.5 seconds in the future in real‐time using Time‐Delay Kernel Regression (TDKR) modeling. Unlike neural network based prediction techniques, kernel regression models are continuously learning new respiration cycles for each patient without the need for computationally and time intensive re‐training. Method and Materials: Recorded respiration data from a real‐time respiratory gating system was used to develop a model to predict the amplitude of the marker block at a future time. An empirical TDKR model was designed to compensate for the latency that occurs between the acquisition of the image and the point at which the beam actually turns on/off in beam tracking systems. This non‐parametric model incorporates the temporal information present in the input data. The model was tested using respiration data from 4 patients. Results: The rootmean squared error (RMSE) between the model predictions and the measured data was computed for each patient at different latencies, and then the average was taken over all the patients. For predictions 1.5 seconds into the future the average RMSE was 1.4%. For predictions 1 second into the future, the RMSE dropped to 1.2%, and for 0.5 seconds it was only 0.7%. The average uncertainty for the predictions at 0.5, 1, and 1.5 seconds into the future was 2.4%, 3.2%, and 3.4%, respectively. Conclusion: This study proves that a TDKR model can learn the relationships present in respiration data. The reported results showed that the TDKR model has the same, if not better predictive performance, as previously studied parametric models. However, because TDKR is non‐parametric, it has several distinct advantages over these models that make it more suited for respiratory gating applications.

SU‐DD‐A3‐01: Kernel Classification for Assessing Inter‐Fraction Motion in IGRT

Purpose: To develop a method that identifies an IGRTimaging session as either normal or problematic based solely on the amount of right‐left, anterior‐posterior, and superior‐inferior repositioning of the patient over the treatment session. Methods and Materials: A retrospective data set containing over 1100 anterior‐posterior, right‐left lateral, and superior‐inferior patient shift values for 29 prostate patients was examined using a non‐parametric kernel regression classification method to determine if a patient was “normal” or “problematic.” The treatment sessions were grouped as either being “normal”, or affected because they were “overweight”, or had “rectal filling”, or were both “overweight and had rectal filling”. In kernel regression, constants are fitted using a locally weighted criterion. The basis of kernel regression is to estimate a response using a weighted average of points, in a training set, which are local to the query point. A bandwidth is used to determine the definition of local. Leave one out cross validation (LOOCV) was used to select the optimal bandwidth and also evaluate the techniques performance. Results: The method correctly classified 24 of the 29 patients using their respective shift data sets, with four of the misclassifications occurring when the technique correctly identified non‐normal datasets, but assigned them to the wrong problem group. Only one patient was classified as normal incorrectly. Conclusion: Using readily accessible shift data, the kernel regression classification method was able to correctly identify the cause behind IGRT positioning problems for individual prostate patients. This technique is fully automated and can be implemented on a treatment planning computer to determine the reason a patient is having positioning errors early during treatment.