Thomas W. Rogers

Automated X-ray image analysis for cargo security: Critical review and future promise

We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs- and security-related threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo.

Detection of concealed cars in complex cargo X-ray imagery using Deep Learning

BACKGROUND Non-intrusive inspection systems based on X-ray radiography techniques are routinely used at transport hubs to ensure the conformity of cargo content with the supplied shipping manifest. As trade volumes increase and regulations become more stringent, manual inspection by trained operators is less and less viable due to low throughput. Machine vision techniques can assist operators in their task by automating parts of the inspection workflow. Since cars are routinely involved in trafficking, export fraud, and tax evasion schemes, they represent an attractive target for automated detection and flagging for subsequent inspection by operators. OBJECTIVE Development and evaluation of a novel method for the automated detection of cars in complex X-ray cargo imagery. METHODS X-ray cargo images from a stream-of-commerce dataset were classified using a window-based scheme. The limited number of car images was addressed by using an oversampling scheme. Different Convolutional Neural Network (CNN) architectures were compared with well-established bag of words approaches. In addition, robustness to concealment was evaluated by projection of objects into car images. RESULTS CNN approaches outperformed all other methods evaluated, achieving 100% car image classification rate for a false positive rate of 1-in-454. Cars that were partially or completely obscured by other goods, a modus operandi frequently adopted by criminals, were correctly detected. CONCLUSIONS We believe that this level of performance suggests that the method is suitable for deployment in the field. It is expected that the generic object detection workflow described can be extended to other object classes given the availability of suitable training data.

Automated detection of cars in transmission X-ray images of freight containers

We present a method for automated car detection in xraytransmission images of freight containers. A random forest classifier was used to classify image sub-windows as “car” and “non-car” based on image features such as intensity and log-intensity, as well as local structures and symmetries as encoded by Basic Image Features (BIFs) and oriented Basic Image Features (oBIFs). The proposed approach was validated using a dataset of stream of commerce X-ray images. A car detection rate of 100% was achieved while maintaining a false alarm rate of 1.23%. Further reduction in false alarm rate, potentially at the cost of detection rate, was possible by tweaking the classification confidence threshold. This work establishes a framework for the automated classification of X-ray transmission cargo images and their content, paving the way towards the development of tools to assist custom officers faced with an ever increasing number of images to inspect.

Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines

We propose a framework for Threat Image Projection (TIP) in cargo transmission X-ray imagery. The method exploits the approximately multiplicative nature of X-ray imagery to extract a library of threat items. These items can then be projected into real cargo. We show using experimental data that there is no significant qualitative or quantitative difference between real threat images and TIP images. We also describe methods for adding realistic variation to TIP images in order to robustify Machine Learning (ML) based algorithms trained on TIP. These variations are derived from cargo X-ray image formation, and include: (i) translations; (ii) magnification; (iii) rotations; (iv) noise; (v) illumination; (vi) volume and density; and (vii) obscuration. These methods are particularly relevant for representation learning, since it allows the system to learn features that are invariant to these variations. The framework also allows efficient addition of new or emerging threats to a detection system, which is important if time is critical. We have applied the framework to training ML-based cargo algorithms for (i) detection of loads (empty verification), (ii) detection of concealed cars (ii) detection of Small Metallic Threats (SMTs). TIP also enables algorithm testing under controlled conditions, allowing one to gain a deeper understanding of performance. Whilst we have focused on robustifying ML-based threat detectors, our TIP method can also be used to train and robustify human threat detectors as is done in cabin baggage screening.

Reduction of wobble artefacts in images from mobile transmission X-ray vehicle scanners

Detector boom wobble in transmission X-ray vehicle scanners is an unpredictable and currently uncontrollable problem, which lowers the quality of captured X-ray images. We propose (i) a method for image correction which is able to correct for 70% of boom wobble error given estimates of boom wobble, and (ii) a method of wobble estimation, based on the fusion of instantaneous wobble estimates with previous estimates, which is robust against non-Gaussian X-ray beam cross-sections and approaches ground truth accuracy. The combination of the two approaches provides a method for the reduction of wobble artefacts in images. The two methods have good potential for application in analogous scenarios in medical imaging, radiation physics, laser science and biophysics.

Tackling the x-ray cargo inspection challenge using machine learning

The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate exploding com-merce volumes and increasingly stringent regulations. There is a pressing need to develop methods to automate parts of the inspection workflow, enabling expert operators to focus on a manageable number of high-risk images. To tackle this challenge, we developed a modular framework for automated X-ray cargo image inspection. Employing state-of-the-art machine learning approaches, including deep learning, we demonstrate high performance for empty container verification and specific threat detection. This work constitutes a significant step towards the partial automation of X-ray cargo image inspection.

Representation-learning for anomaly detection in complex x-ray cargo imagery

Existing approaches to automated security image analysis focus on the detection of particular classes of threat. However, this mode of inspection is ineffectual when dealing with mature classes of threat, for which adversaries have refined effective concealment techniques. Furthermore, these methods may be unable to detect potential threats that have never been seen before. Therefore, in this paper, we investigate an anomaly detection framework, at X-ray image patch-level, based on: (i) image representations, and (ii) the detection of anomalies relative to those representations. We present encouraging preliminary results, using representations learnt using convolutional neural networks, as well as several contributions to a general-purpose anomaly detection algorithm based on decision-tree learning.

A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery

Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called Small Metallic Threats (SMTs) hidden amongst legitimate goods inside a cargo container. We trained a CNN from scratch on data produced by a Threat Image Projection (TIP) framework that generates images with realistic variation to robustify performance. The system achieved 90% detection of containers that contained a single SMT, while raising 6% false positives on benign containers. The best CNN architecture used the raw high energy image (single-energy) and its logarithm as input channels. Use of the logarithm improved performance, thus echoing studies on human operator performance. However, it is an unexpected result with CNNs. In this work, we (i) investigate methods to exploit material information captured in dual-energy images, and (ii) introduce a new CNN training scheme that generates ‘spot-the-difference’ benign and threat pairs on-the-fly. To the best of our knowledge, this is the first time that CNNs have been applied directly to raw dual-energy X-ray imagery, in any field. To exploit dual-energy, we experiment with adapting several physics-derived approaches to material discrimination from the cargo literature, and introduce three novel variants. We hypothesise that CNNs can implicitly learn about the material characteristics of objects from the raw dual-energy images, and use this to suppress false positives. The best performing method is able to detect 95% of containers containing a single SMT, while raising 0.4% false positives on benign containers. This is a step change improvement in performance over our prior work

Transferring X-ray based automated threat detection between scanners with different energies and resolution

A significant obstacle to developing high performance Deep Learning algorithms for Automated Threat Detection (ATD) in security X-ray imagery, is the difficulty of obtaining large training datasets. In our previous work, we circumvented this problem for ATD in cargo containers, using Threat Image Projection and data augmentation. In this work, we investigate whether data scarcity for other modalities, such as parcels and baggage, can be ameliorated by transforming data from one domain so that it approximates the appearance of another. We present an ontology of ATD datasets to assess where transfer learning may be applied. We define frameworks for transfer at the training and testing stages, and compare the results for both methods against ATD where a common data source is used for training and testing. Our results show very poor transfer, which we attribute to the difficulty of accurately matching the blur and contrast characteristics of different scanners.