Thomas Popham

Road type classification through data mining

In this paper we investigate data mining approaches to road type classification based on CAN (controller area network) bus data collected from vehicles on UK roads. We consider three related classification problems: road type (A, B, C and Motorway), signage (None, White, Green and Blue) and carriageway type (Single or Double). Knowledge of these classifications has a number of uses, including tuning the engine and adapting the user interface according to the situation. Furthermore, the current road type and surrounding area gives an indication of the drivers workload. In a residential area the driver is likely to be overloaded, while they may be under stimulated on a highway. Several data mining and temporal analysis techniques are investigated, along with selected ensemble classifiers and initial attempts to deal with a class imbalance present in the data. We find that the Random Forest ensemble algorithm has the best performance, with an AUC of 0.89 when used with a wavelet-Gaussian summary of the previous 2.5 seconds of speed and steering wheel angle recordings. We show that this technique is at least as good as a model-based solution that was manually created using domain expertise.

Data Mining for Vehicle Telemetry

ABSTRACT This article presents a data mining methodology for driving-condition monitoring via CAN-bus data that is based on the general data mining process. The approach is applicable to many driving condition problems, and the example of road type classification without the use of location information is investigated. Location information from Global Positioning Satellites and related map data are often not available (for business reasons), or cannot represent the full dynamics of road conditions. In this work, Controller Area Network (CAN)-bus signals are used instead as inputs to models produced by machine learning algorithms. Road type classification is formulated as two related labeling problems: Road Type (A, B, C, and Motorway) and Carriageway Type (Single or Dual). An investigation is presented into preprocessing steps required prior to applying machine learning algorithms, that is, signal selection, feature extraction, and feature selection. The selection methods used include principal components analysis (PCA) and mutual information (MI), which are used to determine the relevance and redundancy of extracted features and are performed in various combinations. Finally, because there is an inherent bias toward certain road and carriageway labelings, the issue of class imbalance in classification is explained and investigated. A system is produced, which is demonstrated to successfully ascertain road type from CAN-bus data, and it is shown that the classification correlates well with input signals such as vehicle speed, steering wheel angle, and suspension height.

A comparison of features for regression-based driver head pose estimation under varying illumination conditions

Head pose estimation provides key information about driver activity and awareness. Prior comparative studies are limited to temporally consistent illumination conditions under the assumption of brightness constancy. By contrast the illumination conditions inside a moving vehicle vary considerably with environmental conditions. In this study we present a base comparison of three features for head pose estimation, via support vector machine regression, based on Histogram of Oriented Gradient (HOG) features, Gabor filter responses and Active Shape Model (ASM) landmark features. These, reputedly illumination invariant, are presented through a common face localization framework from which we estimate driver head pose in two degrees-of-freedom and compare against a baseline approach for recovering head pose via weak perspective geometry. Evaluation is performed over a number of invehicle sequences, exhibiting uncontrolled illumination variation, in addition to ground truth data-sets, with controlled illumination changes, upon which we achieve a minimal ~12° and ~15° mean error in pitch and yaw respectively via ASM landmark features.

Key Learning Features as Means for Terrain Classification

Modern vehicles seek autonomous subsystems adaptability to ever-changing terrain types in pursuit of enhanced drivability and maneuverability. The impact of key features on the classification accuracy of terrain types using a colour camera is investigated. A handpicked combination of texture and colour as well as a simple unsupervised feature representation is proposed. Although the results are restricted to only four classes {grass, tarmac, dirt, gravel} the learned features can be tailored to suit more classes as well as different scenarios altogether. The novel aspect stems from the feature representation itself as a global gist for three quantities of interest within each image: background, foreground and noise. In addition to that, the frequency affinity of the Gabor wavelet gist component to perspective images is mitigated by inverse homography mapping. The emphasis is thus on feature selection in an unsupervised manner and a framework for integrating learned features with standard off the shelf machine learning algorithms is provided. Starting with a colour hue and saturation histogram as fundamental building block, more complex features such as GLCM, k-means and GMM quantities are gradually added to observe their integrated effect on class prediction for three parallel regions of interest. The terrain classification problem is tackled with promising results using a forward facing camera.

Warwick-JLR driver monitoring dataset (DMD): statistics and early findings

Driving is a safety critical task that requires a high levels of attention and workload from the driver. Despite this, people often also perform secondary tasks such as eating or using a mobile phone, which increase workload levels and divert cognitive and physical attention from the primary task of driving. If a vehicle is aware that the driver is currently under high workload, the vehicle functionality can be changed in order to minimize any further demand. Traditionally, workload measurements have been performed using intrusive means such as physiological sensors. Another approach may be to monitor workload online using readily available and robust sensors accessible via the vehicles Controller Area Network (CAN). In this paper, we present details of the Warwick-JLR Driver Monitoring Dataset (DMD) collected for this purpose, and to announce its publication for driver monitoring research. The collection protocol is briefly introduced, followed by statistical analysis of the dataset to describe its structure. Finally, the public release of the dataset, for use in both driver monitoring and data mining research, is announced.

Modelling received signal strength from on-vehicle BLE beacons using skewed distributions: A preliminary study

This paper describes a study on modelling the Received Signal Strength Indicator (RSSI) measured by the smartphone of a vehicle user. The present transmissions are emitted by dedicated radio frequency sources, such as Bluetooth Low Energy (BLE) beacons, mounted to the vehicle to determine the driver/passenger(s) proximity or relative position(s). Based on empirical data, a model of the measurements noise, which utilises skewed distributions, is proposed to capture inconsistencies in reception and the impact of occlusions on the RSSI profile in an automotive setting, for example occlusions in car parks. Experimental data is used to demonstrate the suitability of the introduced model.

Deep Passenger State Monitoring Using Viewpoint Warping

The advent of autonomous and semi-autonomous vehicles has meant passengers now play a more significant role in the safety and comfort of vehicle journeys. In this paper, we propose a deep learning method to monitor and classify passenger state with camera data. The training of a convolutional neural network is supplemented by data captured from vehicle occupants in different seats and from different viewpoints. Existing driver data or data from one vehicle is augmented by viewpoint warping using planar homography, which does not require knowledge of the source camera parameters, and overcomes the need to re-train the model with large amounts of additional data. To analyse the performance of our approach, data is collected on occupants in two different vehicles, from different viewpoints inside the vehicle. We show that the inclusion of the additional training data and augmentation by homography increases the average passenger state classification rate by 11.1%. We conclude by proposing how occupant state may be used holistically for activity recognition and intention prediction for intelligent vehicle features.

Investigating the feasibility of vehicle telemetry data as a means of predicting driver workload

Driving is a safety critical task that requires a high level of attention and workload from the driver. Despite this, people often also perform secondary tasks such as eating or using a mobile phone, which increase workload levels and divert cognitive and physical attention from the primary task of driving. If a vehicle is aware that the driver is currently under high workload, the vehicle functionality can be changed in order to minimize any further demand. Traditionally, workload measurements have been performed using intrusive means such as physiological sensors. Another approach may be to use vehicle telemetry data as a performance measure for workload. In this paper, we present the Warwick-JLR Driver Monitoring Dataset (DMD) and analyse it to investigate the feasibility of using vehicle telemetry data for determining the driver workload. We perform a statistical analysis of subjective ratings, physiological data, and vehicle telemetry data collected during a track study. A data mining methodology is then presented to build predictive models using this data, for the driver workload monitoring problem.

Superpixel based semantic segmentation for assistance in varying terrain driving conditions

Vehicle drivability and maneuverability can be improved by increasing the environment awareness via sensory inputs. In particular, off-road capable vehicles possess subsystems which are configurable to the driving conditions. In this work, a vision solution is explored as a precursor to autonomous toggling between different operating modes. The emphasis is on selecting an appropriate response to transitions from one terrain type to another. Given a forward facing camera, images are partitioned into pixel subsets known as superpixels in order to be classified. The quality of this semantic segmentation is considered for classes such as {grass, tree, sky, tarmac, dirt, gravel, shrubs}. Colour and texture are combined together to form visual cues and address this image recognition problem with good segmentation results.

Estimating scene flow using an interconnected patch surface model with belief-propagation inference

This article presents a novel method for estimating the dense three-dimensional motion of a scene from multiple cameras. Our method employs an interconnected patch model of the scene surfaces. The interconnected nature of the model means that we can incorporate prior knowledge about neighbouring scene motions through the use of a Markov Random Field, whilst the patch-based nature of the model allows the use of efficient techniques for estimating the local motion at each patch. An important aspect of our work is that the method takes account of the fact that local surface texture strongly dictates the accuracy of the motion that can be estimated at each patch. Even with simple squared-error cost functions, it produces results that are either equivalent to or better than results from a method based upon a state-of-the-art optical flow technique, which uses well-developed robust cost functions and energy minimisation techniques.