Burak Uzkent

Feature Matching and Adaptive Prediction Models in an Object Tracking DDDAS

We consider the optical remote sensing tracking problem for vehicles in a complex environment using an adaptive sensor that can take spectral data at a small number of locations. The Dynamic Data-Driven Applications Systems (DDDAS) paradigm is well-suited for dynamically controlling such an adaptive sensor by using the prediction of object movement and its interaction with the environment to guide the location of spectral measurements. The spectral measurements are used for target identification through feature matching. We consider several adaptive sampling strategies for how to assign locations for spectral measurements in order to distinguish between multiple targets. In addition to guiding the measurement process, the tracking system pulls in additional data from OpenStreetMap to identify road networks and intersections. When a vehicle enters a detected intersection, it triggers the use of a multiple model prediction system to sample all possible turning options. The result of this added information is more accurate predictions and analysis from data assimilation using a Gaussian Sum filter (GSF).

Feature Matching With an Adaptive Optical Sensor in a Ground Target Tracking System

We consider methods to address the optical feature-aided remote sensing tracking problem for vehicles in a challenging environment. Our approach is to apply the dynamic data driven application systems computing paradigm to implement control of an adaptive sensor. This adaptive sensor acquires a panchromatic image while simultaneously allowing the collection of visible-near infrared spectral data at specified pixels. This sensor holds the promise of delivering the increased accuracy of targeted spectral sensing without the enormous data volume of full spectral images. The target of interest is optimally imaged by the sensor based on the targets forecasted location and motion relative to the extracted content of the background. Background context is both extracted from the image and created from the OpenStreetMap road network. We describe the implementation of the tracking framework and testing of some of the components using simulated imagery created with the digital imaging and remote sensing image generation model. The Gaussian sum filter is employed to solve the data assimilation problem by forming a multimodel forecasting set that is used to increase the robustness and flexibility of tracking. For feature matching, we create an efficient sampling strategy that is informed by the viewing conditions to adaptively determine which pixels to measure spectrally in order to distinguish between different targets using a spectral distance measure.

Real-Time Vehicle Tracking in Aerial Video Using Hyperspectral Features

Vehicle tracking from a moving aerial platform poses a number of unique challenges including the small number of pixels representing a vehicle, large camera motion, and parallax error. This paper considers a multi-modal sensor to design a real-time persistent aerial tracking system. Wide field of view (FOV) panchromatic imagery is used to remove global camera motion whereas narrow FOV hyperspectral image is used to detect the target of interest (TOI). Hyperspectral features provide distinctive information to reject objects with different reflectance characteristics from the TOI. This way the density of detected vehicles is reduced, which increases tracking consistency. Finally, we use a spatial data based classifier to remove spurious detections. With such framework, parallax effect in non-planar scenes is avoided. The proposed tracking system is evaluated in a dense, synthetic scene and outperforms other state-of-theart traditional and aerial object trackers.

Spectral Validation of Measurements in a Vehicle Tracking DDDAS

Abstract Vehicle tracking in adverse environments is a challenging problem because of the high number of factors constraining their motion and possibility of frequent occlusion. In such conditions, identification rates drop dramatically. Hyperspectral imaging is known to improve the robustness of target identification by recording extended data in many wavelengths. However, it is impossible to transmit such a high rate data in real time with a conventional full hyperspectral sensor. Thus, we present a persistent ground-based target tracking system, taking advantage of a state-of-the-art, adaptive, multi-modal sensor controlled by Dynamic Data Driven Applications Systems (DDDAS) methodology. This overcomes the data challenge of hyperspectral tracking by only using spectral data as required. Spectral features are inserted in a feature matching algorithm to identify spectrally likely matches and simplify multidimensional assignment algorithm. The sensor is tasked for spectra acquisition by the prior estimates from the Gaussian Sum Filter and foreground mask generated by the background subtraction. Prior information matching the target features is used to tackle false negatives in the background subtraction output. The proposed feature-aided tracking system is evaluated in a challenging scene with a realistic vehicular simulation.

Integrating Hyperspectral Likelihoods in a Multidimensional Assignment Algorithm for Aerial Vehicle Tracking

Tracking vehicles through dense environments is an important and challenging task that is mostly tackled using visible and near IR wavelengths. Hyperspectral imaging is known to improve the robustness of target identification, but the massive increase in data created is usually prohibitive for tracking many targets. We present a persistent real-time aerial target tracking system, taking advantage of an adaptive, multimodal sensor concept and blending the hyperspectral likelihoods with kinematic likelihoods in a multidimensional assignment framework. The adaptive sensor is capable of providing wide field of view panchromatic images as well as the spectra of small number of pixels. The proposed system does not require large amount of hyperspectral data collection as we focus on tracking fewer number of targets with higher persistency. This overcomes the data challenge of hyperspectral tracking by following dynamic data-driven application systems (DDDAS) principles to control hyperspectral data collection where most beneficial. The DDDAS framework for controlling hyperspectral data collection is developed by incorporating prior information from the filter movement predictions and information from motion detection. The proposed multidimensional hyperspectral feature-aided tracker is compared to a 2-D hyperspectral feature-aided tracker and another cascaded hyperspectral data based tracker by generating a synthetic, realistic, aerial video on a dense scene.

Efficient integration of spectral features for vehicle tracking utilizing an adaptive sensor

Object tracking in urban environments is an important and challenging problem that is traditionally tackled using visible and near infrared wavelengths. By inserting extended data such as spectral features of the objects one can improve the reliability of the identification process. However, huge increase in data created by hyperspectral imaging is usually prohibitive. To overcome the complexity problem, we propose a persistent air-to-ground target tracking system inspired by a state-of-the-art, adaptive, multi-modal sensor. The adaptive sensor is capable of providing panchromatic images as well as the spectra of desired pixels. This addresses the data challenge of hyperspectral tracking by only recording spectral data as needed. Spectral likelihoods are integrated into a data association algorithm in a Bayesian fashion to minimize the likelihood of misidentification. A framework for controlling spectral data collection is developed by incorporating motion segmentation information and prior information from a Gaussian Sum filter (GSF) movement predictions from a multi-model forecasting set. An intersection mask of the surveillance area is extracted from OpenStreetMap source and incorporated into the tracking algorithm to perform online refinement of multiple model set. The proposed system is tested using challenging and realistic scenarios generated in an adverse environment.

Aerial Vehicle Tracking by Adaptive Fusion of Hyperspectral Likelihood Maps

Hyperspectral cameras provide unique spectral signatures that can be used to solve surveillance tasks. This paper proposes a novel real-time hyperspectral likelihood maps-aided tracking method (HLT) inspired by an adaptive hyperspectral sensor. We focus on the target detection part of a tracking system and remove the necessity to build any offline classifiers and tune large amount of hyper-parameters, instead learning a generative target model in an online manner for hyperspectral channels ranging from visible to infrared wavelengths. The key idea is that our adaptive fusion method can combine likelihood maps from multiple bands of hyperspectral imagery into one single more distinctive representation increasing the margin between mean value of foreground and background pixels in the fused map. Experimental results show that the HLT not only outperforms all established fusion methods but is on par with the current state-of-the-art hyperspectral target tracking frameworks.

3-D MRI cardiac segmentation using graph cuts

We consider the use of graph cuts technique to efficiently segment the full tissue volume of the entire heart or important parts of it such as ventricles in Magnetic Resonance Imaging (MRI) scans for different species. With the segmented 3-D volume of the heart, simulations of electrical waves propagating through the tissue can be done. The modeled wave results can then be compared directly to the experimental data. We use the graph cuts technique to achieve segmentation while letting the user initiate some of the energy cost function parameters. A Mumford-Shah type energy cost function is used. In addition to the data and smoothness terms, two spatial distance terms are incorporated to improve segmentation results. The user has the option of eliminating these terms in case the input data does not contain any useful spatial distance content.

Background image understanding and adaptive imaging for vehicle tracking

We describe our effort to create an imaging-based vehicle tracking system that uses the principles of dynamic data driven applications systems to observe, model, and collect new within a dynamic feedback loop. Several unique aspects of the system include tracking of user-defined vehicles, the use of an adaptive sensor that can change modality, and a reliance on background image understanding to improve tracking and minimize error. We describe the system and show results demonstrated within the DIRSIG image simulation model that show improved tracking results for the system.