Amir Shirkhodaie

LabVIEW program design for on-line data acquisition and predictive maintenance

The online design for data acquisition and predictive maintenance for a fan-motor system using the graphical programming language, LabVIEW is presented. The data set were created for different faults under varying rpm levels from the synthetically generated fault data and the normal base line data acquired from the fan-motor system. The data were processed and the extracted features were fed to a two layer backpropagation neural network. The design is to be implemented on an online basis.

A survey of imagery techniques for semantic labeling of human-vehicle interactions in persistent surveillance systems

Understanding and semantic annotation of Human-Vehicle Interactions (HVI) facilitate fusion of Hard sensor (HS) and Human Intelligence (HUMINT) in a cohesive way. By characterization, classification, and discrimination of HVI patterns pertinent threats may be realized. Various Persistent Surveillance System (PSS) imagery techniques have been proposed in the past decade for identifying human interactions with various objects in the environment. Understanding of such interactions facilitates to discover human intentions and motives. However, without consideration of incidental context, reasoning and analysis of such behavioral activities is a very challenging and difficult task. This paper presents a current survey of related publications in the area of context-based Imagery techniques applied for HVI recognition, in particular, it discusses taxonomy and ontology of HVI and presents a summary of reported robust image processing techniques for spatiotemporal characterization and tracking of human targets in urban environments. The discussed techniques include model-based, shape-based and appearance-based techniques employed for identification and classification of objects. A detailed overview of major past research activities related to HVI in PSS with exploitation of spatiotemporal reasoning techniques applied to semantic labeling of the HVI is also presented.

Cooperative visual team working and target tracking of mobile robots

The problem of visual coordination and target tracking mobile robots cooperating in an unstructured environment is addressed. In this paper, we consider a team of semi-autonomous robots controlled by a remote supervisory control system. We present an algorithm for visual position tracking of individual cooperative robots within their working environment. Initially, we present a technique suitable for visual servoing of a robot toward its landmark targets. Then, we present an image-processing technique that utilizes images from a remote surveillance camera for localization of the robots within the operational environment. In this algorithm, the surveillance camera can be either stationary or mobile. The supervisor control system keeps tracks of relative locations of individual robots and utilizes relative coordinates information of the robots to coordinate their cooperative activities. We present some results of this research effort that illustrates the effectiveness of the proposed algorithms for cooperative robotic systems visual team working and target tracking.

Context-based semantic labeling of human-vehicle interactions in persistent surveillance systems

The improved Situational awareness in Persistent Surveillance Systems (PSS) is an ongoing research effort of the Department of Defense. Most PSS generate huge volume of raw data and they heavily rely on human operators to interpret and inference data in order to detect potential threats. Many outdoor apprehensive activities involve vehicles as their primary source of transportation to and from the scene where a plot is executed. Vehicles are employed to bring in and take out ammunitions, supplies, and personnel. Vehicles are also used as a disguise, hide-out, a meeting place to execute threat plots. Analysis of the Human-Vehicle Interactions (HVI) helps us to identify cohesive patterns of activities representing potential threats. Identification of such patterns can significantly improve situational awareness in PSS. In our approach, image processing technique is used as the primary source of sensing modality. We use HVI taxonomy as a means for recognizing different types of HVI activities. HVI taxonomy may comprise multiple threads of ontological patterns. By spatiotemporal linking of ontological patterns, a HVI pattern is hypothesized to pursue a potential threat situation. The proposed technique generates semantic messages describing ontology of HVI. This paper also discusses a vehicle zoning technique for HVI semantic labeling and demonstrates efficiency and effectiveness of the proposed technique for identifying HVI.

Vehicle identifications using acoustic sensing

Increase in the complexity of battlefield fought in urban areas has brought a high demand for efficient techniques for vehicles detection, classification, identification and tracking in areas of interest. The demand for such efficient techniques is due to complexity of the environment and to sensor limitations. Multi-sensor data fusion can be used for vehicle identification in practical applications such as battlefield surveillance. Multi-sensor data fusion provides significant advantages over single sensor data source. The purpose of this research is to design and implement data fusion software for vehicle surveillance applications using a distributed network of acoustic sensors. The data fusion software will be used in vehicles traffic monitoring in a battlefield and events detection and tracking in secured areas. Implementation and testing results of the developed software are obtained from real data collect from civilian and military vehicles.

Soft computing for visual terrain perception and traversability assessment by planetary robotic systems

This paper discusses technical challenges and navigational skill requirements of mobile robots for traversable path planning in natural environments similar to Mars surface terrains. Different methods for detecting salient terrain features based on imaging texture analysis techniques are described. In particular, three competing soft computing techniques are presented for terrain traversability assessment: a rule-based terrain classifier, a neural network-based terrain classifier, and a fuzzy-logic terrain classifier. Each terrain classifier divides a region of natural terrain into finite sub-terrain regions and classifies terrain condition exclusively within each sub-terrain region based on terrain visual clues. Image processing techniques are applied for aggregative fusion of sub-terrain assessment results. Results of a comparative performance evaluation of all three terrain classifiers are presented. The last two terrain classifiers are shown to have remarkable capability for traversability assessment, which facilitates navigation in unstructured natural terrain environments.

Visual terrain mapping for traversable path planning of mobile robots

In this paper, we have primarily discussed technical challenges and navigational skill requirements of mobile robots for traversability path planning in natural terrain environments similar to Mars surface terrains. We have described different methods for detection of salient terrain features based on imaging texture analysis techniques. We have also presented three competing techniques for terrain traversability assessment of mobile robots navigating in unstructured natural terrain environments. These three techniques include: a rule-based terrain classifier, a neural network-based terrain classifier, and a fuzzy-logic terrain classifier. Each proposed terrain classifier divides a region of natural terrain into finite sub-terrain regions and classifies terrain condition exclusively within each sub-terrain region based on terrain visual clues. The Kalman Filtering technique is applied for aggregative fusion of sub-terrain assessment results. The last two terrain classifiers are shown to have remarkable capability for terrain traversability assessment of natural terrains. We have conducted a comparative performance evaluation of all three terrain classifiers and presented the results in this paper.

Supervised control of cooperative multi-agent robotic vehicles

In this paper, we address some of technical challenges associated with supervised tactical mobility behaviors modeling and control of a group of cooperative multi-agent robotic vehicles. Cooperative task planning of robotic systems is a dynamic and complex problem and very challenging. The challenge comes from many avenues including task decomposition, assignment, resource allocation, and task execution and monitoring. Furthermore, the behavior-based control system requires a deliberative and reactive task planning capability for execution of cooperative tasks of mobile robots as well as adequate adaptability to learn new cooperative schemes to improve system performance in over-time. Another challenge of this problem is associated with sensory data gathering, distributed data processing, centralized/decentralized data fusion, and intelligent world perception modeling and comprehension of cooperative mobile robots. We propose a multilayered supervisory control architecture for coordinated task planning of a group of multi-agent robots. The supervisory controller allows control of robotic vehicles in either teleoperation, semi-autonomous, and/or autonomous modes.

Acoustic semantic labeling and fusion of human-vehicle interactions

Situational awareness in a Persistent Surveillance System (PSS) can be significantly improved by fusion of Data from physical (Hard) sensors and information provided by human observers (as Soft/biological sensors) from the field. One of the major limitations that this trend brings about is, however, the integration and fusion of the sensory data collected from hard sensors along with soft data gathered from human agents in a consistent and cohesive way. This paper presents a proposed approach for semantic labeling of vehicular non-stationary acoustic events in the context of PSS. Two techniques for feature extraction based on discrete wavelet and short-time Fourier transforms are described. A correlation-based classifier is proposed for classifying and semantic labeling of vehicular acoustic events. The presented result demonstrates the proposed solution is both reliable and effective, and can be extended to future PSS applications.

Multiple Target Vehicles Detection and Classification with Low-Rank Matrix Decomposition

Considerable interest has arisen in the recent years utilizing inexpensive acoustic sensors in the battlefield to perform targets of interest identification and classification. They require no line of sight and provide many capabilities for target detection, bearing estimation, target tracking, classification and identification. In practice, however, many environment noise, time-varying, and uncertainties factors affect their performance in detecting targets of interest reliably and accurate. In this paper, we have proposed a novel feature extraction approach for robust classification and identification of moving target vehicles to reduce those factors. The approach is based on Low Rank Matrix Decomposition. Using Low Rank Matrix Decomposition, dominant features of vehicle acoustic signatures can be extracted appropriately with respect to vehicle operational responses and used for robust identification and classification of target vehicles. The performance of the proposed approach has been evaluated based on a set of experimental acoustic data from multiple vehicle test-runs. It is demonstrated that the approach yields verv promising results to reduce uncertainties associated with classification of target vehicles based on acoustic signatures at different operation speeds in the field.