Dragan Obradovic

An Information-Theoretic Approach to Neural Computing

From the Publisher: Neural networks provide a powerful new technology to model and control nonlinear and complex systems. In this book, the authors present a detailed formulation of neural networks from the information-theoretic viewpoint. They show how this perspective provides new insights into the design theory of neural networks. In particular, they show how these methods may be applied to the topics of supervised and unsupervised learning, including feature extraction, linear and nonlinear independent component analysis, and Boltzmann machines. Readers are assumed to have a basic understanding of neural networks, but all of the relevant concepts from information theory are carefully introduced and explained. Consequently, readers from several different scientific disciplines - notably, cognitive scientists, engineers, physicists, statisticians, and computer scientists - will find this book to be a very valuable contribution to this topic.

Signal Processing Techniques for Knowledge Extraction and Information Fusion

This book brings together the latest research achievements from various areas of signal processing and related disciplines in order to consolidate the existing and proposed new directions in DSP based knowledge extraction and information fusion. Within the book contributions presenting both novel algorithms and existing applications, especially those (but not restricted to) on-line processing of real world data are included. The areas of Knowledge Extraction and Information Fusion are naturally linked and aim at detecting and estimating the signal of interest and its parameters, and further at combining measurements from multiple sensors (and associated databases if appropriate) to achieve improved accuracies and more specific inferences which cannot be achieved by using only a single signal modality. The subject therefore is of major interest for modern biomedical, environmental, and industrial applications to provide a state of the art and propose new techniques in order to combine heterogeneous information sources.

Fusion of Sensor Data in Siemens Car Navigation System

Car navigation systems have three main tasks, namely 1) positioning; 2) routing; and 3) navigation (guidance). Positioning of the car is carried out by appropriately combining information from several sensors and information sources, including odometers, gyroscopes, Global Positioning System (GPS) information, and digital maps. This paper describes two sensor-fusion steps implemented in commercial Siemens car navigation systems. The first step is the fusion of the odometer, gyroscope, and GPS sensory information. The dynamic model of the car movement is implemented in a Kalman filter, which relays the GPS signal as a teacher. In the second step, the available digital map is used to find the most likely position on the roads. Contrary to the standard application of the digital map, where the current estimated car position is just projected on the road map, the approach presented here compares the features of the integrated vehicle path with the features of the candidate roads from the digital map. In addition, this paper presents the results of the experimental drives. The developed car navigation system was awarded the best car navigation system among ten competing systems in 2002 by the Auto Build magazine

Synchronization Performance of the Precision Time Protocol in Industrial Automation Networks

This paper analyzes the factors that affect the synchronization performance in peer-to-peer precision time protocol (PTP). We first study the influence of frequency drift in the absence of jitter and compare the gravity of the master drift with that of the slave drift. Then, we study the influence of jitter under the assumption of constant frequencies and the effect of averaging. The analytic formulas provide a theoretical ground for understanding the simulation results, some of which are presented, as well as the guidelines for choosing both system and control parameters.

Data fusion for modern engineering applications: an overview

An overview of data fusion approaches is provided from the signal processing viewpoint. The general concept of data fusion is introduced, together with the related architectures, algorithms and performance aspects. Benefits of such an approach are highlighted and potential applications are identified. Case studies illustrate the merits of applying data fusion concepts in real world applications.

Fusion of Map and Sensor Data in a Modern Car Navigation System

The main tasks of car navigation systems are positioning, routing, and guidance. This paper describes a novel, two-step approach to vehicle positioning founded on the appropriate combination of the in-car sensors, GPS signals, and a digital map. The first step is based on the application of a Kalman filter, which optimally updates the model of car movement based on the in-car odometer and gyroscope measurements, and the GPS signal. The second step further improves the position estimate by dynamically comparing the continuous vehicle trajectory obtained in the first step with the candidate trajectories on a digital map. This is in contrast with standard applications of the digital map where the current position estimate is simply projected on the digital map at every sampling instant.

Information maximization and independent component analysis: is there a difference?

This article provides a detailed and rigorous analysis of the two commonly used methods for redundancy reduction: linear independent component analysis (ICA) posed as a direct minimization of a suitably chosen redundancy measure and information maximization (InfoMax) of a continuous stochastic signal transmitted through an appropriate nonlinear network. The article shows analytically that ICA based on the Kullback-Leibler information as a redundancy measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance. The practical issues of applying ICA and InfoMax are also discussed and illustrated on the problem of extracting statistically independent factors from a linear, pixel-by-pixel mixture of images.

Initialization and Online-Learning of RSS Maps for Indoor / Campus Localization

Common approaches for indoor positioning based on cellular communication systems use as measurements the received signal strength (RSS). In order to work properly, such a system often requires many calibration points before its start. This paper presents a two-fold approach achieving high indoor localization accuracies without requiring too many calibration points. The basic idea is to use an initial propagation model with few parameters, which can be adapted by a few measurements, e.g. mutual measurements of access points. Then the model is refined by incorporating additional parameters and using online learning. Investigations on the requirements and potentials of different approaches and results for DECT and WLAN setups are given. The first approach uses predefined paths that should be passed through by a service technician with measurement equipment. The second approach uses a Kohonen-like learning algorithm to adapt the model on-the-fly. For both approaches linear propagation models and more involved dominant path models incorporating map information are applied for the initialization.

Synchronization Performance of the Precision Time Protocol

This paper analyzes the factors that affect the synchronization performance in using the precision time protocol (FTP). We first study the influence of jitter under the assumption of no frequency drifts. Then we study the influence of frequency drift in the absence of jitter. The analytic formulas provide a theoretical ground for the understanding of simulation results as well as guidelines for choosing both system and control parameters when applying FTP.

Independent component analysis for semi-blind signal separation in MIMO mobile frequency selective communication channels

In this paper we address the problem of semi-blind source separation (SBSS) in frequency selective MIMO mobile communication channels. Semi-blindness stems from the fact that some average properties of the time-varying channel (mixing domain) are available at the transmitter. In this paper we first analytically show that when orthogonal frequency division multiplexing (OFDM) is employed, the original BSS problem is transformed into a set of standard ICA problems with complex mixing matrices. Each ICA problem is associated with one of the orthogonal subcarriers. This is special case of performing ICA in frequency domain where no inverse Fourier transformation of the separated signals is necessary. Secondly, we show that the statistical correlation between the different frequency bins (at each orthogonal sub-carrier) can be exploited to avoid the frequency dependent permutation problem, intrinsic to the ICA solution. Our approach has been tested on a realistic channel model and the results are presented.