Deepak Khosla

Maturation of human central auditory system activity: separating auditory evoked potentials by dipole source modeling

OBJECTIVES Previous studies have shown that observed patterns of auditory evoked potential (AEP) maturation depend on the scalp location of the recording electrodes. Dipole source modeling incorporates the AEP information recorded at all electrode locations. This should provide a more robust description of auditory system maturation based on age-related changes in AEPs. Thus, the purpose of this study was to evaluate central auditory system maturation based dipole modeling of multi-electrode long-latency AEPs recordings. METHODS AEPs were recorded at 30 scalp-electrode locations from 118 subjects between 5 and 20 years of age. Regional dipole source analysis, using symmetrically located sources, was used to generate a spatio-temporal source model of age-related changes in AEP latency and magnitude. RESULTS The regional dipole source model separated the AEPs into distinct groups depending on the orientation of the component dipoles. The sagittally oriented dipole sources contained two AEP peaks, comparable in latency to Pa and Pb of the middle latency response (MLR). Although some magnitude changes were noted, latencies of Pa and Pb showed no evidence of age-related change. The tangentially oriented sources contained activity comparable to P1, N1b, and P2. There were various age-related changes in the latency and magnitude of the AEPs represented in the tangential sources. The radially oriented sources contained activity comparable to the T-complex, including Ta, and Tb, that showed only small latency changes with age. In addition, a long-latency component labeled TP200 was observed. CONCLUSIONS It is possible to distinguish 3 maturation groups: one group reaching maturity at age 6 and comprising the MLR components Pa and Pb, P2, and the T-complex. A second group that was relatively fast to mature (50%/year) was represented by N2. A third group was characterized by a slower pattern of maturation with a rate of 11-17%/year and included the AEP peaks P1, N1b, and TP200. The observed latency differences combined with the differences in maturation rate indicate that P2 is not identical to TP200. The results also demonstrated the independence of the T-complex components, represented in the radial dipoles, from the P1, N1b, and P2 components, contained in the tangentially oriented dipole sources.

Plasticity in the adult human central auditory system: evidence from late-onset profound unilateral deafness.

Experience-related changes in central nervous system (CNS) activity have been observed in the adult brain of many mammalian species, including humans. In humans, late-onset profound unilateral deafness creates an opportunity to study plasticity in the adult CNS consequent to monaural auditory deprivation. CNS activity was assessed by measuring long-latency auditory evoked potentials (AEPs) recorded from teens and adults with late-onset (post-childhood) profound unilateral deafness. Compared to monaurally stimulated normal-hearing subjects, the AEPs recorded from central electrode sites located over auditory cortical areas showed significant increases in inter-hemispheric waveform cross-correlation coefficients, and in inter-hemispheric AEP peak amplitude correlations. These increases provide evidence of substantial changes from the normal pattern of asymmetrical (contralateral > ipsilateral amplitude) and asynchronous (contralateral earlier than ipsilateral) central auditory system activation in the normal-hearing population to a much more symmetrical and synchronous activation in the unilaterally deaf. These cross-sectional analyses of AEP data recorded from the unilaterally deaf also suggest that the changes in cortical activity occur gradually and continue for at least 2 years after the onset of hearing loss. Analyses of peak amplitude correlations suggest that the increased inter-hemispheric symmetry may be a consequence of changes in the generators producing the N (approximately 100 ms peak latency) potential. These experience-related changes in central auditory system activity following late-onset profound unilateral deafness thus provide evidence of the presence and the time course of auditory system plasticity in the adult brain.

Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

Deep-learning neural networks such as convolutional neural network (CNN) have shown great potential as a solution for difficult vision problems, such as object recognition. Spiking neural networks (SNN)-based architectures have shown great potential as a solution for realizing ultra-low power consumption using spike-based neuromorphic hardware. This work describes a novel approach for converting a deep CNN into a SNN that enables mapping CNN to spike-based hardware architectures. Our approach first tailors the CNN architecture to fit the requirements of SNN, then trains the tailored CNN in the same way as one would with CNN, and finally applies the learned network weights to an SNN architecture derived from the tailored CNN. We evaluate the resulting SNN on publicly available Defense Advanced Research Projects Agency (DARPA) Neovision2 Tower and CIFAR-10 datasets and show similar object recognition accuracy as the original CNN. Our SNN implementation is amenable to direct mapping to spike-based neuromorphic hardware, such as the ones being developed under the DARPA SyNAPSE program. Our hardware mapping analysis suggests that SNN implementation on such spike-based hardware is two orders of magnitude more energy-efficient than the original CNN implementation on off-the-shelf FPGA-based hardware.

Differential Ear Effects of Profound Unilateral Deafness on the Adult Human Central Auditory System

This study investigates the effects of profound acquired unilateral deafness on the adult human central auditory system by analyzing long-latency auditory evoked potentials (AEPs) with dipole source modeling methods. AEPs, elicited by clicks presented to the intact ear in 19 adult subjects with profound unilateral deafness and monaurally to each ear in eight adult normal-hearing controls, were recorded with a 31-channel system. The responses in the 70–210 ms time window, encompassing the N1b/P2 and Ta/Tb components of the AEPs, were modeled by a vertically and a laterally oriented dipole source in each hemisphere. Peak latencies and amplitudes of the major components of the dipole waveforms were measured in the hemispheres ipsilateral and contralateral to the stimulated ear. The normal-hearing subjects showed significant ipsilateral–contralateral latency and amplitude differences, with contralateral source activities that were typically larger and peaked earlier than the ipsilateral activities. In addition, the ipsilateral–contralateral amplitude differences from monaural presentation were similar for left and for right ear stimulation. For unilaterally deaf subjects, the previously reported reduction in ipsilateral–contralateral amplitude differences based on scalp waveforms was also observed in the dipole source waveforms. However, analysis of the source dipole activity demonstrated that the reduced inter-hemispheric amplitude differences were ear dependent. Specifically, these changes were found only in those subjects affected by profound left ear unilateral deafness.

A bio-inspired system for spatio-temporal recognition in static and video imagery

This paper presents a bio-inspired method for spatio-temporal recognition in static and video imagery. It builds upon and extends our previous work on a bio-inspired Visual Attention and object Recognition System (VARS). The VARS approach locates and recognizes objects in a single frame. This work presents two extensions of VARS. The first extension is a Scene Recognition Engine (SCE) that learns to recognize spatial relationships between objects that compose a particular scene category in static imagery. This could be used for recognizing the category of a scene, e.g., office vs. kitchen scene. The second extension is the Event Recognition Engine (ERE) that recognizes spatio-temporal sequences or events in sequences. This extension uses a working memory model to recognize events and behaviors in video imagery by maintaining and recognizing ordered spatio-temporal sequences. The working memory model is based on an ARTSTORE1 neural network that combines an ART-based neural network with a cascade of sustained temporal order recurrent (STORE)1 neural networks. A series of Default ARTMAP classifiers ascribes event labels to these sequences. Our preliminary studies have shown that this extension is robust to variations in an objects motion profile. We evaluated the performance of the SCE and ERE on real datasets. The SCE module was tested on a visual scene classification task using the LabelMe2 dataset. The ERE was tested on real world video footage of vehicles and pedestrians in a street scene. Our system is able to recognize the events in this footage involving vehicles and pedestrians.

Bio-inspired visual attention and object recognition

This paper describes a bio-inspired Visual Attention and Object Recognition System (VARS) that can (1) learn representations of objects that are invariant to scale, position and orientation; and (2) recognize and locate these objects in static and video imagery. The system uses modularized bio-inspired algorithms/techniques that can be applied towards finding salient objects in a scene, recognizing those objects, and prompting the user for additional information to facilitate interactive learning. These algorithms are based on models of human visual attention, search, recognition and learning. The implementation is highly modular, and the modules can be used as a complete system or independently. The underlying technologies were carefully researched in order to ensure they were robust, fast, and could be integrated into an interactive system. We evaluated our systems capabilities on the Caltech-101 and COIL-100 datasets, which are commonly used in machine vision, as well as on simulated scenes. Preliminary results are quite promising in that our system is able to process these datasets with good accuracy and low computational times.

Hybrid genetic approach for the dynamic weapon-target allocation problem

This paper addresses the problem of threat engagement and dynamic weapon-target allocation (WTA) across the force or network-centric force optimization. The objective is to allocate and schedule defensive weapon resources over a given period of time so as to minimize surviving target value subject to resource availability and temporal constraints. The dynamic WTA problem is a NP-complete problem and belongs to a class of multiple-resource-constrained optimal scheduling problems. Inherent complexities in the problem of determining the optimal solution include limited weapon resources, time windows under which threats must be engaged, load-balancing across weapon systems, and complex interdependencies of various assignments and resources. We present a new hybrid genetic algorithm (GA) which is a combination of a traditional genetic algorithm and a simulated annealing-type algorithm for solving the dynamic WTA problem. The hybrid GA approach proposed here uses a simulated annealing-type heuristics to compute the fitness of a GA-selected population. This step also optimizes the temporal dimension (scheduling) under resource and temporal constraints. The proposed method provides schedules that are near-optimal in short cycle times and have minimal perturbation from one cycle to the next. We compare the performance of the proposed approach with a baseline WTA algorithm.

Emulating Mammalian Vision on Reconfigurable Hardware

A significant challenge in creating machines with artificial vision is designing systems which can process visual information as efficiently as the brain. To address this challenge, we identify key algorithms which model the process of attention and recognition in the visual cortex of mammals. This paper presents Cover - an FPGA framework for generating systems which can potentially emulate the visual cortex. We have designed accelerators for models of attention and recognition in the cortex and integrated them to realize an end-to-end attention-recognition system. Evaluation of our system on a Dinigroup multi-FPGA platform shows high performance and accuracy for attention and recognition systems and speedups over existing CPU, GPU and FPGA implementations. Results show that our end-to-end system which emulates the cortex can achieve near real-time speeds for high resolution images. This system can be applied to many artificial vision applications such as augmented virtual reality and autonomous vehicle navigation.

A reconfigurable accelerator for neuromorphic object recognition

Advances in neuroscience have enabled researchers to develop computational models of auditory, visual and learning perceptions in the human brain. HMAX, which is a biologically inspired model of the visual cortex, has been shown to outperform standard computer vision approaches for multi-class object recognition. HMAX, while computationally demanding, can be potentially applied in various applications such as autonomous vehicle navigation, unmanned surveillance and robotics. In this paper, we present a reconfigurable hardware accelerator for the time-consuming S2 stage of the HMAX model. The accelerator leverages spatial parallelism, dedicated wide data buses with on-chip memories to provide an energy efficient solution to enable adoption into embedded systems. We present a systolic array-based architecture which includes a run-time reconfigurable convolution engine which can perform multiple variable-sized convolutions in parallel. An automation flow is described for this accelerator which can generate optimal hardware configurations for a given algorithmic specification and also perform run-time configuration and execution seamlessly. Experimental results on Virtex-6 FPGA platforms show 5X to 11X speedups and 14X to 33X higher performance-per-Watt over a CNS-based implementation on a Tesla GPU.

A neuromorphic system for object detection and classification

Unattended object detection, recognition and tracking on unmanned reconnaissance platforms in battlefields and urban spaces are topics of emerging importance. In this paper, we present an unattended object recognition system that automatically detects objects of interest in videos and classifies them into various categories (e.g., person, car, truck, etc.). Our system is inspired by recent findings in visual neuroscience on feed-forward object detection and recognition pipeline and mirrors that via two main neuromorphic modules (1) A front-end detection module that combines form and motion based visual attention to search for and detect “integrated” object percepts as is hypothesized to occur in the human visual pathways; (2) A back-end recognition module that processes only the detected object percepts through a neuromorphic object classification algorithm based on multi-scale convolutional neural networks, which can be efficiently implemented in COTS hardware. Our neuromorphic system was evaluated using a variety of urban area video data collected from both stationary and moving platforms. The data are quite challenging as it includes targets at long ranges, occurring under variable conditions of illuminations and occlusion with high clutter. The experimental results of our system showed excellent detection and classification performance. In addition, the proposed bio-inspired approach is good for hardware implementation due to its low complexity and mapping to off-the-shelf conventional hardware.