Richard F. Lyon

An analog electronic cochlea

An analog electronic cochlea has been built in CMOS VLSI technology using micropower techniques. The key point of the model and circuit is that a cascade of simple, nearly linear, second-order filter stages with controllable Q parameters suffices to capture the physics of the fluid-dynamic traveling-wave system in the cochlea, including the effects of adaptation and active gain involving the outer hair cells. Measurements on the test chip suggest that the circuit matches both the theory and observations from real cochleas. >

A computational model of filtering, detection, and compression in the cochlea

We claim that speech analysis algorithms should be based on computational models of human audition, starting at the ears. While much is known about how hearing works, little of this knowledge has been applied in the speech analysis field. We propose models of the inner ear, or cochlea, which are expressed as time- and place-domain signal processing operations; i.e. the models are computational expressions of the important functions of the cochlea. The main parts of the models concern mechanical filtering effects and the mapping of mechanical vibrations into neural representation. Our model cleanly separates these effects into time-invariant linear filtering based on a simple cascade/parallel filterbank network of second-order sections, plus transduction and compression based on half-wave rectification with a nonlinear coupled automatic gain control network. Compared to other speech analysis techniques, this model does a much better job of preserving important detail in both time and frequency, which is important for robust sound analysis. We discuss the ways in which this model differs from more detailed cochlear models.

Google Street View: Capturing the World at Street Level

Street View serves millions of Google users daily with panoramic imagery captured in hundreds of cities in 20 countries across four continents. A team of Google researchers describes the technical challenges involved in capturing, processing, and serving street-level imagery on a global scale.

A computational model of binaural localization and separation

Multiple sound signals, such as speech and interfering noises, can be fairly well separated, localized, and interpreted by human listeners with normal binaural hearing. The computational model presented here, based on earlier cochlear modeling work, is a first step at approaching human levels of performance on the localization and separation tasks. This combination of cochlear and binaural models, implemented as real-time algorithms, could provide the front end for a robust sound interpretation system such as a speech recognizer. The cochlear model used is basically a bandpass filterbank with frequency channels corresponding to places on the basilar membrane; filter outputs are half-wave rectified and amplitude-compressed, maintaining fine time resolution. In the binaural model, outputs of corresponding frequency channels from the two ears are combined by cross-correlation. Peaks in the short-time cross-correlation functions are then interpreted as direction. With appropriate preprocessing, the correlation peaks integrate cues based on signal phase, envelope modulation, onset time, and loudness. Based on peaks in the correlation functions, sources can be recognized, localized, and tracked. Through quickly varying gains, sound fragments are separated into streams representing different sources. Preliminary tests of the algorithms are very encouraging.

A Low-Power Wide-Linear-Range Transconductance Amplifier

The linear range of approximately ±75mV of traditional subthreshold transconductance amplifiers istoo small for certain applications—for example, for filtersin electronic cochleas, where it is desirable to handle loudsounds without distortion and to have a large dynamic range.We describe a transconductance amplifier designed for low-power(< 1 µW) subthreshold operation with a wideinput linear range. We obtain wide linear range by widening thetanh, or decreasing the ratio of transconductance to bias current,by a combination of four techniques. First, the well terminalsof the input differential-pair transistors are used as the amplifierinputs. Then, feedback techniques known as source degeneration(a common technique) and gate degeneration (a new technique)provide further improvements. Finally, a novel bump-linearizationtechnique extends the linear range even further. We present signal-flowdiagrams for speedy analysis of such circuit techniques. Ourtransconductance reduction is achieved in a compact 13-transistorcircuit without degrading other characteristics such as dc-inputoperating range. In a standard 2 µm process,we were able to obtain a linear range of ±1.7V.Using our wide-linear-range amplifier and a capacitor, we constructa follower–integrator with an experimental dynamic rangeof 65 dB. We show that, if the amplifiers noise is predominantlythermal, then an increase in its linear range increases thefollower–integratorsdynamic range. If the amplifiers noise is predominantly 1/f,then an increase in its linear range has no effect on thefollower–integratorsdynamic range. To preserve follower–integrator bandwidth,power consumption increases proportionately with an increasein the amplifiers linear range. We also present data for changesin the subthreshold exponential parameter with current leveland with gate-to-bulk voltage that should be of interest to alllow-power designers. We have described the use of our amplifierin a silicon cochlea [1, 2].

A perceptual pitch detector

A pitch detector based on Lickliders (1979) duplex theory of pitch perception was implemented and tested on a variety of stimuli from human perceptual tests. It is believed that this approach accurately models how people perceive pitch. It is shown that it correctly identifies the pitch of complex harmonic and inharmonic stimuli and that it is robust in the face of noise and phase changes. This perceptual pitch detector combines a cochlear model with a bank of autocorrelators. By performing an independent autocorrelation for each channel, the pitch detector is relatively insensitive to phase changes across channels. The information in the correlogram is filtered, nonlinearly enhanced, and summed across channels. Peaks are identified and a pitch is then proposed that is consistent with the peaks.<<ETX>>

A low-power wide-dynamic-range analog VLSI cochlea

Low-power wide-dynamic-range systems are extremely hard to build. The biological cochlea is one of the most awesome examples of such a system: It can sense sounds over 12 orders of magnitude in intensity, with an estimated power dissipation of only a few tens of microwatts. In this paper, we describe an analog electronic cochlea that processes sounds over 6 orders of magnitude in intensity, and that dissipates 0.5 mW. This 117-stage, 100 Hz to 10 KHz cochlea has the widest dynamic range of any artificial cochlea built to date. The wide dynamic range is attained through the use of a wide-linear-range transconductance amplifier, of a low-noise filter topology, of dynamic gain control (AGC) at each cochlear stage, and of an architecture that we refer to as overlapping cochlear cascades. The operation of the cochlea is made robust through the use of automatic offset-compensation circuitry. A BiCMOS circuit approach helps us to attain nearly scale-invariant behavior and good matching at all frequencies. The synthesis and analysis of our artificial cochlea yields insight into why the human cochlea uses an active traveling-wave mechanism to sense sounds, instead of using bandpass filters. The low power, wide dynamic range, and biological realism make our cochlea well suited as a front end for cochlear implants.

Auditory model inversion for sound separation

Techniques to recreate sounds from perceptual displays known as cochleagrams and correlograms are developed using a convex projection framework. Prior work on cochlear-model inversion is extended to account for rectification and gain adaptation. A prior technique for phase recovery in spectrogram inversion is combined with the synchronized overlap-and-add technique of speech rate modification, and is applied to inverting the short-time autocorrelation function representation in the auditory correlogram. Improved methods of initial phase estimation are explored. A range of computational cost options, with and without iteration, produce a range of quality levels from fair to near perfect.<<ETX>>

Computational models of neural auditory processing

Explicit neuron firing models are investigated for use in computational modeling of auditory processing. Models for primary auditory neurons are driven by the receptor signal from a hair cell model, which is driven in turn by a filtering model of basilar membrane motion. The output of the primary auditory neuron model is a times-of-firing representation of neural signals. Certain types of processing, such as auto-correlation and cross-correlation, are very simple with this representation, not requiring multiplication. The neuron model used is a leaky-integrate-to-threshold model with a refractory period. Several neurons are modeled at each hair cell, or filter channel. It is found in experiments with these models that the detailed time-of-firing information contains most of the cues of speech formants, pitch, direction, etc. The more conventionally studied firing rate vs. place representation misses important aspects of these cues. Models of pitch perception, binaural directional perception, and sound separation are being based on the cochlear and neural models. The models are all implemented as computational algorithms, and are used in support of related speech recognition and hearing research.

ASIC implementation of the Lyon cochlea model

The design and development of an application-specific integrated circuit (ASIC) to implement the filtering and AGC sections of a new digital approximation to the Lyon/Mead analog cochlea model (1988, 1989) are described. The ASIC uses a fully synchronous bit-serial design methodology originally developed for MOS technologies by Denyer and Renshaw (1985) and adaptive subsequently for use with low cost standard cell design tools by Summerfield and Jabri (1989). The resulting device is an example of a new class of signal processing ASIC, referred to as an application-specific signal processor (ASSP). As well as presenting details of the filter and AGC structure of the new cochlea model, the paper describes the bit-serial design methodology used to implement the algorithm in real-time and in low cost ASIC form. The approach is a generic one and can be applied to a wide range of speech signal processing functions.<<ETX>>