Andrew J. Lundberg

Three‐dimensional tongue surface shapes of English consonants and vowels

This paper presents three-dimensional tongue surfaces reconstructed from multiple coronal cross-sectional slices of the tongue. Surfaces were reconstructed for sustained vocalizations of the American English sounds [symbol: see text]. Electropalatography (EPG) data were also collected for the sounds to compare tongue surface shapes with tongue-palate contact patterns. The study was interested also in whether 3-D surface shapes of the tongue were different for consonants and vowels. Previous research and speculation had found that there were differences in production, acoustics, and linguistic usage between the two groups. The present study found that four classes of tongue shape were adequate to categorize all the sounds measured. These classes were front raising, complete groove, back raising, and two-point displacement. The first and third classes have been documented before in the midsagittal plane [cf. R. Harshman, P. Ladefoged, and L. Goldstein, J. Acoust. Soc. Am. 62, 693-707 (1976)]. The first three classes contained both vowels and consonants, the last only consonants. Electropalatographic patterns of the sounds indicated three categories of tongue-palate contact: bilateral, cross-sectional, and combination of the two. Vowels used only the first pattern, consonants used all three. The EPG data provided an observable distinction in contact pattern between consonants and vowels. The ultrasound tongue surface data did not. The conclusion was that the tongue actually has a limited repertoire of shapes and positions them against the palate in different ways for consonants versus vowels to create narrow channels, divert airflow, and produce sound.

Modeling the motion of the internal tongue from tagged cine-MRI images

A new technique, tagged Cine-Magnetic Resonance Imaging (tMRI), was used to develop a mechanical model that represented local, homogeneous, internal tongue deformation during speech. The goal was to infer muscle activity within the tongue from tissue deformations seen on tMRI. Measurements were made in three sagittal slices (left, middle, right) during production of the syllable /ka/. Each slice was superimposed with a grid of tag lines, and the approximately 40 tag line intersections were tracked at 7 time-phases during the syllable. A local model, similar to a finite element analysis, represented planar stretch and shear between the consonant and vowel at 110 probed locations within the tongue. Principal strains were calculated at these locations and revealed internal compression and extension patterns from which inferences could be drawn about the activities of the Verticalis, Hyoglossus, and Superior Longitudinal muscles, among others.

Three-dimensional tongue surface reconstruction: practical considerations for ultrasound data.

This paper discusses methods for reconstructing the tongue from sparse data sets. Sixty ultrasound slices already have been used to reconstruct three-dimensional (3D) tongue surface shapes [Stone and Lundberg, J. Acoust. Soc. Am. 99, 3728-3737 (1996)]. To reconstruct 3D surfaces, particularly in motion, collecting 60 slices would be impractical, and possibly unnecessary. The goal of this study was to select a sparse set of slices that would best reconstruct the 18 measured speech sounds. First a coronal sparse set was calculated from 3D surface reconstructions. Selection of contours was globally optimized using coarse to fine search. Sparse and dense reconstructions were compared using maximum error, standard deviation error, and surface coverage. For all speech sounds, maximum error was less than 1.5 mm, standard deviation error was less than 0.32 mm, and average reconstruction coverage was 80%. To generalize the method across subjects, optimal slice locations were calculated from only the midsagittal contour. Six midsagittal points were optimized to reconstruct the midsagittal contour. Corresponding coronal slices were then used to reconstruct 3D surfaces. For data collection planning, a midsagittal sample can be collected first and optimal coronal slices can be determined from it. Errors and reconstruction coverage from the midsagittal source set were comparable to the optimized coronal sparse set. These sparse surfaces reconstructed static 3D surfaces, and should be usable for motion sequences as well.

System for imaging the bladder during voiding

The system includes an ultrasound apparatus which provides successive images of a bladder during the voiding process, the images being approximately one second long so as to provide an accurate picture of the action of the bladder during voiding. The images are then processed to form a substantially continuous moving image, i.e. a video or motion picture. The video covers approximately five minutes so as to capture the entire voiding event. The system is initiated typically by the user by pressing a button or other implement on the ultrasound apparatus which is carried by a belt or on a garment adjacent the abdomen of the user. Several processing techniques are used to compensate for any movement of the user or the device during the voiding process so as to provide a coherent and continuous moving image of the bladder during the voiding process.

Image understanding from thermal emission polarization

Existing polarization-based image understanding techniques use information only from reflected light. Apart from incandescent bodies thermally emitted light radiation from elements of a scene in the visible spectrum is insignificant. However, at longer wavelengths such as in the infrared thermal emission is typically quite prevalent from a number of scene elements of interest. FLIR imagery of both indoor and outdoor scenes reveals that many objects thermally emit a significant amount of radiation. Polarization from thermally emitting objects has been observed as long as 170 years ago from incandescent objects but since then there have only been a limited number of empirical investigations into this phenomenon. This paper presents a comprehensive model for explaining polarization of thermal emission from both rough and smooth surfaces, in agreement with empirical data, that can significantly enhance the image understanding of FLIR imagery. In particular it is possible to discern metal from dielectric materials under certain conditions, and from an accurate model for thermally emitted polarization it is possible to predictively model polarization signatures from CAD models of importance to automatic target recognition.

New perspectives on geometric reflection theory from rough surfaces

Exact wave theories of specular reflectance from rough surfaces are computationally intractable thus motivating the practical need for geometric reflectance models which treat only the geometric ray nature of light reflection. The cornerstone of geometric reflectance modeling from rough surfaces in computer vision and computer graphics over the past two decades has been the Torrance-Sparrow model. This model has worked well as an intuitive description of rough surfaces as a collection of planar Fresnel reflectors called microfacets together with the concept of geometric attenuation for light which is obscured during reflection under an assumed rough surface geometry. Experimental data and analysis show that the current conceptualization of how specularly reflected light rays geometrically interact with rough surfaces needs to be seriously revised. The Torrance-Sparrow model while in qualitative agreement with specular reflection from rough surfaces is seen to be quantitatively inaccurate. Furthermore there are conceptual inconsistencies upon which derivation of this reflectance model is based. We show how significant quantitative improvement can be achieved for a geometric reflectance model by making some fundamental revisions to notions of microfacet probability distributions and geometric attenuation. Work is currently undergoing, to relate physical surface reconstructions from Atomic Force Microscope data to reflectance data from these same surfaces.

Using principal component analysis of tongue surface shapes to distinguish among vowels and speakers

The present study uses principal component analysis (PCA) to examine sagittal tongue contours for five English vowels taken from ultrasound images. The vowels are repeated three to five times each in a /pVp/ carrier utterance. Of particular interest is the use of plots of the coefficients of the eigenvectors to distinguish both subjects and vowels, and the building of a linear model to fit their family of data. Data will be transformed to normalize surface length across contours. Short contours will be stretched in the x direction to the length of the longest curve. The x and y dimensions will be stretched equally for each curve to preserve scale. Preliminary data indicate fairly good success distinguishing among three subjects and four vowels using a linear model based on the first few components. Methods of improving this result are being explored using factor analysis or optimal fitting.

Image Intensification for Low-Light Face Recognition

This paper presents the first published systematic study of face recognition performance as a function of light level using intensified near infrared imagery. This technology is the most prevalent in both civilian and military night vision equipment, and provides enough intensification for human operators to perform standard tasks under extremely low-light conditions. We describe a comprehensive data collection effort undertaken by the authors to image subjects under carefully controlled illumination and quantify the performance of standard face recognition algorithms on visible and intensified imagery as a function of light level. Performance comparisons for automatic face recognition are reported using the standardized implementations from the CSU Face Identification Evaluation System. The results contained in this paper should constitute the initial step for analysis and deployment of face recognition systems designed to work in low-light level condi

Three-dimensional tongue surfaces from ultrasound images

This paper presents 3D tongue surfaces reconstructed from sixty cross-sectional slices of the tongue. Surfaces were reconstructed for sustained vocalizations of 18 American English sounds. Electropalatography (EPG) data also were collected for the sounds to compare tongue surface shape with tongue-palate contact patterns. The ultrasound data were grouped into four tongue shape categories. These classes were front raising, complete channel, back raising, two-point displacement. The first three categories contained both vowels and consonants, the last only consonants. The EPG data indicated three categories of tongue-palate contact: bilateral, cross-sectional, combination of the two. Vowels used only the first pattern, consonants used all three. The EPG data provided an observably distinction in contact pattern between consonants and vowels. The ultrasound tongue surface data did not. The conclusion was that the tongue actually has a limited repertoire of shapes, and positions them against the palate in different ways for consonants vs. vowels to create narrow channels, divert airflow and produce sound.

Face Recognition in Low-Light Environments Using Fusion of Thermal Infrared and Intensified Imagery

This chapter presents a study of face recognition performance as a function of light level using intensified near infrared imagery in conjunction with thermal infrared imagery. Intensification technology is the most prevalent in both civilian and military night vision equipment and provides enough enhancement for human operators to perform standard tasks under extremely low light conditions. We describe a comprehensive data collection effort undertaken to image subjects under carefully controlled illumination and quantify the performance of standard face recognition algorithms on visible, intensified, and thermal imagery as a function of light level. Performance comparisons for automatic face recognition are reported using the standardized implementations from the Colorado State University Face Identification Evaluation System, as well as Equinoxs algorithms. The results contained in this chapter should constitute the initial step for analysis and deployment of face recognition systems designed to work in low-light conditions.