Aleksandr Y. Aravkin

Improvements to Deep Convolutional Neural Networks for LVCSR

Deep Convolutional Neural Networks (CNNs) are more powerful than Deep Neural Networks (DNN), as they are able to better reduce spectral variation in the input signal. This has also been confirmed experimentally, with CNNs showing improvements in word error rate (WER) between 4-12% relative compared to DNNs across a variety of LVCSR tasks. In this paper, we describe different methods to further improve CNN performance. First, we conduct a deep analysis comparing limited weight sharing and full weight sharing with state-of-the-art features. Second, we apply various pooling strategies that have shown improvements in computer vision to an LVCSR speech task. Third, we introduce a method to effectively incorporate speaker adaptation, namely fMLLR, into log-mel features. Fourth, we introduce an effective strategy to use dropout during Hessian-free sequence training. We find that with these improvements, particularly with fMLLR and dropout, we are able to achieve an additional 2-3% relative improvement in WER on a 50-hour Broadcast News task over our previous best CNN baseline. On a larger 400-hour BN task, we find an additional 4-5% relative improvement over our previous best CNN baseline.

Positron Emission Mammography: Correlation of Estrogen Receptor, Progesterone Receptor, and Human Epidermal Growth Factor Receptor 2 Status and 18F-FDG

OBJECTIVE The study objective was to assess the correlation between (18)F-FDG uptake values on positron emission mammography (PEM), expressed as maximum uptake value and lesion-to-background ratio, and receptor status (i.e., estrogen receptor [ER], progesterone receptor [PR], and human epidermal growth factor receptor 2 [HER2]), tumor histology, and tumor grade. We also evaluated for the correlation between maximum uptake value on PEM and maximum uptake value on a whole-body PET/CT. MATERIALS AND METHODS We retrospectively reviewed our database for patients with newly diagnosed breast cancer who were referred for PEM between June 2007 and September 2009. A subset of patients also underwent a whole-body PET/CT scan. The original pathology reports were reviewed to establish the histologic type, grade, and receptor status. RESULTS The study involved 98 patients with 100 lesions. ER-negative tumors and PR-negative tumors had significantly higher mean lesion-to-background ratio than did their respective receptor-positive tumors (p = 0.02). Triple-negative tumors (i.e., ER-negative, PR-negative, and HER2-negative tumors) had statistically higher mean lesion-to-background ratio than did ER-positive PR-positive HER2-negative tumors (p = 0.04). Infiltrating ductal carcinomas had significantly higher PEM FDG uptake values than did infiltrating lobular carcinomas (p = 0.02-0.04). Breast tumors with higher histologic grade also had significantly higher PEM FDG uptake values than did those with lower grade (p = 0.03 and p < 0.001). A moderately high correlation (0.76-0.79) was seen between whole-body PET/CT and PEM uptake values. CONCLUSION This study shows a correlation between PEM FDG uptake values and the prognostic factors that have been shown to predict breast cancer survival.

Risk Factors for Neurotoxicity in Newborns With Severe Neonatal Hyperbilirubinemia

OBJECTIVE: To evaluate the importance of total serum bilirubin (TSB) and neurotoxicity risk factors in predicting acute bilirubin encephalopathy (ABE) at admission or posttreatment bilirubin encephalopathy (BE) in infants with severe hyperbilirubinemia. METHODS: We analyzed the interaction of TSB and risk factors as determinants of ABE and BE in 249 newborns admitted with a TSB level of ≥25 mg/dL (427 μmol/L) to Cairo University Childrens Hospital during a 12-month period. RESULTS: Admission TSB values ranged from 25 to 76.4 mg/dL. Forty-four newborns had moderate or severe ABE at admission; 35 of 249 infants (14%) had evidence of BE at the time of discharge or death. Rh incompatibility (odds ratio [OR]: 48.6) and sepsis (OR: 20.6) greatly increased the risk for ABE/BE, but TSB levels correlated poorly with the presence or absence of ABE or BE in these patients. The OR for ABO incompatibility with anemia (1.8) was not statistically significant. Low admission weight (OR: 0.83 per 100 g) increased the risk for BE, especially when other risk factors were present. The threshold TSB level that identified 90% of infants with ABE/BE was 25.4 mg/dL when neurotoxicity risk factors were present. In contrast, neurotoxicity was first observed at a TSB level of >31.5 mg/dL in 111 infants without risk factors. CONCLUSIONS: Newborns without risk factors for neurotoxicity have a higher tolerance for hyperbilirubinemia than recognized in management guidelines. The risk for BE in hemolytic disease varies with etiology. The great variation in response to TSB indicates that biological factors other than TSB values are important in the pathogenesis of BE.

Seismic Waveform Inversion by Stochastic Optimization

We explore the use of stochastic optimization methods for seismic waveform inversion. The basic principle of such methods is to randomly draw a batch of realizations of a given misfit function and goes back to the 1950s. The ultimate goal of such an approach is to dramatically reduce the computational cost involved in evaluating the misfit. Following earlier work, we introduce the stochasticity in waveform inversion problem in a rigorous way via a technique called randomized trace estimation. We then review theoretical results that underlie recent developments in the use of stochastic methods for waveform inversion. We present numerical experiments to illustrate the behavior of different types of stochastic optimization methods and investigate the sensitivity to the batch size and the noise level in the data. We find that it is possible to reproduce results that are qualitatively similar to the solution of the full problem with modest batch sizes, even on noisy data. Each iteration of the corresponding stochastic methods requires an order of magnitude fewer PDE solves than a comparable deterministic method applied to the full problem, which may lead to an order of magnitude speedup for waveform inversion in practice.

Robust inversion, dimensionality reduction, and randomized sampling

We consider a class of inverse problems where it is possible to aggregate the results of multiple experiments. This class includes problems where the forward model is the solution operator to linear ODEs or PDEs. The tremendous size of such problems motivates dimensionality reduction techniques based on randomly mixing experiments. These techniques break down, however, when robust data-fitting formulations are used, which are essential in cases of missing data, unusually large errors, and systematic features in the data unexplained by the forward model. We survey robust methods within a statistical framework, and propose a semistochastic optimization approach that allows dimensionality reduction. The efficacy of the methods are demonstrated for a large-scale seismic inverse problem using the robust Students t-distribution, where a useful synthetic velocity model is recovered in the extreme scenario of 60% data missing at random. The semistochastic approach achieves this recovery using 20% of the effort required by a direct robust approach.

An

Robustness is a major problem in Kalman filtering and smoothing that can be solved using heavy tailed distributions; e.g., ℓ1-Laplace. This paper describes an algorithm for finding the maximum a posteriori (MAP) estimate of the Kalman smoother for a nonlinear model with Gaussian process noise and ℓ1 -Laplace observation noise. The algorithm uses the convex composite extension of the Gauss-Newton method. This yields convex programming subproblems to which an interior point path-following method is applied. The number of arithmetic operations required by the algorithm grows linearly with the number of time points because the algorithm preserves the underlying block tridiagonal structure of the Kalman smoother problem. Excellent fits are obtained with and without outliers, even though the outliers are simulated from distributions that are not ℓ1 -Laplace. It is also tested on actual data with a nonlinear measurement model for an underwater tracking experiment. The ℓ1-Laplace smoother is able to construct a smoothed fit, without data removal, from data with very large outliers.

\ell _{1}

BACKGROUND AND OBJECTIVE: Bilirubin/albumin ratio (B/A) may provide a better estimate of free bilirubin than total serum bilirubin (TSB), thus improving identification of newborns at risk for bilirubin encephalopathy. The objective of the study was to identify thresholds and compare specificities of TSB and B/A in detecting patients with acute and posttreatment auditory and neurologic impairment. METHODS: A total of 193 term/near-term infants, admitted for severe jaundice to Cairo University Children’s Hospital, were evaluated for neurologic status and auditory impairment (automated auditory brainstem response), both at admission and posttreatment by investigators blinded to laboratory results. The relationships of TSB and B/A to advancing stages of neurotoxicity were compared by using receiver operating characteristic curves. RESULTS: TSB and B/A ranged from 17 to 61 mg/dL and 5.4 to 21.0 mg/g, respectively; 58 (30%) of 193 subjects developed acute bilirubin encephalopathy, leading to kernicterus in 35 infants (13 lethal). Auditory impairment was identified in 86 (49%) of 173 infants at admission and in 22 of 128 at follow-up. In the absence of clinical risk factors, no residual neurologic or hearing impairment occurred unless TSB exceeded 31 mg/dl. However, transient auditory impairment occurred at lower TSB and B/A (22.9 mg/dL and 5.7 mg/g, respectively). Intervention values of TSB and B/A set at high sensitivity to detect different stages of neurotoxicity had nearly the same specificity. CONCLUSIONS: Both TSB and B/A are strong predictors of neurotoxicity, but B/A does not improve prediction over TSB alone. Threshold values detecting all affected patients (100% sensitivity) increase with advancing severity of neurotoxicity.

-Laplace Robust Kalman Smoother

Many inverse problems include nuisance parameters which, while not of direct interest, are required to recover primary parameters. The structure of these problems allows efficient optimization strategies—a well-known example is variable projection, where nonlinear least-squares problems which are linear in some parameters can be very efficiently optimized. In this paper, we extend the idea of projecting out a subset over the variables to a broad class of maximum likelihood and maximum a posteriori likelihood problems with nuisance parameters, such as variance or degrees of freedom (d.o.f.). As a result, we are able to incorporate nuisance parameter estimation into large-scale constrained and unconstrained inverse problem formulations. We apply the approach to a variety of problems, including estimation of unknown variance parameters in the Gaussian model, d.o.f. parameter estimation in the context of robust inverse problems, and automatic calibration. Using numerical examples, we demonstrate improvement in recovery of primary parameters for several large-scale inverse problems. The proposed approach is compatible with a wide variety of algorithms and formulations, and its implementation requires only minor modifications to existing algorithms.

Serum Bilirubin and Bilirubin/Albumin Ratio as Predictors of Bilirubin Encephalopathy

The goal of image stitching is to create natural-looking mosaics free of artifacts that may occur due to relative camera motion, illumination changes, and optical aberrations. In this paper, we propose a novel stitching method, that uses a smooth stitching field over the entire target image, while accounting for all the local transformation variations. Computing the warp is fully automated and uses a combination of local homography and global similarity transformations, both of which are estimated with respect to the target. We mitigate the perspective distortion in the non-overlapping regions by linearizing the homography and slowly changing it to the global similarity. The proposed method is easily generalized to multiple images, and allows one to automatically obtain the best perspective in the panorama. It is also more robust to parameter selection, and hence more automated compared with state-of-the-art methods. The benefits of the proposed approach are demonstrated using a variety of challenging cases.

Estimating nuisance parameters in inverse problems

Regularization plays a key role in a variety of optimization formulations of inverse problems. A recurring theme in regularization approaches is the selection of regularization parameters and their effect on the solution and on the optimal value of the optimization problem. The sensitivity of the value function to the regularization parameter can be linked directly to the Lagrange multipliers. This paper characterizes the variational properties of the value functions for a broad class of convex formulations, which are not all covered by standard Lagrange multiplier theory. An inverse function theorem is given that links the value functions of different regularization formulations (not necessarily convex). These results have implications for the selection of regularization parameters, and the development of specialized algorithms. Numerical examples illustrate the theoretical results.