Felix Agakov

Using Machine Learning to Focus Iterative Optimization

Iterative compiler optimization has been shown to outperform static approaches. This, however, is at the cost of large numbers of evaluations of the program. This paper develops a new methodology to reduce this number and hence speed up iterative optimization. It uses predictive modelling from the domain of machine learning to automatically focus search on those areas likely to give greatest performance. This approach is independent of search algorithm, search space or compiler infrastructure and scales gracefully with the compiler optimization space size. Off-line, a training set of programs is iteratively evaluated and the shape of the spaces and program features are modelled. These models are learnt and used to focus the iterative optimization of a new program. We evaluate two learnt models, an independent and Markov model, and evaluate their worth on two embedded platforms, the Texas Instrument C67I3 and the AMD Au1500. We show that such learnt models can speed up iterative search on large spaces by an order of magnitude. This translates into an average speedup of 1.22 on the TI C6713 and 1.27 on the AMD Au1500 in just 2 evaluations.

Rapidly Selecting Good Compiler Optimizations using Performance Counters

Applying the right compiler optimizations to a particular program can have a significant impact on program performance. Due to the non-linear interaction of compiler optimizations, however, determining the best setting is non-trivial. There have been several proposed techniques that search the space of compiler options to find good solutions; however such approaches can be expensive. This paper proposes a different approach using performance counters as a means of determining good compiler optimization settings. This is achieved by learning a model off-line which can then be used to determine good settings for any new program. We show that such an approach outperforms the state-of-the-art and is two orders of magnitude faster on average. Furthermore, we show that our performance counter-based approach outperforms techniques based on static code features. Using our technique we achieve a 17% improvement over the highest optimization setting of the commercial PathScale EKOPath 2.3.1 optimizing compiler on the SPEC benchmark suite on a AMD Athlon 64 3700+ platform

Automatic performance model construction for the fast software exploration of new hardware designs

Developing an optimizing compiler for a newly proposed architecture is extremely difficult when there is only a simulator of the machine available. Designing such a compiler requires running many experiments in order to understand how different optimizations interact. Given that simulators are orders of magnitude slower than real processors, such experiments are highly restricted. This paper develops a technique to automatically build a performance model for predicting the impact of program transformations on any architecture, based on a limited number of automatically selected runs. As a result, the time for evaluating the impact of any compiler optimization in early design stages can be drastically reduced such that all selected potential compiler optimizations can be evaluated. This is achieved by first evaluating a small set of sample compiler optimizations on a prior set of benchmarks in order to train a model, followed by a very small number of evaluations, or probes, of the target program.We show that by training on less than 0. 7% of all possible transformations (640 samples collected from 10 benchmarks out of 880000 possible samples, 88000 per training benchmark) and probing the new program on only 4 transformations, we can predict the performance of all program transformations with an error of just 7. 3% on average. As each prediction takes almost no time to generate, this scheme provides an accurate method of evaluating compiler performance, which is several orders of magnitude faster than current approaches.

Application of high-dimensional feature selection: evaluation for genomic prediction in man

In this study, we investigated the effect of five feature selection approaches on the performance of a mixed model (G-BLUP) and a Bayesian (Bayes C) prediction method. We predicted height, high density lipoprotein cholesterol (HDL) and body mass index (BMI) within 2,186 Croatian and into 810 UK individuals using genome-wide SNP data. Using all SNP information Bayes C and G-BLUP had similar predictive performance across all traits within the Croatian data, and for the highly polygenic traits height and BMI when predicting into the UK data. Bayes C outperformed G-BLUP in the prediction of HDL, which is influenced by loci of moderate size, in the UK data. Supervised feature selection of a SNP subset in the G-BLUP framework provided a flexible, generalisable and computationally efficient alternative to Bayes C; but careful evaluation of predictive performance is required when supervised feature selection has been used.

Diet, Environmental Factors, and Lifestyle Underlie the High Prevalence of Vitamin D Deficiency in Healthy Adults in Scotland, and Supplementation Reduces the Proportion That Are Severely Deficient

Vitamin D deficiency has recently been implicated as a possible risk factor in the etiology of numerous diseases, including nonskeletal conditions. In humans, skin synthesis following exposure to UVB is a potent source of vitamin D, but in regions with low UVB, individuals are at risk of vitamin D deficiency. Our objectives were to describe the prevalence of vitamin D deficiency and to investigate determinants of plasma 25-hydroxyvitamin D (25-OHD) concentrations in a high northern latitude country. Detailed dietary, lifestyle, and demographic data were collected for 2235 healthy adults (21-82 y) from Scotland. Plasma 25-OHD was measured by liquid chromatography-tandem MS. Among study participants, 34.5% were severely deficient (25-OHD <25 nmol/L) and 28.9% were at high risk of deficiency (25-40 nmol/L). Only 36.6% of participants were at low risk of vitamin D deficiency or had adequate levels (>40 nmol/L). Among participants who were taking supplements, 21.3% had a May-standardized 25-OHD concentration >50 nmol/L, 54.2% had 25-50 nmol/L, and 24.5% had <25 nmol/L, whereas this was 15.6, 43.3, and 41%, respectively, among those who did not take supplements (P < 0.0001). The most important sources of vitamin D were supplements and fish consumption. Vitamin D deficiency in Scotland is highly prevalent due to a combination of insufficient exposure to UVB and insufficient dietary intake. Higher dietary vitamin D intake modestly improved the plasma 25-OHD concentration (P = 0.02) and reduced the proportion of severely deficient individuals (P < 0.0001). In regions with low UVB exposure, dietary and supplement intake may be much more important than previously thought and consideration should be given to increasing the current recommended dietary allowance of 0-10 μg/d for adults in Scotland.

The association of dietary intake of purine-rich vegetables, sugar-sweetened beverages and dairy with plasma urate, in a cross-sectional study

Introduction Hyperuricemia is a strong risk factor for gout. The incidence of gout and hyperuricemia has increased recently, which is thought to be, in part, due to changes in diet and lifestyle. Objective of this study was to investigate the association between plasma urate concentration and: a) food items: dairy, sugar-sweetened beverages (SSB) and purine-rich vegetables; b) related nutrients: lactose, calcium and fructose. Methods A total of 2,076 healthy participants (44% female) from a population-based case-control study in Scotland (1999–2006) were included in this study. Dietary data was collected using a semi-quantitative food frequency questionnaire (FFQ). Nutrient intake was calculated using FFQ and composition of foods information. Urate concentration was measured in plasma. Results Mean urate concentration was 283.8±72.1 mmol/dL (females: 260.1±68.9 mmol/dL and males: 302.3±69.2 mmol/dL). Using multivariate regression analysis we found that dairy, calcium and lactose intakes were inversely associated with urate (p = 0.008, p = 0.003, p = 0.0007, respectively). Overall SSB consumption was positively associated with urate (p = 0.008), however, energy-adjusted fructose intake was not associated with urate (p = 0.66). The intake of purine-rich vegetables was not associated to plasma urate (p = 0.38). Conclusions Our results suggest that limiting purine-rich vegetables intake for lowering plasma urate may be ineffectual, despite current recommendations. Although a positive association between plasma urate and SSB consumption was found, there was no association with fructose intake, suggesting that fructose is not the causal agent underlying the SSB-urate association. The abundant evidence supporting the inverse association between plasma urate concentration and dairy consumption should be reflected in dietary guidelines for hyperuricemic individuals and gout patients. Further research is needed to establish which nutrients and food products influence plasma urate concentration, to inform the development of evidence-based dietary guidelines.

Biomarkers of rapid chronic kidney disease progression in type 2 diabetes

Here we evaluated the performance of a large set of serum biomarkers for the prediction of rapid progression of chronic kidney disease (CKD) in patients with type 2 diabetes. We used a case-control design nested within a prospective cohort of patients with baseline eGFR 30-60 ml/min per 1.73 m(2). Within a 3.5-year period of Go-DARTS study patients, 154 had over a 40% eGFR decline and 153 controls maintained over 95% of baseline eGFR. A total of 207 serum biomarkers were measured and logistic regression was used with forward selection to choose a subset that were maximized on top of clinical variables including age, gender, hemoglobin A1c, eGFR, and albuminuria. Nested cross-validation determined the best number of biomarkers to retain and evaluate for predictive performance. Ultimately, 30 biomarkers showed significant associations with rapid progression and adjusted for clinical characteristics. A panel of 14 biomarkers increased the area under the ROC curve from 0.706 (clinical data alone) to 0.868. Biomarkers selected included fibroblast growth factor-21, the symmetric to asymmetric dimethylarginine ratio, β2-microglobulin, C16-acylcarnitine, and kidney injury molecule-1. Use of more extensive clinical data including prebaseline eGFR slope improved prediction but to a lesser extent than biomarkers (area under the ROC curve of 0.793). Thus we identified several novel associations of biomarkers with CKD progression and the utility of a small panel of biomarkers to improve prediction.

Protein biomarkers for the prediction of cardiovascular disease in type 2 diabetes

Aims/hypothesisWe selected the most informative protein biomarkers for the prediction of incident cardiovascular disease (CVD) in people with type 2 diabetes.MethodsIn this nested case–control study we measured 42 candidate CVD biomarkers in 1,123 incident CVD cases and 1,187 controls with type 2 diabetes selected from five European centres. Combinations of biomarkers were selected using cross-validated logistic regression models. Model prediction was assessed using the area under the receiver operating characteristic curve (AUROC).ResultsSixteen biomarkers showed univariate associations with incident CVD. The most predictive subset selected by forward selection methods contained six biomarkers: N-terminal pro-B-type natriuretic peptide (OR 1.69 per 1 SD, 95% CI 1.47, 1.95), high-sensitivity troponin T (OR 1.29, 95% CI 1.11, 1.51), IL-6 (OR 1.13, 95% CI 1.02, 1.25), IL-15 (OR 1.15, 95% CI 1.01, 1.31), apolipoprotein C-III (OR 0.79, 95% CI 0.70, 0.88) and soluble receptor for AGE (OR 0.84, 95% CI 0.76, 0.94). The prediction of CVD beyond clinical covariates improved from an AUROC of 0.66 to 0.72 (AUROC for Framingham Risk Score covariates 0.59). In addition to the biomarkers, the most important clinical covariates for improving prediction beyond the Framingham covariates were estimated GFR, insulin therapy and HbA1c.Conclusions/interpretationWe identified six protein biomarkers that in combination with clinical covariates improved the prediction of our model beyond the Framingham Score covariates. Biomarkers can contribute to improved prediction of CVD in diabetes but clinical data including measures of renal function and diabetes-specific factors not included in the Framingham Risk Score are also needed.

Products of Gaussians and probabilistic minor component analysis

Recently, Hinton introduced the products of experts architecture for density estimation, where individual expert probabilities are multiplied and renormalized. We consider products of gaussian pancakes equally elongated in all directions except one and prove that the maximum likelihood solution for the model gives rise to a minor component analysis solution. We also discuss the covariance structure of sums and products of gaussian pancakes or one-factor probabilistic principal component analysis models.

Glycosylation of plasma IgG in colorectal cancer prognosis

In this study we demonstrate the potential value of Immunoglobulin G (IgG) glycosylation as a novel prognostic biomarker of colorectal cancer (CRC). We analysed plasma IgG glycans in 1229 CRC patients and correlated with survival outcomes. We assessed the predictive value of clinical algorithms and compared this to algorithms that also included glycan predictors. Decreased galactosylation, decreased sialylation (of fucosylated IgG glycan structures) and increased bisecting GlcNAc in IgG glycan structures were strongly associated with all-cause (q < 0.01) and CRC mortality (q = 0.04 for galactosylation and sialylation). Clinical algorithms showed good prediction of all-cause and CRC mortality (Harrell’s C: 0.73, 0.77; AUC: 0.75, 0.79, IDI: 0.02, 0.04 respectively). The inclusion of IgG glycan data did not lead to any statistically significant improvements overall, but it improved the prediction over clinical models for stage 4 patients with the shortest follow-up time until death, with the median gain in the test AUC of 0.08. These glycan differences are consistent with significantly increased IgG pro-inflammatory activity being associated with poorer CRC prognosis, especially in late stage CRC. In the absence of validated biomarkers to improve upon prognostic information from existing clinicopathological factors, the potential of these novel IgG glycan biomarkers merits further investigation.