Irina Rish

Mini-buckets: A general scheme for bounded inference

This article presents a class of approximation algorithms that extend the idea of bounded-complexity inference, inspired by successful constraint propagation algorithms, to probabilistic inference and combinatorial optimization. The idea is to bound the dimensionality of dependencies created by inference algorithms. This yields a parameterized scheme, called mini-buckets, that offers adjustable trade-off between accuracy and efficiency. The mini-bucket approach to optimization problems, such as finding the most probable explanation (MPE) in Bayesian networks, generates both an approximate solution and bounds on the solution quality. We present empirical results demonstrating successful performance of the proposed approximation scheme for the MPE task, both on randomly generated problems and on realistic domains such as medical diagnosis and probabilistic decoding.

Prediction and interpretation of distributed neural activity with sparse models

We explore to what extent the combination of predictive and interpretable modeling can provide new insights for functional brain imaging. For this, we apply a recently introduced regularized regression technique, the Elastic Net, to the analysis of the PBAIC 2007 competition data. Elastic Net regression controls via one parameter the number of voxels in the resulting model, and via another the degree to which correlated voxels are included. We find that this method produces highly predictive models of fMRI data that provide evidence for the distributed nature of neural function. We also use the flexibility of Elastic Net to demonstrate that model robustness can be improved without compromising predictability, in turn revealing the importance of localized clusters of activity. Our findings highlight the functional significance of patterns of distributed clusters of localized activity, and underscore the importance of models that are both predictive and interpretable.

Directional Resolution: The Davis-Putnam Procedure, Revisited

The paper presents algorithm directional resolution, a variation on the original Davis-Putnam algorithm, and analyzes its worst-case behavior as a function of the topological structure of the theories. The notions of induced width and diversity are shown to play a key role in bounding the complexity of the procedure. The importance of our analysis lies in highlighting structure-based tractable classes of satisfiability and in providing theoretical guarantees on the time and space complexity of the algorithm. Contrary to previous assessments, we show that for many theories directional resolution could be an effective procedure. Our empirical tests confirm theoretical prediction, showing that on problems with special structures, like chains, directional resolution greatly outperforms one of the most effective satisfiability algorithm known to date, namely the popular Davis-Putnam procedure.

Resolution versus Search: Two Strategies for SAT

The paper compares two popular strategies for solving propositional satisfiability, backtracking search and resolution, and analyzes the complexity of a directional resolution algorithm (DR) as a function of the “width” (w*) of the problems graph. Our empirical evaluation confirms theoretical prediction, showing that on low-w* problems DR is very efficient, greatly outperforming the backtracking-based Davis–Putnam–Logemann–Loveland procedure (DP). We also emphasize the knowledge-compilation properties of DR and extend it to a tree-clustering algorithm that facilitates query answering. Finally, we propose two hybrid algorithms that combine the advantages of both DR and DP. These algorithms use control parameters that bound the complexity of resolution and allow time/space trade-offs that can be adjusted to the problem structure and to the users computational resources. Empirical studies demonstrate the advantages of such hybrid schemes.

Real-time problem determination in distributed systems using active probing

We describe algorithms and an architecture for a real-time problem determination system that uses online selection of most-informative measurements - the approach called herein active probing. Probes are end-to-end test transactions which gather information about system components. Active probing allows probes to be selected and sent on-demand, in response to ones belief about the state of the system. At each step the most informative next probe is computed and sent. As probe results are received, belief about the system state is updated using probabilistic inference. This process continues until the problem is diagnosed. We demonstrate through both analysis and simulation that the active probing scheme greatly reduces both the number of probes and the time needed for localizing the problem when compared with non-active probing schemes.

Optimizing Probe Selection for Fault Localization

We investigate the use of probing technology for the purpose of problem determination and fault localization in networks. We present a framework for addressing this issue and implement algorithms that exploit interactions between probe paths to find a small collection of probes that can be used to locate faults. Small probe sets are desirable in order to minimize the costs imposed by probing, such as additional network load and data management requirements. Our results show that although finding the optimal collection of probes is expensive for large networks, efficient approximation algorithms can be used to find a nearly-optimal set.

A decomposition of classes via clustering to explain and improve Naive Bayes

We propose a method to improve the probability estimates made by Naive Bayes to avoid the effects of poor class conditional probabilities based on product distributions when each class spreads into multiple regions. Our approach is based on applying a clustering algorithm to each subset of examples that belong to the same class, and to consider each cluster as a class of its own. Experiments on 26 real-world datasets show a significant improvement in performance when the class decomposition process is applied, particularly when the mean number of clusters per class is large.

Closed-form supervised dimensionality reduction with generalized linear models

We propose a family of supervised dimensionality reduction (SDR) algorithms that combine feature extraction (dimensionality reduction) with learning a predictive model in a unified optimization framework, using data- and class-appropriate generalized linear models (GLMs), and handling both classification and regression problems. Our approach uses simple closed-form update rules and is provably convergent. Promising empirical results are demonstrated on a variety of high-dimensional datasets.

Predictive Dynamics of Human Pain Perception

While the static magnitude of thermal pain perception has been shown to follow a power-law function of the temperature, its dynamical features have been largely overlooked. Due to the slow temporal experience of pain, multiple studies now show that the time evolution of its magnitude can be captured with continuous online ratings. Here we use such ratings to model quantitatively the temporal dynamics of thermal pain perception. We show that a differential equation captures the details of the temporal evolution in pain ratings in individual subjects for different stimulus pattern complexities, and also demonstrates strong predictive power to infer pain ratings, including readouts based only on brain functional images.

Learning sparse Gaussian Markov networks using a greedy coordinate ascent approach

In this paper, we introduce a simple but efficient greedy algorithm, called SINCO, for the Sparse INverse COvariance selection problem, which is equivalent to learning a sparse Gaussian Markov Network, and empirically investigate the structure-recovery properties of the algorithm. Our approach is based on a coordinate ascent method which naturally preserves the sparsity of the network structure. We show that SINCO is often comparable to, and, in various cases, outperforms commonly used approaches such as glasso [7] and COVSEL [1], in terms of both structure-reconstruction error (particularly, false positive error) and computational time. Moreover, our method has the advantage of being easily parallelizable. Finally, we show that SINCOs greedy nature allows reproduction of the regularization path behavior by applying the method to one (sufficiently small) instance of the regularization parameter λ only; thus, SINCO can obtain a desired number of network links directly, without having to tune the λ parameter. We evaluate our method empirically on various simulated networks and real-life data from biological and neuroimaging applications.