Ofer Meshi

An alternating direction method for dual MAP LP relaxation

Maximum a-posteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper we present an algorithm for solving the LP relaxation optimization problem. In order to overcome the lack of strict convexity, we apply an augmented Lagrangian method to the dual LP. The algorithm, based on the alternating direction method of multipliers (ADMM), is guaranteed to converge to the global optimum of the LP relaxation objective. Our experimental results show that this algorithm is competitive with other state-of-the-art algorithms for approximate MAP estimation.

Evolutionary conservation and over-representation of functionally enriched network patterns in the yeast regulatory network

BackgroundLocalized network patterns are assumed to represent an optimal design principle in different biological networks. A widely used method for identifying functional components in biological networks is looking for network motifs – over-represented network patterns. A number of recent studies have undermined the claim that these over-represented patterns are indicative of optimal design principles and question whether localized network patterns are indeed of functional significance. This paper examines the functional significance of regulatory network patterns via their biological annotation and evolutionary conservation.ResultsWe enumerate all 3-node network patterns in the regulatory network of the yeast S. cerevisiae and examine the biological GO annotation and evolutionary conservation of their constituent genes. Specific 3-node patterns are found to be functionally enriched in different exogenous cellular conditions and thus may represent significant functional components. These functionally enriched patterns are composed mainly of recently evolved genes suggesting that there is no evolutionary pressure acting to preserve such functionally enriched patterns. No correlation is found between over-representation of network patterns and functional enrichment.ConclusionThe findings of functional enrichment support the view that network patterns constitute an important design principle in regulatory networks. However, the wildly used method of over-representation for detecting motifs is not suitable for identifying functionally enriched patterns.

Train and test tightness of LP relaxations in structured prediction

Structured prediction is used in areas such as computer vision and natural language processing to predict structured outputs such as segmentations or parse trees. In these settings, prediction is performed by MAP inference or, equivalently, by solving an integer linear program. Because of the complex scoring functions required to obtain accurate predictions, both learning and inference typically require the use of approximate solvers. We propose a theoretical explanation to the striking observation that approximations based on linear programming (LP) relaxations are often tight on real-world instances. In particular, we show that learning with LP relaxed inference encourages integrality of training instances, and that tightness generalizes from train to test data.

Logistic Markov Decision Processes.

User modeling in advertising and recommendation has typically focused on myopic predictors of user responses. In this work, we consider the long-term decision problem associated with user interaction. We propose a concise specification of long-term interaction dynamics by combining factored dynamic Bayesian networks with logistic predictors of user responses, allowing state-of-the-art prediction models to be seamlessly extended. We show how to solve such models at scale by providing a constraint generation approach for approximate linear programming that overcomes the variable coupling and nonlinearity induced by the logistic regression predictor. The efficacy of the approach is demonstrated on advertising domains with up to 2 states and 2 actions.