New data-driven approaches to improve probabilistic model structure learning

Abstract

To learn the network structures used in probabilistic models (e.g., Bayesian network), many researchers proposed structure learning algorithms to extract the network structure from data. However, structure learning is a challenging problem due to the extremely large number of possible structure candidates. One challenge relates to structure learning in Bayesian network is the conflicts among local structures obtained from the local structure learning algorithms. This is the so-called symmetry correction problem. Another challenge is the V-structure selection problem, which is related to the determination of edge orientation in Bayesian network. In this thesis, we investigate the above two challenges in structure learning and propose novel data-driven approaches to overcome these challenges when building a Bayesian network. First, two new data-driven symmetry correction methods are developed to learn an undirected graph of Bayesian network. The proposed methods outperform the existing heuristic rule. Second, a weighted maximum satisfiability (MAX-SAT) problem is formulated to solve the V-structures selection problem. The weights are learned from data to quantify the strength of the Vstructures. Our proposed solution outperforms existing methods. Besides, we investigate how transfer learning can be used for structure learning with limited training examples and a source structure. In particular, we propose a transfer learning approach to learn the structure of a Sum-Product Network (SPN) which can be converted to a Bayesian network under certain conditions. Our novel approach allows one to construct the target SPN with limited training examples, given an existing source SPN from a similar domain.