Finnegan Southey

Discriminative unsupervised learning of structured predictors

We present a new unsupervised algorithm for training structured predictors that is discriminative, convex, and avoids the use of EM. The idea is to formulate an unsupervised version of structured learning methods, such as maximum margin Markov networks, that can be trained via semidefinite programming. The result is a discriminative training criterion for structured predictors (like hidden Markov models) that remains unsupervised and does not create local minima. To reduce training cost, we reformulate the training procedure to mitigate the dependence on semidefinite programming, and finally propose a heuristic procedure that avoids semidefinite programming entirely. Experimental results show that the convex discriminative procedure can produce better conditional models than conventional Baum-Welch (EM) training.

Metric-Based Methods for Adaptive Model Selection and Regularization

We present a general approach to model selection and regularization that exploits unlabeled data to adaptively control hypothesis complexity in supervised learning tasks. The idea is to impose a metric structure on hypotheses by determining the discrepancy between their predictions across the distribution of unlabeled data. We show how this metric can be used to detect untrustworthy training error estimates, and devise novel model selection strategies that exhibit theoretical guarantees against over-fitting (while still avoiding under-fitting). We then extend the approach to derive a general training criterion for supervised learning—yielding an adaptive regularization method that uses unlabeled data to automatically set regularization parameters. This new criterion adjusts its regularization level to the specific set of training data received, and performs well on a variety of regression and conditional density estimation tasks. The only proviso for these methods is that sufficient unlabeled training data be available.

Notio - A Java API for Developing CG Tools

Notio [1] is a Java API for constructing conceptual graph tools and systems. Rather than attempting to provide a comprehensive toolset, Notio attempts to address the widely varying needs of the CG community by providing a platform for the development of tools and applications. It is, first and foremost, an API specification for which different underlying implementations may be constructed. A pure Java reference implementation is currently available for development and evaluation of the API, and to guide future implementations. An overview of the motivation, design, and features of Notio is provided.

Multidisciplinary students and instructors: a second-year games course

Computer games are a multi-billion dollar industry and have become an important part of our private and social lives. It is only natural, then, that the technology used to create games should become part of a computing science curriculum. However, game development is more than a massive programming endeavor. Todays games are largely about generating content within multidisciplinary teams. CMPUT 250 is a new computing science course at the University of Alberta that emphasizes creating games in multidisciplinary teams. This paper describes our experiences with the course, emphasizing the issues of multidisciplinary interactions: teaching, teamwork, and evaluation.

Tangent-corrected embedding

Images and other high-dimensional data can frequently be characterized by a low dimensional manifold (e.g. one that corresponds to the degrees of freedom of the camera). Recently, nonlinear manifold learning techniques have been used to map images to points in a lower dimension space, capturing some of the dynamics of the camera or the subjects. In general, these methods do not take advantage of any prior understanding of the dynamics we might have, relying instead on local Euclidean distances that can be misleading in image space. In practice, we frequently have some prior knowledge regarding the transformations that relate images (e.g. rotation, translation, etc). We present a method for augmenting existing embedding techniques with additional information derived from known transformations, either in the form of tangent spaces that locally characterize the manifold or distances derived from reconstruction errors. The extra information is incorporated directly into the cost function of the embedding technique. The techniques we augment are largely attractive because there is a closed form solution for their cost optimization. Our approach likewise produces a closed form solution for the augmented cost function. Experiments demonstrate the effectiveness of the approach on a variety of image data.

Ossa - A Conceptual Modelling System for Virtual Realities

As virtual reality systems achieve new heights of visual and auditory realism, the need for improving the underlying conceptual modelling facilities becomes increasingly apparent. The Ossa system provides a media-independent modelling environment based on a production system model that uses conceptual graphs to represent both the facts and the rules. Using conceptual graphs allows for interaction with the virtual world using multiple modalities (e.g. graphics and natural language). Conceptual graphs also allow for highly expressive facts and rules, and a diagrammatic programming technique. The motivation, design, and implementation of the Ossa system are discussed.

Constraint metrics for local search

Over the years, a steadily improving series of local search solvers for propositional satisfiability (SAT) have been constructed. However, these solvers are often fragile, in that they have apparently minor details in their implementation that dramatically affect performance and confound understanding. In order to understand and predict the success of differing strategies, various local search metrics have been proposed. Many of these metrics summarize properties of the boolean assignments examined during the search. This has two consequences: first, they only capture one side of satisfiability, failing to characterize the behaviour with respect to constraints. Secondly, the boolean requirement limits the applicability of these metrics to more general constraint satisfaction problems (CSPs), which can have non-boolean domains. In response, we present dual metrics, derived from existing primal (boolean assignment) metrics, that are based on the states of constraints during the search. Experimental results show a strong relationship between the primal and dual versions of these metrics on a variety of random and structured problems. This dual perspective can be easily applied to both SAT and general CSPs, allowing for new insights into the workings of a broad class of local search methods.