Egon C. Pasztor

Learning Low-Level Vision

We describe a learning-based method for low-level vision problems—estimating scenes from images. We generate a synthetic world of scenes and their corresponding rendered images, modeling their relationships with a Markov network. Bayesian belief propagation allows us to efficiently find a local maximum of the posterior probability for the scene, given an image. We call this approach VISTA—Vision by Image/Scene TrAining.We apply VISTA to the “super-resolution” problem (estimating high frequency details from a low-resolution image), showing good results. To illustrate the potential breadth of the technique, we also apply it in two other problem domains, both simplified. We learn to distinguish shading from reflectance variations in a single image under particular lighting conditions. For the motion estimation problem in a “blobs world”, we show figure/ground discrimination, solution of the aperture problem, and filling-in arising from application of the same probabilistic machinery.

Hyperscore: a graphical sketchpad for novice composers

The Hyperscore graphical computer-assisted composition system for users with limited or no musical training takes freehand drawing as input, letting users literally sketch their pieces. Designing an intelligent, intuitive system that enables novices-particularly children-to compose music is a difficult task. We can view the problem as a spectrum of tasks that range from the development of musical algorithms for automating the compositional process to designing an appropriate interface for humans to interact with the machine. The Hyperscore software tool attempts to address both of these issues.

Learning style translation for the lines of a drawing

We present an example-based method for translating line drawings into different styles. We fit each line as a linear combination of similar lines in a training set, and interpolate between the corresponding training examples in the output style. The synthesized lines preserve the desired stylistic features of the output style.