Joseph Schlecht

Sampling bedrooms

We propose a top down approach for understanding indoor scenes such as bedrooms and living rooms. These environments typically have the Manhattan world property that many surfaces are parallel to three principle ones. Further, the 3D geometry of the room and objects within it can largely be approximated by non overlapping simple structures such as single blocks (e.g. the room boundary), thin blocks (e.g. picture frames), and objects that are well modeled by single blocks (e.g. simple beds). We separately model the 3D geometry, the imaging process (camera parameters), and edge likelihood, to provide a generative statistical model for image data. We fit this model using data driven MCMC sampling. We combine reversible jump Metropolis Hastings samples for discrete changes in the model such as the number of blocks, and stochastic dynamics to estimate continuous parameter values in a particular parameter space that includes block positions, block sizes, and camera parameters. We tested our approach on two datasets using room box pixel orientation. Despite using only bounding box geometry and, in particular, not training on appearance, our method achieves results approaching those of others. We also introduce a new evaluation method for this domain based on ground truth camera parameters, which we found to be more sensitive to the task of understanding scene geometry.

Machine-learning approaches for classifying haplogroup from Y chromosome STR data.

Genetic variation on the non-recombining portion of the Y chromosome contains information about the ancestry of male lineages. Because of their low rate of mutation, single nucleotide polymorphisms (SNPs) are the markers of choice for unambiguously classifying Y chromosomes into related sets of lineages known as haplogroups, which tend to show geographic structure in many parts of the world. However, performing the large number of SNP genotyping tests needed to properly infer haplogroup status is expensive and time consuming. A novel alternative for assigning a sampled Y chromosome to a haplogroup is presented here. We show that by applying modern machine-learning algorithms we can infer with high accuracy the proper Y chromosome haplogroup of a sample by scoring a relatively small number of Y-linked short tandem repeats (STRs). Learning is based on a diverse ground-truth data set comprising pairs of SNP test results (haplogroup) and corresponding STR scores. We apply several independent machine-learning methods in tandem to learn formal classification functions. The result is an integrated high-throughput analysis system that automatically classifies large numbers of samples into haplogroups in a cost-effective and accurate manner.

Contour-based Object Detection

The arrival of appearance-based image features has dramatically influenced the field of visual object recognition. Previous work has shown, however, that contour curvature and junctions are important for shape representation and detection. We investigate a local representation of contours for object detection that complements appearance-based information, such as texture. We present a non-parametric representation of contours, curvature, and junctions which enables their accurate localization. We combine contour and appearance information into a general, voting-based detection algorithm. Besides detecting objects, we demonstrate that this approach reveals the most relevant contours and junctions supporting each object hypothesis. The experiments confirm that our contourbased representation compliments appearance information and the performance of baseline voting methods is significantly improved.

Inferring Grammar-based Structure Models from 3D Microscopy Data

We present a new method to fit grammar-based stochastic models for biological structure to stacks of microscopic images captured at incremental focal lengths. Providing the ability to quantitatively represent structure and automatically fit it to image data enables important biological research. We consider the case where individuals can be represented as an instance of a stochastic grammar, similar to L-systems used in graphics to produce realistic plant models. In particular, we construct a stochastic grammar of Alternaria, a genus of fungus, and fit instances of it to microscopic image stacks. We express the image data as the result of a generative process composed of the underlying probabilistic structure model together with the parameters of the imaging system. Fitting the model then becomes probabilistic inference. For this we create a reversible-jump MCMC sampler to traverse the parameter space. We observe that incorporating spatial structure helps fit the model parts, and that simultaneously fitting the imaging system is also very helpful.

Nonverbal Communication in Medieval Illustrations Revisited by Computer Vision and Art History

This contribution discusses how computer vision combined with art history can analyze the visual codes and artistic representations of embodied communication represented in medieval culture. Our computer-based detection algorithms are able to search directly for gestures in images and thus avoid limitations of retrieval systems that search only textual annotations. As a result, art history is provided not only with a system that enables efficient access to large image datasets, but also a quantitative analysis of the variability and interrelation among gestures or between gestures and other objects. We base our approach on one of four illustrated manuscripts of Eike von Repgows (ca. 1180–ca. 1235) Sachsenspiegel (Mirror of the Saxons), which reveals a visual grammar that arranges the gestures to a certain extent, but within that framework, the drafter composes freely and with artistic perspective.

Statistical Inference of Biological Structure and Point Spread Functions in 3D Microscopy

We present a novel method for detecting and quantifying 3D structure in stacks of microscopic images captured at incremental focal lengths. We express the image data as stochastically generated by an underlying model for biological specimen and the effects of the imaging system. The method simultaneously fits a model for proposed structure and the imaging systems parameters, which include a model of the point spread function. We demonstrate our approach by detecting spores in image stacks of Alternaria, a microscopic genus of fungus. The spores are modeled as opaque ellipsoids and fit to the data using statistical inference. Since the number of spores in the data is not known, model selection is incorporated into the fitting process. Thus, we develop a reversible jump Markov chain Monte Carlo sampler to explore the parameter space. Our results show that simultaneous statistical inference of specimen and imaging models is useful for quantifying biological structures in 3D microscopic images. In addition, we show that inferring a model of the imaging system improves the overall fit of the specimen model to the data.

Detecting gestures in medieval images

We present a template-based detector for gestures visualized in legal manuscripts of the Middle Ages. Depicted persons possess gestures with specific semantic meaning from the perspective of legal history. The hand drawn gestures exhibit noticeable variation in artistic style, size and orientation. They follow a distinct visual pattern, however, without any perspective effects. We present a method to learn a small set of templates representative of the gesture variability. We apply an efficient version of normalized cross-correlation to vote for gesture position, scale and orientation. Non-parametric kernel density estimation is used to identify hypotheses in voting space, and a discriminative verification step ranks the detections. We demonstrate our method on four types of gestures and show promising detection results.

Modelling and visualizing morphology in the fungus Alternaria

Alternaria is one of the most cosmopolitan fungal genera encountered and impacts humans and human activities in areas of material degradation, phytopathology, food toxicology, and respiratory disease. Contemporary methods of taxon identification rely on assessments of morphology related to sporulation, which are critical for accurate diagnostics. However, the morphology of Alternaria is quite complex, and precise characterization can be laborious, time-consuming, and often restricted to experts in this field. To make morphology characterization easier and more broadly accessible, a generalized statistical model was developed for the three-dimensional geometric structure of the sporulation apparatus. The model is inspired by the widely used grammar-based models for plants, Lindenmayer-systems, which build structure by repeated application of rules for growth. Adjusting the parameters of the underlying probability distributions yields variations in the morphology, and thus the approach provides an excellent tool for exploring the morphology of Alternaria under different assumptions, as well as understanding how it is largely the consequence of local rules for growth. Further, different choices of parameters lead to different model groups, which can then be visually compared to published descriptions or microscopy images to validate parameters for species-specific models. The approach supports automated analysis, as the models can be fit to image data using statistical inference, and the explicit representation of the geometry allows the accurate computation of any morphological quantity. Furthermore, because the model can encode the statistical variation of geometric parameters for different species, it will allow automated species identification from microscopy images using statistical inference. In summary, the approach supports visualization of morphology, automated quantification of phenotype structure, and identification based on form.