Melissa Cote

Texture sparseness for pixel classification of business document images

Contemporary business documents contain diverse, multi-layered mixtures of textual, graphical, and pictorial elements. Existing methods for document segmentation and classification do not handle well the complexity and variety of contents, geometric layout, and elemental shapes. This paper proposes a novel document image classification approach that distributes individual pixels into four fundamental classes (text, image, graphics, and background) through support vector machines. This approach uses a novel low-dimensional feature descriptor based on textural properties. The proposed feature vector is constructed by considering the sparseness of the document image responses to a filter bank on a multi-resolution and contextual basis. Qualitative and quantitative evaluations on business document images show the benefits of adopting a contextual and multi-resolution approach. The proposed approach achieves excellent results; it is able to handle varied contents and complex document layouts, without imposing any constraint or making assumptions about the shape and spatial arrangement of document elements.

Robust Texture Classification by Aggregating Pixel-Based LBP Statistics

This letter addresses the texture classification problem through a pixel-based local binary pattern (LBP) statistics aggregation mechanism. Real-world texture images often present challenges for classification algorithms in terms of intra-class variability due, among others, to variable illumination. The LBP operator, a state-of-the-art texture descriptor, possesses key properties for tackling real-world texture images: discriminative power and invariance against monotonic gray level changes. We propose a novel texture classification approach that increases the robustness of LBP-based methods with respect to any type of intra-class variations. The method locally characterizes each pixel with an LBP code histogram and globally computes the label of a textured image by aggregating pixel labels through a voting process. Our approach can be in principle applied to any LBP version, as it focuses on how statistics are computed from LBP codes. We show that the proposed pixel-based approach improves upon traditional LBP block-based approaches in terms of classification accuracy by up to 5.1 p.p. on the public Outex database for the classic LBP with various neighborhoods as well as for various LBP extensions.

Automatic Speaker Identification from Interpersonal Synchrony of Body Motion Behavioral Patterns in Multi-Person Videos

Interpersonal synchrony, i.e. the temporal coordination of persons during social interactions, was traditionally studied by developmental psychologists. It now holds an important role in fields such as social signal processing, usually treated as a dyadic issue. In this paper, we focus on the behavioral patterns from body motion to identify subtle social interactions in the context of multi-person discussion panels, typically involving more than two interacting individuals. We propose a computer-vision based approach for automatic speaker identification that takes advantage of body motion interpersonal synchrony between participants. The approach characterizes human body motion with a novel feature descriptor based on the pixel change history of multiple body regions, which is then used to classify the motor behavioral patterns of the participants into speaking/non-speaking. Our approach was evaluated on a challenging dataset of video segments from discussion panel scenes collected from YouTube. Results are very promising and suggest that interpersonal synchrony of motion behavior is indeed indicative of speaker/listener roles.

Sparseness-Based Descriptors for Texture Segmentation

This paper exploits the concept of sparseness to generate novel contextual multi-resolution texture descriptors. We propose to extract low-dimension features from Gabor-filtered images by considering the sparseness of filter bank responses. We construct several texture descriptors: the basic version describes each pixel by its contextual textural sparseness, while other versions also integrate multi-resolution information. We apply the novel low-dimension sparseness-based descriptors to the problem of texture segmentation and evaluate their performance on the public Outex database. The sparseness-based descriptors show a substantial improvement over Gabor filters with respect not only to computational costs and memory usage, but also to segmentation accuracy. The proposed approach also shows a desirable smooth, monotonic behavior with respect to the dimensionality of the descriptors.

Leaflet Free Edge Detection for the Automatic Analysis of Prosthetic Heart Valve Opening and Closing Motion Patterns from High Speed Video Recordings

Prosthetic heart valves (PHVs) are routinely used in clinical settings to replace defective native heart valves in patients suffering from valvular heart disease. Although PHV designs must be rigorously tested using cardiovascular testing equipment to ensure their optimal characteristics and safe operation, visual data obtained during simulations are typically assessed manually, a tedious and error-prone task. The valve orifice area over time, which informs on the opening and closing motion patterns, constitutes a key quality metric for PHV assessment. In addition to the very fast motion of the valve’s leaflets, a major issue lies in the orifice being partly occluded by the leaflets’ inner side or inaccurately depicted due to its transparency, which is not addressed in the literature. In this paper, we propose a novel orifice segmentation approach for automatic PHV quantitative performance analysis, based on the detection of the leaflet free edges to accurately extract the actual orifice area. Utilizing video frames recorded with a high speed digital camera during in vitro simulations, an initial estimation of the orifice area is first obtained via active contouring and then refined to capture the leaflet free edges via a curve extension scheme based on brightness and smoothness criteria. Evaluation on three different PHVs demonstrated the effectiveness of our approach to detect valve leaflet free edges and extract the actual orifice area, significantly outperforming a baseline algorithm both in terms of valve design evaluation metrics and computer vision evaluation metrics.

Video summarization for remote invigilation of online exams

This paper focuses on video summarization of abnormal behavior for remote invigilation of online exams. While the last decade has seen a massive increase in e-learning and online courses offered at postsecondary institutions, preserving the integrity of online examinations still heavily relies on web video conference invigilation performed by a remote proctor. Live remote invigilation is limited in the number of students that can be handled at once, and manual post-exam review is labor intensive. We propose a novel computer vision-based video content analysis system for the automatic creation of video summaries of online exams to assist remote proctors in post-exam reviews. The proposed method models normal and abnormal student behavior patterns using head pose estimations and a semantically meaningful two-state hidden Markov model. Video summaries are created from detected sequences of abnormal behavior. Experimental results are promising and demonstrate the viability of the proposed approach, which could readily be expanded to generate real-time alerts for live remote invigilation.

Layered ground truth: Conveying structural and statistical information for document image analysis and evaluation

This paper addresses the problem of semantic overlap across document objects in the context of ground truth representation for document layout analysis. Document object categories often share primitives from a low-level perspective (e.g. regions inside bars in a bar chart resemble background), making it difficult to evaluate document layout segmentation methods based on pixel classification, as most datasets and ground truth models focus on document objects. We propose a novel ground truth model that utilizes structural and statistical pattern recognition concepts. Statistical pixel-based data derived from low-level elemental patterns are layered onto high-level structural object-based data. We also present evaluation metrics that take advantage of the layered ground truth model, allowing a contextual evaluation of pixel classification algorithms. We apply the proposed model to two recent pixel classification approaches, evaluated on business document images that exhibit a challenging mixture of textual, graphical, and pictorial elements through varied layouts. The proposed model allows to obtain very detailed, comprehensive, and intuitive information on the strengths and limitations of the evaluated approaches that would be impossible to obtain through other models.