Carlo Colombo

Semantics in visual information retrieval

A compositional approach increases the level of representation that can be automatically extracted and used in a visual information retrieval system. Visual information at the perceptual level is aggregated according to a set of rules. These rules reflect the specific context and transform perceptual words into phrases capturing pictorial content at a higher, and closer to the human, semantic level.

Semantic annotation of sports videos

Taking into consideration the unique qualities of sports videos, we propose a system that semantically annotates them at different layers of semantic significance, using different elements of visual content. We decompose each shot into its visual and graphic content elements and, by combining several different low-level visual primitives, capture the semantic content at a higher level of significance.

Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view

Image analysis and computer vision can be effectively employed to recover the three-dimensional structure of imaged objects, together with their surface properties. In this paper, we address the problem of metric reconstruction and texture acquisition from a single uncalibrated view of a surface of revolution (SOR). Geometric constraints induced in the image by the symmetry properties of the SOR structure are exploited to perform self-calibration of a natural camera, 3D metric reconstruction, and texture acquisition. By exploiting the analogy with the geometry of single axis motion, we demonstrate that the imaged apparent contour and the visible segments of two imaged cross sections in a single SOR view provide enough information for these tasks. Original contributions of the paper are: single view self-calibration and reconstruction based on planar rectification, previously developed for planar surfaces, has been extended to deal also with the SOR class of curved surfaces; self-calibration is obtained by estimating both camera focal length (one parameter) and principal point (two parameters) from three independent linear constraints for the SOR fixed entities; the invariant-based description of the SOR scaling function has been extended from affine to perspective projection. The solution proposed exploits both the geometric and topological properties of the transformation that relates the apparent contour to the SOR scaling function. Therefore, with this method, a metric localization of the SOR occluded parts can be made, so as to cope with them correctly. For the reconstruction of textured SORs, texture acquisition is performed without requiring the estimation of external camera calibration parameters, but only using internal camera parameters obtained from self-calibration.

Image-based robot task planning and control using a compact visual representation

We present an approach for the design and control of both reflexive and purposive visual tasks. The approach is based on the bidimensional appearance of the objects in the environment and explicitly takes into account independent object motions. A linear model of camera-object interaction is embedded in the control scheme, which dramatically simplifies visual analysis and control by reducing the size of visual representation. We describe the implementation of three visual tasks of increasing complexity, obtained with the proposed scheme and based on the active contour analysis and polynomial planning of image contour transformations. Both simulations and real-time experiments with a robotic eye-in-hand configuration are discussed, validating the approach in terms of robustness and applicability to visual navigation, active exploration and perception, and human-robot interaction.

Retrieval of Commercials by Semantic Content: The Semiotic Perspective

Video information processing and retrieval is a key aspect of future multimedia technologies and applications. Commercial videos encode several planes of expression through a rich and dense use of colors, editing effects, viewpoints and rhythms, which are exploited together to attract potential purchasers. Databases of commercials can be accessed in order to analyze how a commercial has been developed, retrieve commercials similar to an example, catalog commercials according to the kind of message conveyed to the user. In this paper, we present a system allowing the retrieval of commercial streams based on their salient semantics. Semantics is regarded from the semiotics perspective: collections of signs and semantic features like colors, editing effects, motion, etc. are used as basic blocks with which the meaning of a commercial is constructed. In our system, it is possible to retrieve commercials according to both the meaning they convey and to their similarity to examples.

Retrieval of commercials by video semantics

Videos convey information through several planes of communication, encompassing what is represented in the images how the images are linked together and how the subject is imaged. This feature is stressed in commercials where colors, editing effects, rhythms, and object motion are exploited to influence human purchasing habits. In this paper, based on researches in the marketing field, a link is formalized between low level features of a commercial video and feelings that the video would inspire in the observer. This link is used to define high level indices capturing the main semantics of the video. These indices are embedded in a video retrieval system to support access to a database of video based on their semantics.

On the use of linear camera-object interaction models in visual servoing

We investigate the exploitation of linear models of camera-object interaction for an efficient modeling and control of image-based visual servoing systems. The approach includes a method for coping with those representation ambiguities typical of linear interaction models which may affect both planning and control. The implementation of an eye-in-hand servoing system based on affine camera models and using image contours as relevant visual features is described and discussed. The system, including an image planner, a two-dimensional/three-dimensional (2-D/3-D) controller, and a visual analysis module, allows an intuitive specification and execution of relative positioning tasks w.r.t. still or moving rigid objects. Results of real-time experiments with a robotic platform featuring a PUMA manipulator provide a further insight into characteristics and performance of affine visual servoing systems.

Automatic extraction and annotation of soccer video highlights

Broadcasters are demonstrating interest in systems that ease the process of annotation the huge amount of live and archived video materials. Exploitation of such assets is considered a key method for the improvement of production quality, and sport videos are one of the most marketable assets. In particular, in Europe, soccer is one of the most relevant sport types. This paper deals with detection and recognition of soccer highlights, using an approach based on temporal logic models.

Robust tracking and remapping of eye appearance with passive computer vision

A single-camera iris-tracking and remapping approach based on passive computer vision is presented. Tracking is aimed at obtaining accurate and robust measurements of the iris/pupil position. To this purpose, a robust method for ellipse fitting is used, employing search constraints so as to achieve better performance with respect to the standard RANSAC algorithm. Tracking also embeds an iris localization algorithm (working as a bootstrap multiple-hypotheses generation step), and a blink detector that can detect voluntary eye blinks in human-computer interaction applications. On-screen remapping incorporates a head-tracking method capable of compensating for small user-head movements. The approach operates in real time under different light conditions and in the presence of distractors. An extensive set of experiments is presented and discussed. In particular, an evaluation method for the choice of layout of both hardware components and calibration points is described. Experiments also investigate the importance of providing a visual feedback to the user, and the benefits gained from performing head compensation, especially during image-to-screen map calibration.

A Compact and Retrieval-Oriented Video Representation Using Mosaics

Compact yet intuitive representations of digital videos are required to combine high quality storage with interactive video indexing and retrieval capabilities. The advent of video mosaicing has provided a natural way to obtain content-based video representations which are both retrieval-oriented and compression-efficient. In this paper, an algorithm for extracting a robust mosaic representation of video content from sparse interest image points is described. The representation, which is obtained via visual motion clustering and segmentation, features the geometric and kinematic description of all salient objects in the scene, being thus well suited for video browsing, indexing and retrieval by visual content. Results of experiments on several TV sequences provide an insight into the main characteristics of the approach.