Paulo Lobato Correia

Objective evaluation of video segmentation quality

Video segmentation assumes a major role in the context of object-based coding and description applications. Evaluating the adequacy of a segmentation result for a given application is a requisite both to allow the appropriate selection of segmentation algorithms as well as to adjust their parameters for optimal performance. Subjective testing, the current practice for the evaluation of video segmentation quality, is an expensive and time-consuming process. Objective segmentation quality evaluation techniques can alternatively be used; however, it is recognized that, so far, much less research effort has been devoted to this subject than to the development of segmentation algorithms. This paper discusses the problem of video segmentation quality evaluation, proposing evaluation methodologies and objective segmentation quality metrics for individual objects as well as for complete segmentation partitions. Both stand alone and relative evaluation metrics are developed to cover the cases for which a reference segmentation is missing or available for comparison.

Automatic Detection and Classification of Traffic Signs

This paper proposes algorithms for the automatic detection of traffic signs from photo or video images and their classification to provide a driver alert system. Several examples taken from Portuguese roads are used to demonstrate the effectiveness of the proposed system. Traffic signs are detected by analyzing color information, notably red and blue, contained on the images. The detected signs are then classified according to their shape characteristics, as triangular, squared and circular shapes. Combining color and shape information, traffic signs are classified into one of the following classes: danger, information, obligation or prohibition. Both the detection and classification algorithms include innovative components to improve the overall system performance.

Objective evaluation of relative segmentation quality

When working in image and video segmentation, the major objective is to design an algorithm producing the appropriate segmentation results for the particular goals of the application addressed. Therefore, the assessment of the segmentation quality assumes a crucial importance to evaluate the likeliness that the application targets are met. Since no well-established methods for objective segmentation quality evaluation are currently available, this papers major goal is to propose objective metrics for the evaluation of relative segmentation quality for both individual objects and the overall segmentation partition. The paper presents a methodology for performing objective evaluation of relative segmentation quality, identifies the relevant features to be compared against those of the reference segmentation, and proposes appropriate objective quality metrics. These metrics build on the existing knowledge of segmentation quality evaluation and also on some relevant aspects from the video quality evaluation field.

The role of analysis in content-based video coding and indexing

Abstract The increasing spread of digital technology in many areas, notably telecommunications, and entertainment (TV/cinema), is nowadays changing the production, delivery, and consumption paradigms for multimedia information. New applications with critical requirements in terms of content-based interactivity are imminent, motivating the evolution of the models used for data representation, notably for coding and indexing. The emerging MPEG-4 and MPEG-7 standards are the recognition, by the industry, of these upcoming needs. This paper addresses the problem of video analysis for content-based coding and indexing in the context of a changing technological landscape. The main video analysis objectives and constraints are identified, the role of user interaction is studied, and some application examples are described.

Classification of video segmentation application scenarios

Video analysis can be used in the context of a wide variety of applications and therefore a multiplicity of techniques has been proposed in the literature. Each of those techniques is usually devoted to solving a specific part of the complete analysis problem, unless the problem is rather simple. Typically, to be able to propose meaningful analysis solutions, the analysis problem must first be appropriately constrained, taking into account the relevant application environment. Then, complementary types of analysis techniques may have to be used in combination to achieve the desired results. This paper proposes a classification of segmentation applications into a set of scenarios, according to the different application constraints and goals. This allows an easier selection of the appropriate video segmentation solution for each specific application. Examples of segmentation solutions for the most relevant scenarios identified are presented.

Identifying and retrieving distress images from road pavement surveys

This paper proposes a system capable of identifying and retrieving distress images from a road pavement survey image database. A database of images acquired during road pavement surface surveys along Portuguese roads is considered. Regions corresponding to cracks are detected over the acquired images, based on a subdivision of the images into a set of non-overlapping windows, which may be classified as containing cracks, or not. Crack detection results, represented by binary images where windows containing crack pixels are set to one, undergo a second classification stage to distinguish several crack types. This classification follows the structure proposed by the Portuguese Distresses Catalog, produced by the national entity in charge of road maintenance. Three crack types are identified at this stage: longitudinal cracks, transversal cracks and miscellaneous cracks. The experimental results, obtained by processing real survey imagery over Portuguese roads, present encouraging results for automating the process of identifying road distresses from images.

Stand-alone objective segmentation quality evaluation

AbstractThe identification of objects in video sequences, that is, video segmentation, plays a major role in emerging interactive multimedia services, such as those enabled by the ISO MPEG- and MPEG- standards. In this context, assessing the adequacy of the identified objects to the application targets, that is, evaluating the segmentation quality, assumes a crucial importance. Video segmentation technology has received considerable attention in the literature, with algorithms being proposed to address various types of applications. However, the segmentation quality performance evaluation of those algorithms is often ad hoc, and a well-established solution is not available. In fact, the field of objective segmentation quality evaluation is still maturing; recently, some more efforts have been made, mainly following the emergence of the MPEG object-based coding and description standards. This paper discusses the problem of objective segmentation quality evaluation in its most difficult scenario: standalone evaluation, that is, when a reference segmentation is not available for comparative evaluation. In particular, objective metrics are proposed for the evaluation of standalone segmentation quality for both individual objects and overall segmentation partitions.

Description schemes for video programs, users and devices

This paper presents a set of description schemes (DS) dealing with video programs, users and devices. Following MPEG-7 terminology, a description of an AV document includes descriptors (termed Ds), which specify the syntax and semantics of a representation entity for a feature of the AV data, and description schemes (termed DSs) which specify the structure and semantics of a set of Ds and DSs. The Program DS is used to describe the physical structure as well as the semantic content of a video program. It focuses on the visual information only. The physical structure is described by the temporal organization of the sequence (segments), the spatial organization of images (regions) as well as the spatio-temporal structure of the video (regions with motion). The semantic description is built around objects and events. Finally, the physical and semantic descriptions are related by a set of links defining where or when instances of specific semantic notions can be found. The User DS is used to describe the personal preferences and usage patterns of a user. It facilitates a smart personalizable device that records and presents to the user audio and video information based upon the users preferences, prior viewing and listening habits, as well as personal characteristics. Finally, the Device DS keeps a record of the users of the device, available programs, and a description of device capabilities. It allows a device to prepare itself based on the existing users, profiles and available programs. These three types of DSs and the common set of descriptors that they share are designed to support personalization, efficient management of AV information and the expected variability in the capabilities of AV information access devices.

CrackIT — An image processing toolbox for crack detection and characterization

This paper presents a comprehensive set of image processing algorithms for detection and characterization of road pavement surface crack distresses, which is being made available to the research community. The toolbox, in the Matlab environment, includes algorithms to preprocess images, to detect cracks and characterize them into types, based on image processing and pattern recognition techniques, as well as modules devoted to the performance evaluation of crack detection and characterization solutions. A sample database of 84 pavement surface images taken during a traditional road survey is provided with the toolbox, since no pavement image databases are publicly available for crack detection and characterization evaluation purposes. Results achieved applying the proposed toolbox to the sample database are discussed, illustrating the potential of the available algorithms.

Biometric identification through palm and dorsal hand vein patterns

Hand vein patterns are among the biometric traits being investigated today for identification purposes, attracting interest from both the research community and industry. This paper presents a multimodal system that combines hand-palm vein and hand-dorsal vein biometrics information at the score level. The palm and dorsal veins are considered as texture samples being automatically extracted from the users hand image. A 2D Gabor filter is employed for texture feature extraction. For matching, the newly acquired biometric samples are compared with those stored in the system database, at the enrolment stage. The metric used is based on the Hamming distance. A palm and dorsal vein database is built using the proposed acquisition system. This paper proposes a novel multimodal system that combines palm and dorsal vein information at the score level, with the experimental results showing that with the proposed system a much lower Equal Error Rate (EER) can be achieved, in comparison to existing unimodal systems.