Luís Filipe Teixeira

Video object matching across multiple independent views using local descriptors and adaptive learning

Object detection and tracking is an essential preliminary task in event analysis systems (e.g. visual surveillance). Typically objects are extracted and tagged, forming representative tracks of their activity. Tagging is usually performed by probabilistic data association, however, in systems capturing disjoint areas it is often not possible to establish such associations, as data may have been collected at different times or in different locations. In this case, appearance matching is a valuable aid. We propose using bag-of-visterms, i.e. an histogram of quantized local feature descriptors, to represent and match tracked objects. This method has proven to be effective for object matching and classification in image retrieval applications, where descriptors can be extracted a priori. An important difference in event analysis systems is that relevant information is typically restricted to the foreground. Descriptors can, therefore, be extracted faster, approaching real-time requirements. Also, unlike image retrieval, objects can change over time and therefore their model needs to be updated continuously. Incremental or adaptive learning is used to tackle this problem. Using independent tracks of 30 different persons, we show that the bag-of-visterms representation effectively discriminates visual object tracks and that it presents high resilience to incorrect object segmentation. Additionally, this methodology allows the construction of scalable object models that can be used to match tracks across independent views.

Breast Contour Detection with Stable Paths

Breast cancer conservative treatment (BCCT), due to its proven oncological safety, is considered, when feasible, the gold standard of breast cancer treatment. However, aesthetic results are heterogeneous and difficult to evaluate in a standardized way, due to the lack of reproducibility of the subjective methods usually applied. The objective assessment methods, considered in the past as being less capable of evaluating all aspects of BCCT, are nowadays being preferred to overcome the drawbacks of the subjective evaluation. A computer-aided medical system was recently developed to objectively and automatically evaluate the aesthetic result of BCCT. In this system, the detection of the breast contour on the patient’s digital photograph is a necessary step to extract the features subsequently used in the evaluation process. In this paper an algorithm based on the shortest path on a graph is proposed to detect automatically the breast contour. The proposed method extends an existing semi-automatic algorithm for the same purpose. A comprehensive comparison with manually-drawn contours reveals the strength of the proposed method.

Analysis of object description methods in a video object tracking environment

A key issue in video object tracking is the representation of the objects and how effectively it discriminates between different objects. Several techniques have been proposed, but without a generally accepted method. While analysis and comparisons of these individual methods have been presented in the literature, their evaluation as part of a global solution has been overlooked. The appearance model for the objects is a component of a video object tracking framework, depending on previous processing stages and affecting those that succeed it. As a result, these interdependencies should be taken into account when analysing the performance of the object description techniques. We propose an integrated analysis of object descriptors and appearance models through their comparison in a common object tracking solution. The goal is to contribute to a better understanding of object description methods and their impact on the tracking process. Our contributions are threefold: propose a novel descriptor evaluation and characterisation paradigm; perform the first integrated analysis of state-of-the-art description methods in a scenario of people tracking; put forward some ideas for appearance models to use in this context. This work provides foundations for future tests and the proposed assessment approach contributes to the informed selection of techniques more adequately for a given tracking application context.

Learning from evolving video streams in a multi-camera scenario

Nowadays, video surveillance systems are taking the first steps toward automation, in order to ease the burden on human resources as well as to avoid human error. As the underlying data distribution and the number of concepts change over time, the conventional learning algorithms fail to provide reliable solutions for this setting. In this paper, we formalize a learning concept suitable for multi-camera video surveillance and propose a learning methodology adapted to that new paradigm. The proposed framework resorts to the universal background model to robustly learn individual object models from small samples and to more effectively detect novel classes. The individual models are incrementally updated in an ensemble-based approach, with older models being progressively forgotten. The framework is designed to detect and label new concepts automatically. The system is also designed to exploit active learning strategies, in order to interact wisely with operator, requesting assistance in the most ambiguous to classify observations. The experimental results obtained both on real and synthetic data sets verify the usefulness of the proposed approach.

Automatic description of object appearances in a wide-area surveillance scenario

In this paper we present a complete system for object tracking over multiple uncalibrated cameras with or without overlapping fields of view. We employ an approach based on the bag-of-visterms technique to represent and match tracked objects. The tracks are compared with a global object model based on an ensemble of individual object models. The system can globally recognise objects and minimise common tracking problems such as track drift or split. The output is a timeline representing the objects present in a given multi-camera scene. The methods employed in the system are online and can be optimized to operate in real-time.

MarsyasX: multimedia dataflow processing with implicit patching

The design and implementation of multimedia signal processing systems is challenging especially when efficiency and real-time performance is desired. In many modern applications, software systems must be able to handle multiple flows of various types of multimedia data such as audio and video. Researchers frequently have to rely on a combination of different software tools for each modality to assemble proof-of-concept systems that are inefficient, brittle and hard to maintain. Marsyas is a software framework originally developed to address these issues in the domain of audio processing. In this paper we describe MarsyasX, a new open-source cross-modal analysis framework that aims at a broader score of applications. It follows a dataflow architecture where complex networks of processing objects can be assembled to form systems that can handle multiple and different types of multimedia flows with expressiveness and efficiency.

10-Year follow-up of 621 patients treated using high-dose rate brachytherapy as ambulatory boost technique in conservative breast cancer treatment.

PURPOSE Breast conserving treatment, consisting of lumpectomy followed by whole-breast irradiation, is considered the standard of care in early-stage breast cancer. Randomized studies have reported that delivering boost doses to tumor bed improves local control rates, particularly in young women. This study sought to evaluate local control and cosmetic results of delivering boost doses using a high-dose-rate (HDR) brachytherapy (HDRBT) in breast cancer conservative treatment. METHODS We included 621 T1-T2, N0-N1 breast cancer patients who underwent lumpectomy, external irradiation (44Gy over 5weeks), and a boost dose of two fractions of 5Gy to the tumor bed by means of HDR iridium brachytherapy. Implantation was performed during the lumpectomy or 2-3weeks after external irradiation. Population characteristics were as follows: pTis=11.6%; pT1=63.4%; pT2=25.0%; median tumor size=1.5cm; histology: ductal carcinoma in situ (DCIS): 72 (11.6%); infiltrative ductal carcinoma (IDC): 471 (75.8%); other: 78 (12.6%). For IDCs, the surgical margins were positive in 38cases (6.2%) and an extensive intraductal component was present in 254 cases. RESULTS With a median follow-up of 10.3years, 47 local relapses were observed (10-year local relapse rate: 7.4%). Small-volume implantation (V100<45cc) and ductal carcinoma in situ histology both significantly correlated with local relapse. The 10-year overall survival was 91%. Cosmetic results were evaluated in 264patients, proving excellent in 58 (22%), good in 153 (58%), fair in 40 (15%), and poor in 13 (5%). CONCLUSIONS Small implant volume and ductal carcinoma in situ histology significantly correlated with local relapse following HDR brachytherapy dose boost in breast cancer conservative treatment. Modern image-guided breast brachytherapy techniques using surgical clips as a guide may decrease potential treatment targeting errors, consequently improving local control without increasing toxicity.

H.264 Rate-Distortion Analysis Using Subjective Quality Metric

In this paper we provide an analysis of rate-distortion (R-D) relationship in an H.264 codec using as quality metric Structural Similarly Information (SSIM). This study focus on the quantization parameter, namely rate-quantization (R-Q) functions and distortion-quantization (D-Q) functions. Together, these functions allow a better understanding of the rate-distortion (R-D) behaviour of an H.264 video codec, which is the key issue of optimum bit allocation. Initial results are presented and discussed.

Statistical multiplexing of H.264 programs

The advent of H.264/AVC is going to change the way digital television programs are broadcast. Each program can be independently encoded or jointly encoded resulting thus in a more efficient way to distribute the available channel bandwidth. This paper presents a combined coding scheme for multi-program video transmission in which the channel capacity is distributed among the programs according to the program complexities. A complexity bit rate control algorithm based on the structural similarity index (SSIM) is proposed. SSIM metric is presented under the hypothesis that the human visual system (HSV) is very specialized in extracting structural information from a video sequence but not in extracting the errors. Thus, a measurement on structural distortion should give a better correlation to the subjective impression. Current simulations have demonstrated very promising results showing that the algorithm can effectively control the complexity of the multi-program encoding process whilst improving overall subjective.

Cascaded change detection for foreground segmentation

The extraction of relevant objects (foreground) from a background is an important first step in many applications. We propose a technique that tackles this problem using a cascade of change detection tests, including noise-induced, illumination variation and structural changes. An objective comparison of pixel-wise modellingmethods is first presented. Given its best relation performance/complexity, the mixture of Gaussians was chosen to be used in the proposed method to detect structural changes. Experimental results show that the cascade technique consistently outperforms the commonly used mixture of Gaussians, without additional post-processing and without the expense of processing overheads.