Matthew Roach

Video genre classification using dynamics

The problem addressed here is the classification of videos at the highest level into pre-defined genre. The approach adopted is based on the dynamic content of short sequences (/spl sim/30 secs). This paper presents two methods of extracting motion from a video sequence: foreground object motion and background camera motion. These dynamics are extracted, processed and applied to classify 3 broad classes: sports, cartoons and news. Experimental results for this 3 class problem give error rates of 17%, 8% and 6% for camera motion, object motion and both combined respectively, on /spl sim/30 second sequences.

Motion-based classification of cartoons

This paper describes a simple high-level classification of multimedia broadcast material into cartoon non-cartoon. The input video sequences are from a broad range of material which is representative of entertainment viewing. Classification of this type of high-level video genre is difficult because of its large inter-class variation. The task is made more difficult when classification is over a small time (10s of seconds) introducing a great deal of intra-class variation. This paper presents a purely dynamic based approach for content-based classification of video sequences in the form of a new global motion measure of foreground objects. Experiments are reported on a diverse database consisting of: 8 cartoon and 20 non-cartoon sequences. Results are shown in identification error rates against time of sequence used for classification. The system produces a best identification error rate of 3% on 66 separate decisions based on 23 second sequences trained using a total of /spl sim/20 minutes of video.

Enhancing face detection in colour images using a skin probability map

The paper describes the development and quantitative assessment of an approach to face detection (FD), with the application of image classification in mind. The approach adopted is a direct extension of an earlier approach by T.S. Huang and G.Z. Yang (1994). Huangs intensity based approach is found to be susceptible to variations in lighting conditions and complex backgrounds. It is hypothesised that by integrating colour information into Huangs approach, the number of false faces can be reduced. A skin probability map (SPM) is generated from a large quantity of labeled data (530 images containing faces and 714 images that do not) and is used to pre-process colour test images. The SPM allows image regions to be ranked in terms of their skin content, thus removing improbable face regions. The performance improvements are shown in terms of false acceptance (FA) and false rejection (FR) scores. As a front-end to Huangs approach, the benefits of skin segmentation can be seen by a reduction in the FA score from 79% to 15% with a negligible impact on FR.

Video genre verification using both acoustic and visual modes

This paper reports on the verification of the video genre: sport, cartoon, news, commercial and music. Results for the two modes, acoustic and visual, and for combined modes show an average equal error rate (ERR) of 16%, 15% and 10%, respectively. These reflect verification accuracy and as such are believed to be the first of their kind; previously published work has focused on closed set identification, assuming the video is known to belong to one of a fixed set. The results also demonstrate the influence of the genre to be classified: the best performance for the visual mode has an EER of 4% (cartoons), and the best performance for the acoustic mode has EER of 0.6% (news). Finally, the combination of the modes presents a more consistent accuracy across the five genre with an EER of 10%.

Acoustic and facial features for speaker recognition

This paper gives an insight into biometrics used for speaker recognition. Three different biometrics are presented, based on: acoustic, geometric lip, and holistic facial features. Experiments are carried out using a corpus of the DAVID audio-visual database. Recognition accuracy is found to be similar in the 2 domains. The geometric visual feature is based on a method of signature coding of the contour of the lips and the holistic feature is based on a mean dynamic signature, a method of capturing the motions of the face during a spoken utterance. Physical biometrics (static measurements) demand only small model sizes, perhaps just a single template, and therefore require less training data. Conversely behavioral biometrics contain more variation and demand more training data.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

Camera Motion Extraction Using Correlation for Motion-Based Video Classification

This paper considers camera motion extraction with application to automatic video classification. Video motion is subdivided into 3 components, one of which, camera motion, is considered here. The extraction of the camera motion is based on correlation. Both subjective and objective measures of the performance of the camera motion extraction are presented. This approach is shown to be simple but efficient and effective. This form is separated and extracted as a discriminant for video classification. In a simple classification experiment it is shown that sport and non-sport videos can be classified with an identification rate of 80%. The system is shown to be able to verify the genre of a short sequence (only 12 seconds), for sport and non-sport, with a false acceptance rate of 10% on arbitrarily chosen test sequences.