Wallapak Tavanapong

Video delivery technologies for large-scale deployment of multimedia applications

Deployment of a large-scale multimedia streaming application requires an enormous amount of server and network resources. The simplest delivery technique allocates server resources for each specific request. This technique is very expensive and is not scalable to support a very large user community such as the Internet. Hence, the past decade has witnessed tremendous research efforts to facilitate cost-effective, large-scale deployment of multimedia streaming applications. In this paper, we describe three complementary research approaches: server transmission schemes using multicast, streaming strategies with application layer multicast, and proxy caching techniques. We discuss pros and cons of these technologies and provide our observations on current business solutions.

Image retrieval based on regions of interest

Query-by-example is the most popular query model in recent content-based image retrieval (CBIR) systems. A typical query image includes relevant objects (e.g., Eiffel Tower), but also irrelevant image areas (including background). The irrelevant areas limit the effectiveness of existing CBIR systems. To overcome this limitation, the system must be able to determine similarity based on relevant regions alone. We call this class of queries region-of-interest (ROI) queries and propose a technique for processing them in a sampling-based matching framework. A new similarity model is presented and an indexing technique for this new environment is proposed. Our experimental results confirm that traditional approaches, such as Local Color Histogram and Correlogram, suffer from the involvement of irrelevant regions. Our method can handle ROI queries and provide significantly better performance. We also assessed the performance of the proposed indexing technique. The results clearly show that our retrieval procedure is effective for large image data sets.

Polyp Detection in Colonoscopy Video using Elliptical Shape Feature

Early detection of polyps and cancers is one of the most important goals of colonoscopy. Computer-based analysis of video files using texture features, as has been proposed for polyps of the stomach and colon, has two major limitations: this method uses a fixed size analysis window and relies heavily on a training set of images for accuracy. To overcome these limitations, we propose a new technique focusing on shape instead of texture in this paper. The proposed polyp region detection method is based on the elliptical shape that is common for nearly all small colon polyps.

Informative frame classification for endoscopy video

Advances in video technology allow inspection, diagnosis and treatment of the inside of the human body without or with very small scars. Flexible endoscopes are used to inspect the esophagus, stomach, small bowel, colon, and airways, whereas rigid endoscopes are used for a variety of minimal invasive surgeries (i.e., laparoscopy, arthroscopy, endoscopic neurosurgery). These endoscopes come in various sizes, but all have a tiny video camera at the tip. During an endoscopic procedure, the tiny video camera generates a video signal of the interior of the human organ, which is displayed on a monitor for real-time analysis by the physician. However, many out-of-focus frames are present in endoscopy videos because current endoscopes are equipped with a single, wide-angle lens that cannot be focused. We need to distinguish the out-of-focus frames from the in-focus frames to utilize the information of the out-of-focus and/or the in-focus frames for further automatic or semi-automatic computer-aided diagnosis (CAD). This classification can reduce the number of images to be viewed by a physician and to be analyzed by a CAD system. We call an out-of-focus frame a non-informative frame and an in-focus frame an informative frame. The out-of-focus frames have characteristics that are different from those of in-focus frames. In this paper, we propose two new techniques (edge-based and clustering-based) to classify video frames into two classes, informative and non-informative frames. However, because intensive specular reflections reduce the accuracy of the classification we also propose a specular reflection detection technique, and use the detected specular reflection information to increase the accuracy of informative frame classification. Our experimental studies indicate that precision, sensitivity, specificity, and accuracy for the specular reflection detection technique and the two informative frame classification techniques are greater than 90% and 95%, respectively.

Computer-Aided Detection of Diagnostic and Therapeutic Operations in Colonoscopy Videos

Colonoscopy is an endoscopic technique that allows a physician to inspect the inside of the human colon and to perform - if deemed necessary - at the same time a number of diagnostic and therapeutic operations. In order to see the inside of the colon, a video signal of the internal mucosa of the colon is generated by a tiny video camera at the tip of the endoscope and displayed on a monitor for real-time analysis by the physician. We have captured and stored these videos in digital format and call these colonoscopy videos. Based on new algorithms for instrument detection and shot segmentation, we introduce new spatio-temporal analysis techniques to automatically identify an operation shot - a segment of visual data in a colonoscopy video that corresponds to a diagnostic or therapeutic operation. Our experiments on real colonoscopy videos demonstrate the effectiveness of the proposed approach. The proposed techniques and software are useful for 1) postprocedure review for causes of complications due to diagnostic or therapeutic operations; 2) establishment of an effective content-based retrieval system to facilitate endoscopic research and education; 3) development of a systematic approach to assess and improve the procedural skills of endoscopists.

Measuring Objective Quality of Colonoscopy

Advances in video technology are being incorporated into todays healthcare practices. Colonoscopy is regarded as one of the most important diagnostic tools for colorectal cancer. Indeed, colonoscopy has contributed to a decline in the number of colorectal-cancer-related deaths. Although colonoscopy has become the preferred screening modality for prevention of colorectal cancer, recent data suggest that there is a significant miss rate for the detection of large polyps and cancers, and methods to investigate why this occurs are needed. To address this problem, we present a new computer-based method that analyzes a digitized video file of a colonoscopic procedure and produces a number of metrics that likely reflect the quality of the procedure. The method consists of a set of novel image-processing algorithms designed to address new technical challenges due to uncommon characteristics of videos captured during colonoscopy. As these measurements can be obtained automatically, our method enables future quality control in large-scale day-to-day medical practice, which is currently not feasible. In addition, our method can be adapted to other endoscopic procedures such as upper gastrointestinal endoscopy, enteroscopy, and bronchoscopy. Last but not least, our method may be useful to assess progress during colonoscopy training.

Automatic measurement of quality metrics for colonoscopy videos

Colonoscopy is the accepted screening method for detection of colorectal cancer or its precursor lesions, colorectal polyps. Indeed, colonoscopy has contributed to a decline in the number of colorectal cancer related deaths. However, not all cancers or large polyps are detected at the time of colonoscopy, and methods to investigate why this occurs are needed. We present a new computer-based method that allows automated measurement of a number of metrics that likely reflect the quality of the colonoscopic procedure. The method is based on analysis of a digitized video file created during colonoscopy, and produces information regarding insertion time, withdrawal time, images at the time of maximal intubation, the time and ratio of clear versus blurred or non-informative images, and a first estimate of effort performed by the endoscopist. As these metrics can be obtained automatically, our method allows future quality control in the day-to-day medical practice setting on a large scale. In addition, our method can be adapted to other healthcare procedures. Last but not least, our method may be useful to assess progress during colonoscopy training, or as part of endoscopic skills assessment evaluations.

Part-Based Multiderivative Edge Cross-Sectional Profiles for Polyp Detection in Colonoscopy

This paper presents a novel technique for automated detection of protruding polyps in colonoscopy images using edge cross-section profiles (ECSP). We propose a part-based multi-derivative ECSP that computes derivative functions of an edge cross-section profile and segments each of these profiles into parts. Therefore, we can model or extract features suitable for each part. Our features obtained from the parts can effectively describe complex properties of protruding polyps including the shape of the parts, texture, and protrusion and smoothness of the polyp surface. We evaluated our method against two existing polyp image detection techniques on 42 different polyps, including those with little protrusion. Each polyp has a large variation of appearance in viewing angles, light conditions, and scales in different images. The evaluation showed that our technique outperformed the existing techniques in both accuracy and analysis time. Our method has a higher area under the free-response receiver operating characteristic curve. For instance, when both techniques have a true positive rate for polyp image detection of 81.4%, the average number of false regions per image of our technique is 0.32 compared to 1.8 of the best existing technique under study. Additionally, our technique can precisely mark edges of candidate polyp regions as visual feedback. These results altogether indicate that our technique is promising to provide visual feedback of polyp regions in clinical practice.

Automatic real-time detection of endoscopic procedures using temporal features

Endoscopy is used for inspection of the inner surface of organs such as the colon. During endoscopic inspection of the colon or colonoscopy, a tiny video camera generates a video signal, which is displayed on a monitor for interpretation in real-time by physicians. In practice, these images are not typically captured, which may be attributed by lack of fully automated tools for capturing, analysis of important contents, and quick and easy retrieval of these contents. This paper presents the description and evaluation results of our novel software that uses new metrics based on image color and motion over time to automatically record all images of an individual endoscopic procedure into a single digitized video file. The software automatically discards out-patient video frames between different endoscopic procedures. We validated our software system on 2464 h of live video (over 265 million frames) from endoscopy units where colonoscopy and upper endoscopy were performed. Our previous classification method achieved a frame-based sensitivity of 100.00%, but only a specificity of 89.22%. Our new method achieved a frame-based sensitivity and specificity of 99.90% and 99.97%, a significant improvement. Our system is robust for day-to-day use in medical practice.

Multimedia and Medicine: Teammates for Better Disease Detection and Survival

Health care has a long history of adopting technology to save lives and improve the quality of living. Visual information is frequently applied for disease detection and assessment, and the established fields of computer vision and medical imaging provide essential tools. It is, however, a misconception that disease detection and assessment are provided exclusively by these fields and that they provide the solution for all challenges. Integration and analysis of data from several sources, real-time processing, and the assessment of usefulness for end-users are core competences of the multimedia community and are required for the successful improvement of health care systems. We have conducted initial investigations into two use cases surrounding diseases of the gastrointestinal (GI) tract, where the detection of abnormalities provides the largest chance of successful treatment if the initial observation of disease indicators occurs before the patient notices any symptoms. Although such detection is typically provided visually by applying an endoscope, we are facing a multitude of new multimedia challenges that differ between use cases. In real-time assistance for colonoscopy, we combine sensor information about camera position and direction to aid in detecting, investigate means for providing support to doctors in unobtrusive ways, and assist in reporting. In the area of large-scale capsular endoscopy, we investigate questions of scalability, performance and energy efficiency for the recording phase, and combine video summarization and retrieval questions for analysis.