Laszlo Böszörmenyi

Relevance Segmentation of Laparoscopic Videos

In recent years, it became common to record video footage of laparoscopic surgeries. This leads to large video archives that are very hard to manage. They often contain a considerable portion of completely irrelevant scenes which waste storage capacity and hamper an efficient retrieval of relevant scenes. In this paper we (1) define three classes of irrelevant segments, (2) propose visual feature extraction methods to obtain irrelevance indicators for each class and (3) present an extensible framework to detect irrelevant segments in laparoscopic videos. The framework includes a training component that learns a prediction model using nonlinear regression with a generalized logistic function and a segment composition algorithm that derives segment boundaries from the fuzzy frame classifications. The experimental results show that our method performs very good both for the classification of individual frames and the detection of segment boundaries in videos and enables considerable storage space savings.

Keyframe extraction in endoscopic video

In medical endoscopy more and more surgeons archive the recorded video streams in a long-term storage. One reason for this development, which is enforced by law in some countries, is to have evidence in case of lawsuits from patients. Another more practical reason is to allow later inspection of previous procedures and also to use parts of such videos for research and for training. However, due to the dramatic amount of video data recorded in a hospital on a daily basis, it is very important to have good preview images for these videos in order to allow for quick filtering of undesired content and for easier browsing through such a video archive. Unfortunately, common shot detection and keyframe extraction methods cannot be used for that video data, because these videos contain unedited and highly similar content, especially in terms of color and texture, and no shot boundaries at all. We propose a new keyframe extraction approach for this special video domain and show that our method is significantly better than a previously proposed approach.

A novel tool for summarization of arthroscopic videos

Arthroscopic surgery is a minimally invasive procedure that uses a small camera to generate video streams, which are recorded and subsequently archived. In this paper we present a video summarization tool and demonstrate how it can be successfully used in the domain of arthroscopic videos. The proposed tool generates a keyframe-based summary, which clusters visually similar frames based on user-selected visual features and appropriate dissimilarity metrics. We discuss how this tool can be used for arthroscopic videos, taking advantage of several domain-specific aspects, without losing its ability to work on general-purpose videos. Experimental results confirm the feasibility of the proposed approach and encourage extending it to other application domains.

Instant Video Browsing: A Tool for Fast Non-sequential Hierarchical Video Browsing

We introduce an easy-to-use video browsing tool which assists users in getting a quick overview of videos as well as in finding segments of interest. It provides a parallel and a tree-based view for browsing the content of videos - or even video collections - in a hierarchical, non-sequential manner. The tool has a plug-in architecture and can be extended both by further presentation methods and by video analysis algorithms.

Content-based processing and analysis of endoscopic images and videos: A survey

In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with real-time assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.

Context-Aware UPnP-AV Services for Adaptive Home Multimedia Systems

One possibility to provide mobile multimedia in domestic multimedia systems is the use of Universal Plug and Play Audio Visual (UPnP-AV) devices. In a standard UPnP-AV scenario, multimedia content provided by a Media Server device is streamed to Media Renderer devices by the initiation of a Control Point. However, there is no provisioning of context-aware multimedia content customization. This paper presents an enhancement of standard UPnP-AV services for home multimedia environments regarding context awareness. It comes up with context profile definitions, shows how this context information can be queried from the Media Renderers, and illustrates how a Control Point can use this information to tailor a media stream from the Media Server to one or more Media Renderers. Moreover, since a standard Control Point implementation only queries one Media Server at a time, there is no global view on the content of all Media Servers in the UPnP-AV network. This paper also presents an approach of multimedia content integration on the Media Server side that provides fast search for content on the network. Finally, a number of performance measurements show the overhead costs of our enhancements to UPnP-AV in order to achieve the benefits.

Segmentation of recorded endoscopic videos by detecting significant motion changes

In the medical domain it has become common to store recordings of endoscopic surgeries or procedures. The storage of these endoscopic videos provides not only evidence of the work of the surgeons but also facilitates research, the training of new surgeons and supports explanations to the patients. However, an endoscopic video archive, where tens or hundreds of new videos are added each day, needs content-based analysis in order to provide content-based search. A fundamental first step in content analysis is the segmentation of the video. We propose a new method for segmentation of endoscopic videos, based on spatial and temporal differences of motion in these videos. Through an evaluation with 20 videos we show that our approach provides reasonable performance.

Detection of circular content area in endoscopic videos

The actual content of endoscopic videos is typically limited to a circular area in the image center. This area has a dynamic position and size and is surrounded by a dark, but noisy border. In this paper we present a novel algorithm that (1) classifies which frames of an endoscopic video feature the circular content area and (2) determines its exact position and size, if present. This information is very useful for improving the performance of subsequent analysis techniques. It can also be used for more efficient video encoding and economic printing of still images in findings and reports. The evaluation shows that the proposed method is very accurate, robust and efficient in terms of runtime.

Towards a self-organizing replication model for non-sequential media access

Due to the vast amount of video available in the Internet new access patterns emerge. Users do not always want to watch all of the content sequentially - such as in a movie - but want to pick specific parts, which are interesting for them. Based on a model of small and semantically meaningful and active video units, we derive an artificial hormone replication system that provides the flexibility for non-sequential access of units. We evaluate our model by simulation and compare it to a reference system. We show that simple local decisions contribute to global properties such as delay and robustness. We further introduce a clean-up function, which leads to adaptive management of the number of replicas.

Investigation of the Impact of Compression on the Perceptional Quality of Laparoscopic Videos

In recent years it has become common practice to archive video recordings of laparoscopic surgeries for documentation purposes and for retrospective review. Typically, the videos are captured in High Definition (HD) format but encoded with legacy coding standards like MPEG-2 requiring an enormous storage capacity. In this paper we present the results of a subjective quality assessment study with 37 medical experts. We identify appropriate encoding configurations for the H.264 AVC coding standard to guarantee a visually loss less quality with a significant bitrate reduction. Further, we show that it is not necessary to capture the highest possible quality for documentation and retrospective analysis. A lower technical quality with a substantially lower bitrate still provides sufficient semantic quality. We finally present basic recommendations for an efficient encoding strategy with an appropriate tradeoff between visual quality and bitrate.