Bernd Münzer

Relevance Segmentation of Laparoscopic Videos

In recent years, it became common to record video footage of laparoscopic surgeries. This leads to large video archives that are very hard to manage. They often contain a considerable portion of completely irrelevant scenes which waste storage capacity and hamper an efficient retrieval of relevant scenes. In this paper we (1) define three classes of irrelevant segments, (2) propose visual feature extraction methods to obtain irrelevance indicators for each class and (3) present an extensible framework to detect irrelevant segments in laparoscopic videos. The framework includes a training component that learns a prediction model using nonlinear regression with a generalized logistic function and a segment composition algorithm that derives segment boundaries from the fuzzy frame classifications. The experimental results show that our method performs very good both for the classification of individual frames and the detection of segment boundaries in videos and enables considerable storage space savings.

Content-based processing and analysis of endoscopic images and videos: A survey

In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with real-time assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.

Detection of circular content area in endoscopic videos

The actual content of endoscopic videos is typically limited to a circular area in the image center. This area has a dynamic position and size and is surrounded by a dark, but noisy border. In this paper we present a novel algorithm that (1) classifies which frames of an endoscopic video feature the circular content area and (2) determines its exact position and size, if present. This information is very useful for improving the performance of subsequent analysis techniques. It can also be used for more efficient video encoding and economic printing of still images in findings and reports. The evaluation shows that the proposed method is very accurate, robust and efficient in terms of runtime.

Investigation of the Impact of Compression on the Perceptional Quality of Laparoscopic Videos

In recent years it has become common practice to archive video recordings of laparoscopic surgeries for documentation purposes and for retrospective review. Typically, the videos are captured in High Definition (HD) format but encoded with legacy coding standards like MPEG-2 requiring an enormous storage capacity. In this paper we present the results of a subjective quality assessment study with 37 medical experts. We identify appropriate encoding configurations for the H.264 AVC coding standard to guarantee a visually loss less quality with a significant bitrate reduction. Further, we show that it is not necessary to capture the highest possible quality for documentation and retrospective analysis. A lower technical quality with a substantially lower bitrate still provides sufficient semantic quality. We finally present basic recommendations for an efficient encoding strategy with an appropriate tradeoff between visual quality and bitrate.

Improving encoding efficiency of endoscopic videos by using circle detection based border overlays

Videos of endoscopic procedures typically feature a circular content area in the image center. This area is surrounded by a dark border that carries no relevant information but is subject to noise. Thus, a considerable proportion of the available bitrate has to be wasted to encode the border regions. We propose to superimpose the border regions with a homogenous black mask so that it can be encoded efficiently with skipped macroblocks. To determine the exact position and size of the circular content area we use an efficient circle detection algorithm. Through an evaluation with 138 videos we show that the border overlay can significantly reduce the bitrate without degrading the visual quality of the content area.

Smart Video Browsing with Augmented Navigation Bars

While accuracy and speed get a lot of attention in video retrieval research, the investigation of interactive retrieval tools gets less attention and is often regarded as trivial. We want to show that even simple ideas have potential to improve the retrieval performance by giving some automated support to the browsing user. We present a video browsing concept where video segments are clustered in several latent classes of similar content. The navigation bars of our video browser are augmented with different colors indicating where elements of these clusters are located. As humans are able to classify the content of clusters fast, they can benefit from this information when browsing a video. We present a study where we investigated how humans can be supported in different video browsing tasks with a color-based and a motion-based clustering of video content.

The ITEC Collaborative Video Search System at the Video Browser Showdown 2018

We present our video search system for the Video Browser Showdown (VBS) 2018 competition. It is based on the collaborative system used in 2017, which already performed well but also revealed high potential for improvement. Hence, based on our experience we introduce several major improvements, particularly (1) a strong optimization of similarity search, (2) various improvements for concept-based search, (3) a new flexible video inspector view, and (4) extended collaboration features, as well as numerous minor adjustments and enhancements, mainly concerning the user interface and means of user interaction. Moreover, we present a spectator view that visualizes the current activity of the team members to the audience to make the competition more attractive.

A Tool to Support Surgical Quality Assessment

In the domain of medical endoscopy an increasing number of surgeons nowadays store video recordings of their interventions in a huge video archive. Among some other purposes, the videos are used for post-hoc surgical quality assessment, since objective assessment of surgical procedures has been identified as essential component for improvement of surgical quality. Currently, such assessment is performed manually and for selected procedures only, since the amount of data and cumbersome interaction is very time-consuming. In the future, quality assessment should be carried out comprehensively and systematically by means of automated assessment algorithms. In this demo paper, we present a tool that supports human assessors in collecting manual annotations and therefore should help them to deal with the huge amount of visual data more efficiently. These annotations will be analyzed and used as training data in the future.

Frame-Based Classification of Operation Phases in Cataract Surgery Videos

Cataract surgeries are frequently performed to correct a lens opacification of the human eye, which usually appears in the course of aging. These surgeries are conducted with the help of a microscope and are typically recorded on video for later inspection and educational purposes. However, post-hoc visual analysis of video recordings is cumbersome and time-consuming for surgeons if there is no navigation support, such as bookmarks to specific operation phases. To prepare the way for an automatic detection of operation phases in cataract surgery videos, we investigate the effectiveness of a deep convolutional neural network (CNN) to automatically assign video frames to operation phases, which can be regarded as a single-label multi-class classification problem. In absence of public datasets of cataract surgery videos, we provide a dataset of 21 videos of standardized cataract surgeries and use it to train and evaluate our CNN classifier. Experimental results display a mean F1-score of about 68% for frame-based operation phase classification, which can be further improved to 75% when considering temporal information of video frames in the CNN architecture.

Lapgyn4: a dataset for 4 automatic content analysis problems in the domain of laparoscopic gynecology

Modern imaging technology enables medical practitioners to perform minimally invasive surgery (MIS), i.e. a variety of medical interventions inflicting minimal trauma upon patients, hence, greatly improving their recoveries. Not only patients but also surgeons can benefit from this technology, as recorded media can be utilized for speeding-up tedious and time-consuming tasks such as treatment planning or case documentation. In order to improve the predominantly manually conducted process of analyzing said media, with this work we publish four datasets extracted from gynecologic, laparoscopic interventions with the intend on encouraging research in the field of post-surgical automatic media analysis. These datasets are designed with the following use cases in mind: medical image retrieval based on a query image, detection of instrument counts, surgical actions and anatomical structures, as well as distinguishing on which anatomical structure a certain action is performed. Furthermore, we provide suggestions for evaluation metrics and first baseline experiments.