Roland Mörzinger

Automatic image annotation using visual content and folksonomies

Automatic image annotation is an important and challenging task, and becomes increasingly necessary when managing large image collections. This paper describes techniques for automatic image annotation that take advantage of collaboratively annotated image databases, so called visual folksonomies. Our approach applies two techniques based on image analysis: First, classification annotates images with a controlled vocabulary and second tag propagation along visually similar images. The latter propagates user generated, folksonomic annotations and is therefore capable of dealing with an unlimited vocabulary. Experiments with a pool of Flickr images demonstrate the high accuracy and efficiency of the proposed methods in the task of automatic image annotation. Both techniques were applied in the prototypical tag recommender “tagr”.

Recommending Tags for Pictures Based on Text, Visual Content and User Context

Imagine you are member of an online social system and want to upload a picture into the community pool. In current social software systems, you can probably tag your photo, share it or send it to a photo printing service and multiple other stuff. The system creates around you a space full of pictures, other interesting content (descriptions, comments) and full of users as well. The one thing current systems do not do, is understand what your pictures are about. We present here a collection of functionalities that make a step in that direction when put together to be consumed by a tag recommendation system for pictures. We use the data richness inherent in social online environments for recommending tags by analysing different aspects of the same data (text, visual content and user context). We also give an assessment of the quality of thus recommended tags.

Patent image retrieval: a survey

Drawings are an important component of patents, and many search tasks in the intellectual property domain rely on the comparison of patent drawings. In this paper, we begin with a review of algorithms developed for the automated retrieval of similar images in the patent domain. There is however a larger body of research dedicated to analysis and retrieval of images found in technical documents that is also applicable to the images found in patents. We present an overview of this research for technical drawings, diagrams, charts, plots and chemical structures. Finally, we discuss the evaluation of image retrieval for patents, including short descriptions of the new patent image evaluation tasks in the CLEF-IP and TREC-CHEM evaluation campaigns in 2011.

Using Gait Features for Improving Walking People Detection

In this paper, we explore a new approach for enriching the HoG method for pedestrian detection in an unconstrained outdoor environment. The proposed algorithm is based on using gait motion since the rhythmic footprint pattern for walking people is considered the stable and characteristic feature for the detection of walking people. The novelty of our approach is motivated by the latest research for people identification using gait. The experimental results confirmed the robustness of our method to enhance HoG to detect walking people as well as to discriminate between single walking subject, groups of people and vehicles with a detection rate of 100%. Furthermore, the results revealed the potential of our method to be used in visual surveillance systems for identity tracking over different camera views.

Real-time detection of unusual regions in image streams

Automatic and real-time identification of unusual incidents is important for event detection and alarm systems. In todays camera surveillance solutions video streams are displayed on-screen for human operators, e.g. in large multi-screen control centers. This in turn requires the attention of operators for unusual events and urgent response. This paper presents a method for the automatic identification of unusual visual content in video streams real-time. In contrast to explicitly modeling specific unusual events, the proposed approach incrementally learns the usual appearances from the visual source and simultaneously identifies potential unusual image regions in the scene. Experiments demonstrate the general applicability on a variety of large-scale datasets including different scenes from public web cams and from traffic monitoring. To further demonstrate the real-time capabilities of the unusual scene detection we actively control a Pan-Tilt-Zoom camera to get close up views of the unusual incidents.

Towards Fully Automatic Image Segmentation Evaluation

Spatial region (image) segmentation is a fundamental step for many computer vision applications. Although many methods have been proposed, less work has been done in developing suitable evaluation methodologies for comparing different approaches. The main problem of general purpose segmentation evaluation is the dilemma between objectivity and generality. Recently, figure ground segmentation evaluation has been proposed to solve this problem by defining an unambiguous ground truth using the most salient foreground object. Although the annotation of a single foreground object is less complex than the annotation of all regions within an image, it is still quite time consuming, especially for videos. A novel framework incorporating background subtraction for automatic ground truth generation and different foreground evaluation measures is proposed, that allows to effectively and efficiently evaluate the performance of image segmentation approaches. The experiments show that the objective measures are comparable to the subjective assessment and that there is only a slight difference between manually annotated and automatically generated ground truth.

Automatic Quality Analysis for Film and Video Restoration

A considerable amount of work in larger film and video restoration projects is dedicated to manually exploring the audiovisual content in order to estimate the costs for restoration and to plan the restoration. Manual exploration is a significant cost factor. In this paper we propose automatic content analysis algorithms and summarization techniques which allow the reduction of manual inspection time in a software based restoration environment. The throughput requirement for analysis of dust and other defects is reached by sparse application of the detectors in the image sequence while retaining sufficient detection accuracy. Analysis result metadata are represented in a MPEG-7 standard compliant way. The proposed defect summary visualization tools facilitate efficient exploration of visually impaired content by the user.

Rapid and Reliable Detection of Film Grain Noise

The knowledge of material specific film grain characteristics can significantly improve the performance of digital film processing algorithms. This paper proposes a rapid and reliable detector for film grain properties. First, homogeneous blocks concerning intensity and texture are determined by a novel measure calculated in the frequency domain. Second, based on these blocks the signal-dependent grain noise level is estimated and an image region of pure film grain is detected. Evaluation based on a wide range of structured images with high frequent electronic noise and low frequent film grain, shows that the method performs with a worst case estimation error of 2.54 dB in typical movie quality (PSNR of 25-40 dB). This meets the requirements of real applications such as digital film restoration and special effects compositing.

Patent images - a glass-encased tool: opening the case

The paper discusses the problem of patent image retrieval. It describes the issues faced when extracting semantic data of images in patents, as well as an integration framework between the data thus extracted and semantic information extracted from text. Combining the two sources of knowledge is on the wish list of many patent information users, as current systems either search only the textual data, or have extremely limited image processing functionality. In practice in the patent domain, depictions of the product or method are often vital to the understanding of the invention. Yet they are almost completely unsearchable. They are tools enclosed in a glass case, at which we can look, but of which we cannot really make use. The IMPEx Project (Image Mining for Patent Exploration) cracks open this case with a new focus on processing this particular type of images. This paper presents the motivations, status and aims of the project.

Automatic Inter-image Homography Estimation from Person Detections

Inter-image homographies are essential for many differenttasks involving projective geometry. This paper proposesan adaptive correspondence estimation approach betweenperson detections in a planar scene not relying oncorrespondence features as it is the case in many otherRANSAC-based approaches. The result is a planar interimagehomography calculated from estimated point correspondences.The approach is self-configurable, adaptiveand provides robustness over time by exploiting temporaland geometric information. We demonstrate the manifoldapplicability of the proposed approach on a variety ofdatasets. Improved results compared to a common baselineapproach are shown and the influence of error sources suchas missed detections, false detections and non overlappingfield of views is investigated.