Kathrin Berkner

Multimedia Clip Generation From Documents for Browsing on Mobile Devices

Small displays on mobile handheld devices, such as personal digital assistants (PDAs) and cellular phones, are the bottlenecks for usability of most content browsing applications. Generally, conventional content such as documents and Web pages need to be modified for effective presentation on mobile devices. This paper proposes a novel visualization for documents, called multimedia thumbnails, which consists of text and image content converted into playable multimedia clips. A multimedia thumbnail utilizes visual and audio channels of small portable devices as well as both spatial and time dimensions to communicate text and image information of a single document. The proposed algorithm for generating multimedia thumbnails includes 1) a semantic document analysis step, where salient content from a source document is extracted; 2) an optimization step, where a subset of this extracted content is selected based on time, display, and application constraints; and 3) a composition step, where the selected visual and audible document content is combined into a multimedia thumbnail. Scalability of MMNails that allows generation of multimedia clips of various lengths is also described. A user study is presented that evaluates the effectiveness of the proposed multimedia thumbnail visualization.

Image formation analysis and high resolution image reconstruction for plenoptic imaging systems

Plenoptic imaging systems are often used for applications like refocusing, multimodal imaging, and multiview imaging. However, their resolution is limited to the number of lenslets. In this paper we investigate paraxial, incoherent, plenoptic image formation, and develop a method to recover some of the resolution for the case of a two-dimensional (2D) in-focus object. This enables the recovery of a conventional-resolution, 2D image from the data captured in a plenoptic system. We show simulation results for a plenoptic system with a known response and Gaussian sensor noise.

Multimedia thumbnails for documents

As small portable devices are becoming standard personal equipments, there is a great need for the adaptation of information content to small displays. Currently, no good solutions exist for viewing formatted documents, such as pdf documents, on these devices. Adapting content of web pages to small displays is usually achieved by complete redesign of a page or automatically reflowing text for small displays. Such techniques may not be applicable to documents whose format needs to be preserved. To address this problem, we propose a new document representation called Multimedia Thumbnail. Multimedia Thumbnail uses the visual and audio channels of small portable devices to communicate document information in form of a multimedia clip, which can be seen as a movie trailer for a document. Generation of such a clip includes a document analysis step, where salient document information is extracted, an optimization step, where the document information to be included in the thumbnail is determined based on display and time constraints, and a synthesis step, where visual and audible information are formed into a playable Multimedia Thumbnail. We also present user study results that evaluate an initial system design and point to further modification on analysis, optimization, and user interface components.

Light Field Scale-Depth Space Transform for Dense Depth Estimation

Recent development of hand-held plenoptic cameras has brought light field acquisition into many practical and low-cost imaging applications. We address a crucial challenge in light field data processing: dense depth estimation of 3D scenes captured by camera arrays or plenoptic cameras. We first propose a method for construction of light field scale-depth spaces, by convolving a given light field with a special kernel adapted to the light field structure. We detect local extrema in such scale-depth spaces, which indicate the regions of constant depth, and convert them to dense depth maps after solving occlusion conflicts in a consistent way across all views. Due to the multi-scale characterization of objects in proposed representations, our method provides depth estimates for both uniform and textured regions, where uniform regions with large spatial extent are captured at coarser scales and textured regions are found at finer scales. Experimental results on the HCI (Heidelberg Collaboratory for Image Processing) light field benchmark show that our method gives state of the art depth accuracy. We also show results on plenoptic images from the RAYTRIX camera and our plenoptic camera prototype.

A new hierarchical scheme for approximating the continuous wavelet transform with applications to edge detection

We propose an approximation of a continuous wavelet transform (CWT) that is based on a hierarchical scheme, similar to the fast discrete wavelet transform. This approximation keeps redundancies in time and scale. Furthermore, it preserves properties of the CWT regarding the characterization of singularities and leads to efficient applications in multiscale edge detection.

Adaptive representation of JPEG2000 images using header-based processing

To bridge the mismatch between the sizes of images and display devices, we present an efficient and automatic algorithm to create an adaptive image representation called SmartNail. Given a digital image and rectangular display frame smaller than the image, we define the SmartNail as an appropriately cropped part of a suitably scaled-down image. We choose the SmartNail-defining parameters - down-scaling factor and cropping location - to maximize a bit-allocation-based cost function that quantifies the visual importance of the image content in the SmartNail. For JPEG2000-encoded images, the SmartNail parameters can be determined using just the header information available in the encoded file. Hence, only the wavelet coefficients required to reconstruct the SmartNail need to be decoded from the entire JPEG2000 code stream. Consequently, the SmartNail construction requires minimal computation and memory requirements. Simulations demonstrate the effectiveness of the SmartNail representations.

Design and optimization of a near-eye multifocal display system for augmented reality

We present a binocular, near-eye multifocal display prototype capable of multiplexing 6-focal-plane, flicker-free virtual content into the eyes. The prototype’s hardware design, focal sampling strategy, and rendered multifocal images are presented.

Optimization of Spectrally Coded Mask for Multi-modal Plenoptic Camera

We introduce a framework to optimize the layout of a spectral filter mask inserted into the aperture of a plenoptic camera. The optimization merit function evaluates spectral crosstalk at the sensor caused by lens aberrations.

Computing a Multimedia Representation for Documents Given Time and Display Constraints

It is difficult to view multipage, high resolution documents on devices with small displays. As a solution, we introduce a multimedia thumbnail representation, which can be seen as a multimedia clip that provides an automated guided tour through a document. Multimedia thumbnails are automatically generated by taking a document image as input and first performing visual and audible information analysis on the document to determine salient document elements. Next, the time and information attributes for each document element are computed by taking into account the display and application constraints. An optimization routine, given a time constraint, selects elements to be included in the multimedia thumbnail. Last, the selected elements are synthesized into animated images and audio to create the final multimedia representation

A new wavelet-based approach to sharpening and smoothing of images in Besov spaces with applications to deblurring

The problem of image enhancement arises in many applications such as scanners, copiers and digital cameras. Enhancement often includes a denoising and a deblurring or sharpening step. Similar to image compression, state-of-the-art denoising techniques use wavelet bases instead of Fourier bases since wavelet domain processing provides local adaptation in smooth and non-smooth parts due to the theoretical link between wavelets and smoothness spaces. In this paper the same smoothness spaces are used to propose a way of performing sharpening and smoothing of signals with wavelets (WSS) in Besov spaces. As an application the completely wavelet-based enhancement of a scanned document is discussed.