Florian Niebling

Reducing application-stage latencies of interprocess communication techniques for real-time interactive systems

Latency jitter is a pressing problem in Virtual Reality (VR) applications. This paper analyzes latency jitter caused by typical interprocess communication (IPC) techniques commonly found in todays computer systems used for VR. Test programs measure the scalability and latencies for various IPC techniques, where increasing number of threads are performing the same task concurrently. We use four different implementations on a vanilla Linux kernel as well as on a real-time (RT) Linux kernel to further assess if a RT variant of a multiuser multiprocess operating system can prevent latency spikes and how this behavior would apply to different programming languages and IPC techniques. We found that Linux RT can limit the latency jitter at the cost of throughput for certain implementations. Further, coarse grained concurrency should be employed to avoid adding up of scheduler latencies, especially for native system space IPC, while actor systems are found to support a higher degree of concurrency granularity and a higher level of abstraction.

Towards browsing repositories of spatially oriented historic photographic images in 3D web environments

Archives and museums store vast collections of historical images of urban areas and make them publicly available through online platforms. Many of these images, often containing historic buildings and landscapes, can be oriented spatially using automatic methods such as structure from motion (SfM). Providing spatially and temporally oriented images of urban architecture, in combination with advanced searching and 2D/3D exploration techniques, offers new potentials in supporting historians in their research. We are developing a 3D web environment usable to historians to spatially search online media repositories containing historic photographic images. We combine 3D models of historic buildings with spatially oriented images, replacing text-based searching through meta-data with spatial and temporal browsing with respect to given focus points in historic city models.

Towards comparable evaluation methods and measures for timing behavior of virtual reality systems

A low latency is a fundamental timeliness requirement to reduce the potential risks of cyber sickness and to increase effectiveness, efficiency, and user experience of Virtual Reality Systems. The effects of uniform latency degradation based on mean or worst-case values are well researched. In contrast, the effects of latency jitter, the distribution pattern of latency changes over time has largely been ignored so far although todays consumer VR systems are extremely vulnerable in this respect. We investigate the applicability of the Walsh, generalized ESD, and the modified z-score test for the detection of outliers as one central latency distribution aspect. The tests are applied to well defined test cases mimicking typical timing behavior expected from concurrent architectures of today. We introduce accompanying graphical visualization methods to inspect, analyze and communicate the latency behavior of VR systems beyond simple mean or worst-case values. As a result, we propose a stacked modified z-score test for more detailed analysis.

Low-cost raycast-based coordinate system registration for consumer depth cameras

We present four raycast-based techniques that determine the transformation between a depth cameras coordinate system and the coordinate system defined by a rectangular surface. In addition, the surfaces dimensions are measured. In contrast to other approaches, these techniques limit additional hardware requirements to commonly available, low-cost artifacts and focus on simple non-laborious procedures. A preliminary study examining our Kinect v2-based proof of concept revealed promising first results. The utilized software is available as an open-source project.

Reducing application-stage latencies for real-time interactive systems

Latency is a pressing problem in Virtual Reality (VR) applications. Low latencies are required for VR to reduce perceptual artifacts and cyber sickness. Additionally, latency jitter denotes the variance in the pattern of latency changes which additionally may cause unwanted effects. This paper analyzes latency jitter caused by typical inter-thread communication (ITC) techniques commonly used in todays computer systems employed for VR, the influence of the operating system scheduler, and the effect of different garbage collection (GC) methods to understand their effect on latency spikes, here for different Java Virtual Machines (JVM). We measure the scalability and latencies for various ITC techniques with an increasing number of threads and actors performing prototypical concurrent tasks. Four different benchmark implementations on a vanilla Linux kernel as well as on a real-time (RT) Linux kernel assess if a RT variant of a multiuser multiprocess operating system can prevent latency spikes and how this behavior would apply to different programming languages and ITC techniques. We confirmed that scheduler and prioritization of the VR application both play an important role and identified the impact they have on the implementation strategies. Also, Linux RT can limit the latency jitter at the cost of throughput for certain implementations. As expected, the choice of a GC method also is critical and will change the latency patterns drastically. As a result, we suggest that coarse grained concurrency should be employed to avoid adding up of scheduler latencies and unwanted latency jitter for the native ITC case, while actor systems are found to support a higher degree of concurrency granularity and a higher level of abstraction.

Research and Communication of Urban History in 4D Using Historical Photographs – A Status Report of the Research Group UrbanHistory4D

This contribution shows the work of the junior research group UrbanHistory4D within one year. It explains the different technical and educational approaches when working with media repositories composed of diverse historical data. The group covers technical aspects like photogrammetry, information sciences and Augmented Reality (AR) as well as humanistic topics like history of art, user case and educational studies. In detail, different photogrammetric techniques, various image repositories, a user case study and possibilities of knowledge transfer are evaluated. The main focus in the first year was the development of two prototype applications – a 4D browser interface and an AR application – supported by the different fields of studies. These applications are still refined considering the results of the varying research topics. As an outcome the contribution presents a part of the project organization which plays an important role when working in such interdisciplinary groups.

4D Augmented City Models, Photogrammetric Creation and Dissemination

The availability of digital image repositories of historical photographs offers new possibilities to historians in their research. In addition to representing a large collection of data records themselves, image archives allow for new methods of research, from large-scale statistical analysis, to algorithmic generation of knowledge, such as historical 3D models, directly from these sources. In this paper, we explore methods to work with digital image libraries, from the creation of 3D or in extension time-annotated 4D models, to the eventual dissemination of research findings in teaching/learning scenarios. We review pedagogical approaches to reach different learning objectives, as well as methods that allow for the inclusion of historic city models employing Augmented Reality in mobile learning environments.