Julius Kammerl

Real-time compression of point cloud streams

We present a novel lossy compression approach for point cloud streams which exploits spatial and temporal redundancy within the point data. Our proposed compression framework can handle general point cloud streams of arbitrary and varying size, point order and point density. Furthermore, it allows for controlling coding complexity and coding precision. To compress the point clouds, we perform a spatial decomposition based on octree data structures. Additionally, we present a technique for comparing the octree data structures of consecutive point clouds. By encoding their structural differences, we can successively extend the point clouds at the decoder. In this way, we are able to detect and remove temporal redundancy from the point cloud data stream. Our experimental results show a strong compression performance of a ratio of 14 at 1 mm coordinate precision and up to 40 at a coordinate precision of 9 mm.

Haptic Data Compression and Communication

The past decade has witnessed how audio-visual communication has shaped the way humans interact with or through technical systems. In contemporary times, the potential of haptic communication has been recognized as being compelling to further augment human-to-human and human-to-machine interaction. In the context of immersive communication, video and audio compression are considered key enabling technologies for high-quality interaction. In contrast, the compression of haptic data is a field of research that is still relatively young and not fully explored. This disregards the fact that we as humans rely heavily on the haptic modality to interact with our environment. True immersion into a distant environment and efficient collaboration between multiple participants both require the ability to physically interact with objects in the remote environment. With recent advances in virtual reality, man-machine interaction, telerobotics, telepresence, and teleaction, haptic communication is proving instrumental in enabling many novel applications. The goal of this overview article is to summarize the state of the art and the challenges of haptic data compression and communication for telepresence and teleaction.

Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion

Recent advances in man-machine interaction, telerobotics, telepresence and teleaction have shown that introducing the haptic modality to multimedia applications has the power to significantly widen their application range and to dramatically improve the user experience. Especially, the human body surface has been considered as an additional means of presenting information using vibrotactile display devices. In this context, spatial displacement of a vibrotactile stimulus can be deployed for information display. By exploiting a psychophysical illusion called “funneling illusion”, we are able to increase the spatial resolution of vibrotactile displays. In this paper, we aim at investigating the spatial resolution of vibrotactile perception on the human forearm when applying multiple “funneling” stimuli. In our psychophysical experiments, we revealed the human spatial perception ability on the human forearm for stationary and moving vibrotactile stimuli.

Investigating the influence of temporal intensity changes on apparent movement phenomenon

This paper aims at investigating the influence of the temporal intensity changes of two low-resolution vibrotactile actuators on the apparent movement phenomenon. In our work, we exploit two human sensory illusions called funneling illusion and apparent movement phenomenon. By temporarily varying the intensities of two adjacent vibrating actuators located on the dorsal of the human forearm, we obtained the illusion of a continuous movement of one tactile stimulus. In this work, we investigated the quality of the apparent movement according to the intensity change of the vibrating motors in a linear and logarithmic pattern. Psychophysical experiments revealed an interesting relationship between the distance and orientation of the two vibrating actuators with the preferred intensity variation, which are presented in this paper.

Perceptual coding of haptic data in time-delayed teleoperation

In telepresence and teleaction systems the haptic communication channel plays a central role. As it closes a global control loop any introduced communication delay possibly destabilizes the system and impairs the performance. The scattering theory is known to solve these stability issues by transmitting wave variables instead of haptic signals, i.e. force and velocity, over the communication channel. For stability and performance additionally high packet rates are required stressing the underlying network resources. Perceptual coding techniques of haptic signals, such as the Weber-inspired deadband approach, are known to successfully reduce the packet rate on the haptic channel. However, as wave variables do not directly represent haptic information but a linear transformation of both signals, perceptual coding is not directly applicable anymore. In this paper, we present a novel control scheme as well as a modification of the deadband approach to take advantage of the stabilization ability of the wave variables while allowing perceptual coding on the communication channel. Simulation results and comparison with the wave variables architecture indicate improved data compression for same degree of transparency for purely stiff as well as free environments.

Combining Contact Models With Perceptual Data Reduction for Efficient Haptic Data Communication in Networked VEs

In cooperative networked virtual environments, haptic rendering enables joint haptic interaction with virtual objects and, thus, shared touch experiences among multiple users. In case of an underlying packet-based communication network (e.g., the Internet), minimizing the end-to-end delay, with the goal of preventing instability of the involved control loops, results in high packet rates. Previously proposed perceptual data reduction approaches address this challenge and satisfy the strict delay constraints. However, significantly increased packet rates still occur during contact events. We present a novel event-based coding scheme based on a distributed haptic-rendering framework that integrates model-based distributed haptic rendering with perceptual data reduction. We also present a comprehensive haptic contact model for signals with multiple degrees of freedom. Furthermore, we show that the integration of event-triggered force transient models from event-based haptics into our local contact model is instrumental for generating convincing haptic feedback. Psychophysical experiments reveal that the approach presented herein allows us to push the data reduction performance beyond what is normally achievable by perceptual data reduction schemes alone while significantly improving the quality of haptic contact feedback.

Deadband-based offline-coding of haptic media

In this work, a novel perceptual coding approach for offline compression of haptic media is presented. Our scheme exploits the properties of human haptic perception and hides coding artifacts introduced by lossy compression below the human perception thresholds. We combine the concept of Just Noticeable Differences with predictive coding in order to achieve high coding efficiency without impairing user perception. In our experiments, we apply the proposed lossy compression scheme to haptic data that has been recorded during a telemanipulation session in a virtual environment. Our user studies reveal that our approach leads to strong data reduction up to 100:1 while preserving a high-quality haptic experience during playback of the compressed haptic data streams.

Error-resilient perceptual coding for networked haptic interaction

The performance of haptic interaction across communication networks critically depends on the successful reconstruction of the bidirectionally transmitted haptic signals, and hence on the quality of the communication channel. We propose a novel error-resilient data reduction scheme for haptic communication which exploits known limits of human haptic perception. Particularly, we show that missing haptic information due to packet loss may strongly impair the users experience during haptic interaction. We present and compare methods that eliminate the disturbing artifacts resulting out of packet loss. Our approach keeps the estimated impact of packet losses below human perception thresholds. A tree of possible cases (packets received or not received) and their respective occurrence probabilities is maintained at the sender side, and the system predicts unacceptable error cases to decide whether extra packets should be sent. We introduce different criteria that can be employed to trigger additional packets. In our experiments, we evaluate both the objective data reduction performance and the subjective system transparency by performing extensive tests using packet loss probability and round trip time as parameters. The proposed scheme shows excellent performances in terms of data reduction while sustaining good subjective ratings for a wide range of packet loss values and round trip times.

Hybrid signal-based and geometry-based prediction for haptic data reduction

Haptic data reduction schemes address the high packet-rate requirements of networked haptics. Perception-driven predictive coding approaches enable strong packet rate reduction while keeping the introduced distortion below human haptic perception thresholds. The performance of predictive coding is strongly influenced by factors such as human behavior, system characteristics, geometric and impedance properties of the environment, etc. In this paper, we first describe a novel surface geometry-based prediction approach for haptic data reduction where local object surface features are approximated with the help of simple geometric models. Secondly, we present a hybrid framework that combines signal-based and geometry-based prediction. Psychophysical experiments are performed to validate this framework. The results of the proposed geometry-based prediction show an improvement in haptic data reduction of about 54% as compared to the signal-based prediction (linear predictor). Furthermore, the presented hybrid prediction technique allows for an additional gain of 15%.

A Theoretical Analysis of Data Reduction Using the Weber Quantizer

We present a theoretical analysis of a perceptual coding approach, the so called Weber quantizer. Extensive studies performed by experimental psychologists and physiologists have unveiled one major conclusion: human perception often follows Webers law. Ernst Weber was an experimental physiologist who in 1834 first discovered the following implication DeltaI = kl, where DeltaI is the so called difference threshold or the just noticeable difference (JND). It describes the smallest amount of change of an (arbitrary) stimulus I which can be detected just as often as it cannot be detected and defines the Weber bound at [(1 - k)I, (1 + k)I].