Pawel Strumillo

Head related transfer functions measurement and processing for the purpose of creating a spatial sound environment

The use of Head Related Transfer Functions (HRTFs) in audio processing is a popular method of obtaining spatialized sound. HRTFs describe disturbances caused in the sound wave by the human body, especially by head and the ear pinnae. Since these shapes are unique, HRTFs differ greatly from person to person. For this reason measurement of personalized HRTFs is justified. Measured HRTFs also need further processing to be utilized in a system producing spatialized sound. This paper describes a system designed for efficient collecting of Head Related Transfer Functions as well as the measurement, interpolation and verification procedures.

A Mobile Phone Application for Recognizing Objects as a Personal Aid for the Visually Impaired Users

In this work we report applications developed for Android smartphones that were designed for the blind and visually impaired users. The main application is capable of matching photographed objects to a database of objects, e.g. food or medicine containers. The two other applications are aimed at detecting major colours and direction location of the maximum brightness regions in the captured scenes. The conducted tests that were run on the modern smartphones indicate that the database objects can be recognized with an accuracy reaching 89% with an average recognition time of approx. 30 sec. We conclude the paper with a short summary of the tests of the software dedicated to blind and partially sighted users.

Verification of visual odometry algorithms with an OpenGL-based software tool

Abstract. We present a software tool called a stereovision egomotion sequence generator that was developed for testing visual odometry (VO) algorithms. Various approaches to single and multicamera VO algorithms are reviewed first, and then a reference VO algorithm that has served to demonstrate the program’s features is described. The program offers simple tools for defining virtual static three-dimensional scenes and arbitrary six degrees of freedom motion paths within such scenes and output sequences of stereovision images, disparity ground-truth maps, and segmented scene images. A simple script language is proposed that simplifies tests of VO algorithms for user-defined scenarios. The program’s capabilities are demonstrated by testing a reference VO technique that employs stereoscopy and feature tracking.

Different Approaches to Aiding Blind Persons in Mobility and Navigation in the “Naviton” and “Sound of Vision” Projects

In this chapter, we summarize several years of research efforts aimed at building ICT (Information and Communications Technologies) based systems for aiding the blind in travel and navigation at the Lodz University of Technology, mainly as part of the “Naviton” project (http://www.naviton.pl), from a Polish Ministry of Higher Education grant. We report on different approaches we undertook in these challenging goals which comprise the following prototype solutions: (1) sonified stereovision system for obstacle avoidance and environment imaging, (2) radio beacons for local navigation, (3) remote assistance system, (4) mobile navigation applications, (5) real-time tracking of public transport vehicles, (6) haptic imaging . We shortly describe these technologies and discuss user feedback from the trials of these technological aids. Finally, we point out the key objectives and first results of a Horizon 2020 project entitled “Sound of Vision: natural sense of vision through acoustics and haptics” (http://www.soundofvision.net) that started in 2015.

Sound of Vision – 3D Scene Reconstruction from Stereo Vision in an Electronic Travel Aid for the Visually Impaired

The paper presents the preliminary results for the parametrization of 3D scene for sonification purposes in an electronic travel aid (ETA) system being built within the European Union’s H2020 Sound of Vision project. The ETA is based on the concept of sensory substitution, in which visual information is transformed into either acoustic or haptic stimuli. In this communication we concentrate on vision-to-audio conversion i.e. employing stereovision for reconstruction of 3D scenes and building a spatial model of the environment for sonification. Two prerequisite approaches for the sonification are proposed. One involves the direct sonification of the so-called “U-disparity” representation of the depth map of the environment, while the other relies on the processing of the depth map to extract obstacles present in the environment and presenting them to the user as auditory icons reflecting specific size and location of the sonified object.

Field trials of a teleassistance system for the visually impaired

The paper reports the proof-of-concept tests, i.e. ergonomic, non-mobility tests with participation of blind volunteers and emphatic field trials (i.e. with the participation of sighted individuals) of a teleassistance system for the visually impaired. The system consists of two terminals connected by the GSM and Internet networks. The mobile terminal, housing a digital camera, GPS receiver and headset, is worn by a guided traveller and a stationary terminal is operated by a remote guide. A video stream from the travellers camera is transmitted to the remote guide who provides assistance by short spoken navigation instructions. The trials were conducted on a university campus with the participation of seven sighted individuals. They walked along three different paths, each approximately 400 m in length, under the teleassisted supervision of the remote guide. During these trials, the sighted individuals who were being remotely guided, noted the number of potential lost ways or possible collisions. Comments from the target users, i.e. the visually impaired who assessed the ergonomic and communication quality in the static, non-mobility trials of the system, were collected. On the basis of the conducted tests and the field trials, we conclude that teleassisted navigation is a prospective solution for aiding the visually impaired in space orientation and mobility.

Pedestrian tracking in video sequences: A particle filtering approach

In this work we study the methods for pedestrian tracking in video sequences and indicate various applications of these methods ranging from surveillance systems to aiding the visually impaired persons. First, we define the general problem of object tracking that comprises the tasks of object detection, identifying the flow of object location in consecutive video images and finally analysis of the tracked trajectory data. We review the well known object tracking techniques i.e. the Mean-Shift and the CAMSHIFT algorithm and discuss their properties. Then we introduce the computational technique known as particle filtering (PF) and explain how we have applied it to the tasks of pedestrian tracking. We compare the PF approach against the Mean-Shift and the CAMSHIFT algorithms in terms of tracking robustness and the required computational demand. We conclude, that on the tested video sequences, the PF tracker outperforms the Mean-Shift and by a small margin the CAMSHIFT algorithm. The PF tracker requires more computational power, however, its tracking performance can be flexibly adjusted to the application requirements.

Synthesizing a 3D auditory scene for use in an electronic travel aid for the blind

A system for auditory presentation of 3D scenes to the blind is presented, with the focus of the paper on the synthesis of sound codes suitable to carry important scene information. First, a short review of existing electronic travel aids for the blind (ETAs) is provided. Second, the project of the wearable ETA device, currently under development at the Technical University of Lodz, is outlined, along with the system modules: 3D scene reconstruction, object (obstacle) selection, synthesis of the sound code and the application of head related transfer functions (HRTFs) for generating spatialized sound. The importance of psychoacoustics, especially Bregmans theory of sound streams, is analyzed and proposed methods of sound code synthesis are presented, along with the software used for their verification.

Adaptive edge-based stereo block matching algorithm for a mobile Graphics Processing Unit

Nowadays most of the applications of the stereo vision are related to the mobile devices, especially in the field of robotics. It is crucial to develop efficient and robust algorithms that would allow real time operation in a wide range of environments. This paper presents an efficient adaptive algorithm of stereo matching that was applied and optimized for the mobile Graphics Processing Unit. It is a well known problem that most of the stereo vision algorithms are based on the dense stereo matching methods that in most of the cases are the main factor for a demanding computation cost. The presented method introduces a novel approach to the stereo matching problem by adaptively combining the cost function that can be computed efficiently for small matching window and sparse accumulated windows similar to those applied in convolutional neural networks. Adaptability of this method is based on detecting edges in the processed images. Such a solution allows obtaining precise subpixel results on highly textured regions of the image as well as to obtain stable results on weakly matchable texture-less regions while sustaining high efficiency and not causing problems related to the memory bandwidth bottleneck.

Interactive sonification of images on mobile devices for the visually impaired

A human-computer mobile interface for visually impaired persons is presented in the article. The authors propose several methods for image analysis and sound synthesis basing on interaction with a mobile device using tactile gestures. The developed algorithms allow to sonify colour and greyscale images, but mainly focus on the presentation of abstract black and white shapes for use in education of blind children. The interaction of the sonification process is achieved through various tactile gestures depending on the sonification mode, ranging from simple point-touches to two-handed multi-touch control. Implementation and pilot test results of the proposed human-computer interface using an Android tablet is included in the article.