Ekaterina Sirazitdinova

IMAGO: Image-guided navigation for visually impaired people

Blind and visually impaired persons face many challenges due to isolation, the most important is lacking education due to social immobility. Yet, since the development of the well-known white-red cane, only few advances have been made to increase the mobility of blind people. GPS systems are often used for pedestrian navigation but lack precision, foremost in rural areas where navigation is most often needed. The aim of the IMAGO project is therefore the development of an inexpensive and unobtrusive navigation method for blind and visually impaired persons. Navigation is performed using structure from motion and image-based localization techniques. Route models are created as 3D point clouds through several steps: (i) image acquisition along the routes; (ii) bulk-transfer of images to a server; (iii) feature extraction; (iv) feature matching between images; (v) creation of 3D point cloud with structure from motion. Similarly, the navigation chains seven steps: (a) image acquisition while walking a route; (b) immediate transfer of image to a server; (c) feature extraction; (d) feature matching between image and 3D route model; (e) localization/camera matrix calculation; (f) navigation/calculation of direction based on localization; (g) transfer of direction to user. Current smartphones are used as devices both for recording of routes as well as navigation. Thereby, a high level of dissemination without additional costs is possible, both, within blind people for navigation, as well as seeing people for route creation. Additionally, haptic feedback can be used via a smart cane to reduce auditive feedback. The proposed system yields a high positioning accuracy of 80% of samples being located within 1.6 m. Thus, the system is usable for pedestrian navigation, especially for visually impaired persons.

Curve-to-Image Based Non-Rigid Registration of Digital Photos and Quantitative Light-Induced Fluorescence Images in Dentistry

Decalcification is an undesirable effect that can arise during orthodontic treatment. In digital photographs, it appears as white spot lesions, i.e. white spots on the tooth surface. To asses the extent of demineralization in a tooth, quantitative light-induced fluorescence (QLF) is used. We propose a method to match digital photographs and QLF images of decalcified teeth, based on the idea of curve-to-image matching. It extracts a curve representing the shape of the tooth from the QLF image and aligns it to the photo. The registration problem is formulated as minimization problem where the objective functional consists of a data term and a higher order, linear elastic prior for the deformation. The data term is constructed using the signed distance function of the tooth region shown in the photo, which is determined in a pre-processing step by classifying the photo into tooth and non-tooth regions. The resulting minimization problem is reformulated as a nonlinear least-square problem and solved numerically using Gauss-Newton. The evaluation is based on 150 image pairs captured from 32 patients. The correctness of the matching is confirmed by visual inspection of dental experts and the alignment improvement quantified using mutual information. The curve-to-image matching idea can be extended to surface-to-voxel tasks.

System design for 3D wound imaging using low-cost mobile devices

The state-of-the art method of wound assessment is a manual, imprecise and time-consuming procedure. Per- formed by clinicians, it has limited reproducibility and accuracy, large time consumption and high costs. Novel technologies such as laser scanning microscopy, multi-photon microscopy, optical coherence tomography and hyper-spectral imaging, as well as devices relying on the structured light sensors, make accurate wound assessment possible. However, such methods have limitations due to high costs and may lack portability and availability. In this paper, we present a low-cost wound assessment system and architecture for fast and accurate cutaneous wound assessment using inexpensive consumer smartphone devices. Computer vision techniques are applied either on the device or the server to reconstruct wounds in 3D as dense models, which are generated from images taken with a built-in single camera of a smartphone device. The system architecture includes imaging (smartphone), processing (smartphone or PACS) and storage (PACS) devices. It supports tracking over time by alignment of 3D models, color correction using a reference color card placed into the scene and automatic segmentation of wound regions. Using our system, we are able to detect and document quantitative characteristics of chronic wounds, including size, depth, volume, rate of healing, as well as qualitative characteristics as color, presence of necrosis and type of involved tissue.

Colorimetric calibration of wound photography with off-the-shelf devices

Digital cameras are often used in recent days for photographic documentation in medical sciences. However, color reproducibility of same objects suffers from different illuminations and lighting conditions. This variation in color representation is problematic when the images are used for segmentation and measurements based on color thresholds. In this paper, motivated by photographic follow-up of chronic wounds, we assess the impact of (i) gamma correction, (ii) white balancing, (iii) background unification, and (iv) reference card-based color correction. Automatic gamma correction and white balancing are applied to support the calibration procedure, where gamma correction is a nonlinear color transform. For unevenly illuminated images, non- uniform illumination correction is applied. In the last step, we apply colorimetric calibration using a reference color card of 24 patches with known colors. A lattice detection algorithm is used for locating the card. The least squares algorithm is applied for affine color calibration in the RGB model. We have tested the algorithm on images with seven different types of illumination: with and without flash using three different off-the-shelf cameras including smartphones. We analyzed the spread of resulting color value of selected color patch before and after applying the calibration. Additionally, we checked the individual contribution of different steps of the whole calibration process. Using all steps, we were able to achieve a maximum of 81% reduction in standard deviation of color patch values in resulting images comparing to the original images. That supports manual as well as automatic quantitative wound assessments with off-the-shelf devices.

Stereo vision for fully automatic volumetric flow measurement in urban drainage structures

Overflows in urban drainage structures, or sewers, must be prevented on time to avoid their undesirable consequences. An effective monitoring system able to measure volumetric flow in sewers is needed. Existing stateof-the-art technologies are not robust against harsh sewer conditions and, therefore, cause high maintenance expenses. Having the goal of fully automatic, robust and non-contact volumetric flow measurement in sewers, we came up with an original and innovative idea of a vision-based system for volumetric flow monitoring. On the contrast to existing video-based monitoring systems, we introduce a second camera to the setup and exploit stereo-vision aiming of automatic calibration to the real world. Depth of the flow is estimated as the difference between distances from the camera to the water surface and from the camera to the canal’s bottom. Camerato-water distance is recovered automatically using large-scale stereo matching, while the distance to the canal’s bottom is measured once upon installation. Surface velocity is calculated using cross-correlation template matching. Individual natural particles in the flow are detected and tracked throughout the sequence of images recorded over a fixed time interval. Having the water level and the surface velocity estimated and knowing the geometry of the canal we calculate the discharge. The preliminary evaluation has shown that the average error of depth computation was 3 cm, while the average error of surface velocity resulted in 5 cm/s. Due to the experimental design, these errors are rough estimates: at each acquisition session the reference depth value was measured only once, although the variation in volumetric flow and the gradual transitions between the automatically detected values indicated that the actual depth level has varied. We will address this issue in the next experimental session.

Abstract: Wound Imaging in 3D Using Low-Cost Mobile Devices

The state-of-the-art method of wound assessment is manually performed by clinicians. Such procedure has limited reproducibility and accuracy, large time consumption and high costs. Novel technologies such as laser scanning microscopy, multi-photon microscopy, optical coherence tomography and hyperspectral imaging [1], as well as devices relying on the structured light sensors [2, 3] have limitations due to high costs and may lack portability and availability. The high prevalence of chronic wounds, however, requires inexpensive and portable devices for 3D imaging of skin lesions.

Introducing Low-Cost Stereo Imaging for Cutaneous Wound Assessment

Nowadays the measurements of chronic wounds are mostly performed manually by clinicians, despite on high limitations of such method. Modern technologies make it possible to do accurate wound assessment. However, due to high costs and limited availability of special equipment, such methods are not widely used. Here, we present a concept of a low-cost wound assessment system, which, with a help of computer vision techniques, will create dense 3D reconstructions of wounds from images taken with a hand-held camera. Our system will also provide wound segmentation and will allow wound tracking over time. Additional color correction will be applied to support qualitative analysis of wounds.

Non-rigid contour-to-pixel registration of photographic and quantitative light-induced fluorescence imaging of decalcified teeth

Quantitative light-induced fluorescence (QLF) is widely used to assess the damage of a tooth due to decalcification. In digital photographs, decalcification appears as white spot lesions, i.e. white spots on the tooth surface. We propose a novel multimodal registration approach for the matching of digital photographs and QLF images of decalcified teeth. The registration is based on the idea of contour-to-pixel matching. Here, the curve, which represents the shape of the tooth, is extracted from the QLF image using a contour segmentation by binarization and morphological processing. This curve is aligned to the photo with a non-rigid variational registration approach. Thus, the registration problem is formulated as minimization problem with an objective function that consists of a data term and a regularizer for the deformation. To construct the data term, the photo is pointwise classified into tooth and non-tooth regions. Then, the signed distance function of the tooth region allows to measure the mismatch between curve and photo. As regularizer a higher order, linear elastic prior is used. The resulting minimization problem is solved numerically using bilinear Finite Elements for the spatial discretization and the Gauss-Newton algorithm. The evaluation is based on 150 image pairs, where an average of 5 teeth have been captured from 32 subjects. All registrations have been confirmed correctly by a dental expert. The contour-to-pixel methods can directly be used in 3D for surface-to-voxel tasks.

Outliers in 3D Point Clouds Applied to Efficient Image-Guided Localization

In this work, the tasks of improving positioning efficiency and minimization of space requirements in image-based navigation are explored. We proved the assumption that it is possible to reduce imagematching time and to increase storage capacities by removing outliers from 3D models used for localization, by applying three outlier removal methods to our datasets and observing the localization associated with the resulting models.