Konstantinos Makantasis

4D Reconstruction of Tangible Cultural Heritage Objects from Web-Retrieved Images

The number of digital images that are available online today has reached unprecedented levels. Recent statistics showed that by the end of 2013 there were over 250 billion photographs stored in just one of the major social media sites, with a daily average upload of 300 million photos. These photos, apart from documenting personal lives, often relate to experiences in well-known places of cultural interest, throughout several periods of time. Thus from the viewpoint of Cultural Heritage professionals, they constitute valuable and freely available digital cultural content. Advances in the fields of Photogrammetry and Computer Vision have led to significant breakthroughs such as the Structure from Motion algorithm which creates 3D models of objects using their 2D photographs. The existence of powerful and affordable computational machinery enables the reconstruction not only of single structures such as artefacts, but also of entire cities. This paper presents an overview of our methodology for producing co...

Deep Convolutional Neural Networks for efficient vision based tunnel inspection

The inspection, assessment, maintenance and safe operation of the existing civil infrastructure consists one of the major challenges facing engineers today. Such work requires either manual approaches, which are slow and yield subjective results, or automated approaches, which depend upon complex handcrafted features. Yet, for the latter case, it is rarely known in advance which features are important for the problem at hand. In this paper, we propose a fully automated tunnel assessment approach; using the raw input from a single monocular camera we hierarchically construct complex features, exploiting the advantages of deep learning architectures. Obtained features are used to train an appropriate defect detector. In particular, we exploit a Convolutional Neural Network to construct high-level features and as a detector we choose to use a Multi-Layer Perceptron due to its global function approximation properties. Such an approach achieves very fast predictions due to the feedforward nature of Convolutional Neural Networks and Multi-Layer Perceptrons.

In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction

One of the main characteristics of Internet era is the free and online availability of extremely large collections of images located on distributed and heterogeneous platforms over the web. The proliferation of millions of shared photographs spurred the emergence of new image retrieval techniques based not only on images’ visual information, but on geo-location tags and camera exif data. These huge visual collections provide a unique opportunity for cultural heritage documentation and 3D reconstruction. The main difficulty, however, is that the internet image datasets are unstructured containing many outliers. For this reason, in this paper a new content-based image filtering is proposed to discard image outliers that either confuse or significantly delay the followed e-documentation tools, such as 3D reconstruction of a cultural heritage object. The presented approach exploits and fuses two unsupervised clustering techniques: DBSCAN and spectral clustering. DBSCAN algorithm is used to remove outliers from the initially retrieved dataset and spectral clustering discriminate the noise free image dataset into different categories each representing characteristic geometric views of cultural heritage objects. To discard the image outliers, we consider images as points onto a multi-dimensional manifold and the multi-dimensional scaling algorithm is adopted to relate the space of the image distances with the space of Gram matrices through which we are able to compute the image coordinates. Finally, structure from motion is utilized for 3D reconstruction of cultural heritage landmarks. Evaluation on a dataset of about 31,000 cultural heritage images being retrieved from internet collections with many outliers indicate the robustness and cost effectiveness of the proposed method towards a reliable and just-in-time 3D reconstruction than existing state-of-the-art techniques.

Vision-based maritime surveillance system using fused visual attention maps and online adaptable tracker

This paper presents a vision-based system for maritime surveillance using moving PTZ cameras. This system is intended to be used as an early warning system by local authorities. It fuses a visual attention method that exploits low-level image features appropriately selected for maritime environment, with an online adaptable neural network tracker, without making any assumptions about environmental or visual conditions. Systems performance was evaluated with videos from cameras placed at Limassol port and Venetian port of Chania and concerns robustness compared to dynamically changing visual conditions and different kinds of vessels, all in real time.

Deep Learning-Based Man-Made Object Detection from Hyperspectral Data

Hyperspectral sensing, due to its intrinsic ability to capture the spectral responses of depicted materials, provides unique capabilities towards object detection and identification. In this paper, we tackle the problem of man-made object detection from hyperspectral data through a deep learning classification framework. By the effective exploitation of a Convolutional Neural Network we encode pixels’ spectral and spatial information and employ a Multi-Layer Perceptron to conduct the classification task. Experimental results and the performed quantitative validation on widely used hyperspectral datasets demonstrating the great potentials of the developed approach towards accurate and automated man-made object detection.

Monocular camera fall detection system exploiting 3d measures: a semi-supervised learning approach

Falls have been reported as the leading cause of injury-related visits to emergency departments and the primary etiology of accidental deaths in elderly. The system presented in this article addresses the fall detection problem through visual cues. The proposed methodology utilize a fast, real-time background subtraction algorithm based on motion information in the scene and capable to operate properly in dynamically changing visual conditions, in order to detect the foreground object and, at the same time, it exploits 3D spaces measures, through automatic camera calibration, to increase the robustness of fall detection algorithm which is based on semi-supervised learning. The above system uses a single monocular camera and is characterized by minimal computational cost and memory requirements that make it suitable for real-time large scale implementations.

Four-dimensional reconstruction of cultural heritage sites based on photogrammetry and clustering

Abstract. A system designed and developed for the three-dimensional (3-D) reconstruction of cultural heritage (CH) assets is presented. Two basic approaches are presented. The first one, resulting in an “approximate” 3-D model, uses images retrieved in online multimedia collections; it employs a clustering-based technique to perform content-based filtering and eliminate outliers that significantly reduce the performance of 3-D reconstruction frameworks. The second one is based on input image data acquired through terrestrial laser scanning, as well as close range and airborne photogrammetry; it follows a sophisticated multistep strategy, which leads to a “precise” 3-D model. Furthermore, the concept of change history maps is proposed to address the computational limitations involved in four-dimensional (4-D) modeling, i.e., capturing 3-D models of a CH landmark or site at different time instances. The system also comprises a presentation viewer, which manages the display of the multifaceted CH content collected and created. The described methods have been successfully applied and evaluated in challenging real-world scenarios, including the 4-D reconstruction of the historic Market Square of the German city of Calw in the context of the 4-D-CH-World EU project.

Content-Based Filtering for Fast 3D Reconstruction from Unstructured Web-Based Image Data

The huge amount of visual collections provides a unique opportunity for cultural heritage e-documentation and 3D reconstruction. The main difficulty, however, is its unstructured nature. In this paper a new content-based image filtering is proposed to discard image outliers that either confuse or significantly delay the 3D reconstruction process. The presented approach exploits a dense-based unsupervised paradigm applied on multi-dimensional manifolds where images are represented as image points. The multidimensional scaling algorithm is adopted to relate the space of the image distances with the space of Gram matrices to compute the image coordinates. Evaluation on a dataset of about 31,000 cultural heritage images being retrieved from internet collections with many outliers indicate the robustness and cost effectiveness of the proposed method towards an affordable 3D reconstruction.

Crack Identification Via User Feedback, Convolutional Neural Networks and Laser Scanners for Tunnel Infrastructures

In this paper, a deep learning approach synergetically to a laser scanning process are employed for the visual detection and accurate description of concrete defects in tunnels. Analysis is performed over raw RGB images; Convolutional Neural Network serves as the crack detector, during the inspection. In case of a positive detection, the tunnel’s cross-section morphology is assessed via 3D point clouds, created by a laser scanner, allowing the identification of deformations in the compartment. The proposed approach, in contrast to the existing ones, emphasizes on applicability (easy initialization, no preprocessing of the input data) and provides a holistic assessment of the structure; reconstructed 3D model allows the fast identification of structural divergence from the original design, alerting the engineers for possible dangers.

Semi-supervised vision-based maritime surveillance system using fused visual attention maps

This paper presents a vision-based system for maritime surveillance, using moving PTZ cameras. The proposed methodology fuses a visual attention method that exploits low-level image features appropriately selected for maritime environment, with appropriate tracker, without making any assumptions about environmental or visual conditions. The offline initialization is based on large graph semi-supervised technique. System’s performance was evaluated with videos from cameras placed at Limassol port and Venetian port of Chania. Results suggest high detection ability, despite dynamically changing visual conditions and different kinds of vessels, all in real time.