Isaak Kavasidis

An innovative web-based collaborative platform for video annotation

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms’ evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system’s usability.

Transcranial Magnetic Stimulation in the Assessment of Motor Cortex Excitability and Treatment of Drug-Resistant Major Depression

Major depression is one of the leading causes of disabling condition worldwide and its treatment is often challenging and unsatisfactory, since many patients become refractory to pharmacological therapies. Transcranial magnetic stimulation (TMS) is a noninvasive neurophysiological investigation mainly used to study the integrity of the primary motor cortex excitability and of the cortico-spinal tract. The development of paired-pulse and repetitive TMS (rTMS) paradigms has allowed investigators to explore the pathophysiology of depressive disorders and other neuropsychiatric diseases linked to brain excitability dysfunctions. Repetitive transcranial magnetic stimulation has also therapeutic and rehabilitative capabilities since it is able to induce changes in the excitability of inhibitory and excitatory neuronal networks that may persist in time. However, the therapeutic effects of rTMS on major depression have been demonstrated by analyzing only the improvement of neuropsychological performance. The aim of this study was to investigate cortical excitability changes on 12 chronically-medicated depressed patients (test group) after rTMS treatment and to correlate neurophysiological findings to neuropsychological outcomes. In detail, we assessed different parameters of cortical excitability before and after active rTMS in the test group, then compared to those of 10 age-matched depressed patients (control group) who underwent sham rTMS. In line with previous studies, at baseline both groups exhibited a significant interhemispheric difference of motor cortex excitability. This neurophysiological imbalance was then reduced in the patients treated with active rTMS, resulting also in a clinical benefit as demonstrated by the improvement in neuropsychological test scores. On the contrary, after sham rTMS, the interhemispheric difference was still evident in the control group. The reported clinical benefits in the test group might be related to the plastic remodeling of synaptic connection induced by rTMS treatment.

A semi-automatic tool for detection and tracking ground truth generation in videos

In this paper we present a tool for the generation of ground-truth data for object detection, tracking and recognition applications. Compared to state of the art methods, such as ViPER-GT, our tool improves the user experience by providing edit shortcuts such as hotkeys and drag-and-drop, and by integrating computer vision algorithms to automate, under the supervision of the user, the extraction of contours and the identification of objects across frames. A comparison between our application and ViPER-GT tool was performed, which showed how our tool allows users to label a video in a shorter time, while at the same time providing a higher ground truth quality.

A texton-based kernel density estimation approach for background modeling under extreme conditions

Abstract Background modeling is a well-know approach to detect moving objects in video sequences. In recent years, background modeling methods that adopt spatial and texture information have been developed for dealing with complex scenarios. However, none of the investigated approaches have been tested under extreme conditions, such as the underwater domain, on which effects compromising the video quality affect negatively the performance of the background modeling process. In order to overcome such difficulties, more significant features and more robust methods must be found. In this paper, we present a kernel density estimation method which models background and foreground by exploiting textons to describe textures within small and low contrasted regions. Comparison with other texture descriptors, namely, local binary pattern (LBP) and scale invariant local ternary pattern (SILTP) shown improved performance. Besides, quantitative and qualitative performance evaluation carried out on three standard datasets showing very complex conditions revealed that our method outperformed state-of-the-art methods that use different features and modeling techniques and, most importantly, it is able to generalize over different scenarios and targets.

Understanding fish behavior during typhoon events in real-life underwater environments

The study of fish populations in their own natural environment is a task that has usually been tackled in invasive ways which inevitably influenced the behavior of the fish under observation. Recent projects involving the installation of permanent underwater cameras (e.g. the Fish4Knowledge (F4K) project, for the observation of Taiwan’s coral reefs) allow to gather huge quantities of video data, without interfering with the observed environment, but at the same time require the development of automatic processing tools, since manual analysis would be impractical for such amounts of videos. Event detection is one of the most interesting aspects from the biologists’ point of view, since it allows the analysis of fish activity during particular events, such as typhoons. In order to achieve this goal, in this paper we present an automatic video analysis approach for fish behavior understanding during typhoon events. The first step of the proposed system, therefore, involves the detection of “typhoon” events and it is based on video texture analysis and on classification by means of Support Vector Machines (SVM). As part of our behavior understanding efforts, trajectory extraction and clustering have been performed to study the differences in behavior when disruptive events happen. The integration of event detection with fish behavior understanding surpasses the idea of simply detecting events by low-level features analysis, as it supports the full semantic comprehension of interesting events.

An integrated computer-controlled system for assisting researchers in cortical excitability studies by using transcranial magnetic stimulation

Transcranial magnetic stimulation (TMS) is the most important technique currently available to study cortical excitability. Additionally, TMS can be used for therapeutic and rehabilitation purposes, replacing the more painful transcranial electric stimulation (TES). In this paper we present an innovative and easy-to-use tool that enables neuroscientists to design, carry out and analyze scientific studies based on TMS experiments for both diagnostic and research purposes, assisting them not only in the practicalities of administering the TMS but also in each step of the entire studys workflow. One important aspect of this tool is that it allows neuroscientists to specify research designs at will, enabling them to define any parameter of a TMS study starting from data acquisition and sample group definition to automated statistical data analysis and RDF data storage. It also supports the diagnosing process by using on-line support vector machines able to learn incrementally from the diseases instances that are continuously added into the system. The proposed system is a neuroscientist-centred tool where the protocols being followed in TMS studies are made explicit, leaving to the users flexibility in exploring and sharing the results, and providing assistance in managing the complexity of the final diagnosis. This type of tool can make the results of medical experiments more easily exploitable, thus accelerating scientific progress.

Deep Learning Human Mind for Automated Visual Classification

What if we could effectively read the mind and transfer human visual capabilities to computer vision methods? In this paper, we aim at addressing this question by developing the first visual object classifier driven by human brain signals. In particular, we employ EEG data evoked by visual object stimuli combined with Recurrent Neural Networks (RNN) to learn a discriminative brain activity manifold of visual categories in a reading the mind effort. Afterward, we transfer the learned capabilities to machines by training a Convolutional Neural Network (CNN)–based regressor to project images onto the learned manifold, thus allowing machines to employ human brain–based features for automated visual classification. We use a 128-channel EEG with active electrodes to record brain activity of several subjects while looking at images of 40 ImageNet object classes. The proposed RNN-based approach for discriminating object classes using brain signals reaches an average accuracy of about 83%, which greatly outperforms existing methods attempting to learn EEG visual object representations. As for automated object categorization, our human brain–driven approach obtains competitive performance, comparable to those achieved by powerful CNN models and it is also able to generalize over different visual datasets.

A crowdsourcing approach to support video annotation

In this paper we present an innovative approach to support efficient large scale video annotation by exploiting the crowdsourcing. In particular, we collect big noisy annotations by an on-line Flash game which aims at taking photos of objects appearing through the game levels. The data gathered (suitably processed) from the game is then used to drive image segmentation approaches, namely the Region Growing and Grab Cut, which allow us to derive meaningful annotations. A comparison against hand-labeled ground truth data showed that the proposed approach constitutes a valid alternative to the existing video annotation approaches and allow a reliable and fast collection of large scale ground truth data for performance evaluation in computer vision.

Content based recommender system by using eye gaze data

In this work, we present a proactive content based recommender system that employs web document clustering performed by using eye gaze data. Generally, recommender systems are used in commercial applications, where information about the users habits and interests are of crucial importance in order to plan marketing strategies, or in information retrieval systems in order to suggest similar resources a user is interested in. Commonly, these systems use explicit relevance feedback techniques (e.g. mouse or keyboard) to improve their performance and to recommend products. In contrast, the proposed system permits to capture users interest by using implicit relevance feedback, based on data acquired by an eye tracker Tobii T60. The purpose of the system is to collect eye gaze data during web navigation and, by employing clustering techniques, to suggest web documents similar to those that the user, implicitly, expressed greater interest. Performance evaluation was carried out on 30 users and the results show that the proposed system enhanced navigation experience in about 73% of the cases.

Quantitative performance analysis of object detection algorithms on underwater video footage

Object detection in underwater unconstrained environments is useful in domains like marine biology and geology, where the scientists need to study fish populations, underwater geological events etc. However, in literature, very little can be found regarding fish detection in unconstrained underwater videos. Nevertheless, the unconstrained underwater video domain constitutes a perfect soil for bringing state-of-the-art object detection algorithms to their limits because of the nature of the scenes, which often present with a number of intrinsic difficulties (e.g. multi-modal backgrounds, complex textures and color patterns, ever-changing illumination etc..). In this paper, we evaluated the performance of six state-of-the-art object detection algorithms in the task of fish detection in unconstrained, underwater video footage, discussing the properties of each of them and giving a detailed report of the achieved performance.