Marina Paolanti

Automatic Analysis of Eye-Tracking Data for Augmented Reality Applications: A Prospective Outlook

Eye-tracking technology is becoming easier and cheaper to use, resulting in its increasing application to numerous fields of research. Recent years have seen rapid developments in this area. In light of the foregoing, in the context of Cultural Heritage (CH), the definition of a modern approach to understand how individuals perceive art is challenging. Despite the art perception is highly subjective and variable according to knowledge and experience, more recently, several scientific study and enterprises started to quantify how subjects observe art by the application of the eye-tracking technology. The aim of this study was to understand the visual behaviour of subjects looking at paintings, using eye-tracking technology, in order to define a protocol for optimizing an existing Augmented Reality (AR) application that allows the visualization of digital contents through a display. The stimuli used are three famous paintings preserved at the National Gallery of Marche (Urbino, Marche Region, Italy). We applied eye-tracking to have a deeper understanding of people visual activities in front of these paintings and to analyse how digital contents eventually influence their behaviour. The description of the applied procedure and the preliminary results are presented.

Person Re-identification Dataset with RGB-D Camera in a Top-View Configuration

Video analytics, involves a variety of techniques to monitor, analyse, and extract meaningful information from video streams. In this light, person re-identification is an important topic in scene monitoring, human computer interaction, retail, people counting, ambient assisted living and many other computer vision research. The existing datasets are not suitable for activity monitoring and human behaviour analysis. For this reason we build a novel dataset for person re-identification that uses an RGB-D camera in a top-view configuration. This setup choice is primarily due to the reduction of occlusions and it has also the advantage of being privacy preserving, because faces are not recorded by the camera. The use of an RGB-D camera allows to extract anthropometric features for the recognition of people passing under the camera. The paper describes in details the collection and construction modalities of the dataset TVPR. This is composed by 100 people and for each video frame nine depth and colour features are computed and provided together with key descriptive statistics.

Modelling and Forecasting Customer Navigation in Intelligent Retail Environments

Understanding shopper behaviour is one of the keys to success for retailers. In particular, it is necessary that managers know which retail attributes are important to which shoppers and their main goal is to improve the consumer shopping experience. In this work, we present sCREEN (Consumer REtail ExperieNce), an intelligent mechatronic system for indoor navigation assistance in retail environments that minimizes the need for active tagging and does not require metrics maps. The tracking system is based on Ultra-wideband technology. The digital devices are installed in the shopping carts and baskets and sCREEN allows modelling and forecasting customer navigation in retail environments. This paper contributes the design of an intelligent mechatronic system with the use of a novel Hidden Markov Models (HMMs) for the representation of shoppers’ shelf/category attraction and usual retail scenarios such as product out of stock or changes on store layout. Observations are viewed as a perceived intelligent system performance. By forecasting consumers next shelf/category attraction, the system can present the item location information to the consumer, including a walking route map to a location of the product in the retail store, and/or the number of an aisle in which the product is located. Effective and efficient design processes for mechatronic systems are a prerequisite for competitiveness in an intelligent retail environment. Experiments are performed in a real retail environment that is a German supermarket, during business hours. A dataset, with consumers trajectories, timestamps and the corresponding ground truth for training as well as evaluating the HMM, have been built and made publicly available. The results in terms of Precision, Recall and F1-score demonstrate the effectiveness and suitability of our approach, with a precision value that exceeds the 76% in all test cases.

Visual and Textual Sentiment Analysis of Brand-Related Social Media Pictures Using Deep Convolutional Neural Networks.

Social media pictures represent a rich source of knowledge for companies to understand consumers’ opinions, as they are available in real time and at low costs and represent an active feedback which is of importance not only for companies developing products, but also to their rivals and potential consumers. In order to estimate the overall sentiment of a picture, it is essential to not only judge the sentiment of the visual elements but also to understand the meaning of the included text. This paper introduces an approach to estimate the overall sentiment of brand-related pictures from social media based on both visual and textual clues. In contrast to existing papers, we do not consider text accompanying a picture, but text embedded in a picture, which is more challenging since the text has to be detected and recognized first, before its sentiment can be identified. Based on visual and textual features extracted from two trained Deep Convolutional Neural Networks (DCNNs), the sentiment of a picture is identified by a machine learning classifier. The approach was applied and tested on a newly collected dataset, “GfK Verein Dataset” and several machine learning algorithms are compared. The experiments yield high accuracy, demonstrating the effectiveness and suitability of the proposed approach.

Mobile robot for retail surveying and inventory using visual and textual analysis of monocular pictures based on deep learning

This paper describes a novel system for automating data collection and surveying in a retail store using mobile robots. The manpower cost for surveying and monitoring the shelves in retail stores are high, because of which these activities are not repeated frequently causing reduced customer satisfaction and loss of revenue. Further, the accuracy of data collected may be improved by avoiding human related factors. We use a mobile robot platform with on-board cameras to monitor the shelves autonomously (based on indoor UWB Localization and planning). The robot is designed to facilitate automatic detection of Shelf Out of Stock (SOOS) situations. The paper contribution is an approach to estimate the overall stock assortment based of pictures from both visual and textual clues. Based on visual and textual features extracted from two trained Convolutional Neural Networks (CNNs), the type of the product is identified by a machine learning classifier. The approach was applied and tested on a newly collected dataset and several machine learning algorithms are compared. The experiments yield high accuracy, demonstrating the effectiveness and suitability of the proposed approach, also in comparison of existing state of the art SOOS solutions.

Optimal production planning by reusing components

Warehouse management aims to optimally organize stock levels and orders according to production planning and market needs. In the area of fashion goods, the variability of the demand tends to re-plan the productions in agreement with the seasonal trends and merchandise news. These changes can lead to considerable levels of raw materials that remain unused and represent a loss of profit for the company. In some cases the whole amount of materials can be spread in many warehouses located in different locations, either in a single country or worldwide. A mathematical model for reusing old stocks (obsolescent material) is presented in this paper. The model is aimed at the optimization of the warehouse management of a business company. Considering the obsolescences and the bill of materials of each product, a mathematical Integer Linear Programming (ILP) model is used to plan the production of finished goods to maximize the revenue at the net of the costs of missing components. The mix production model is refined with side constraints that limit the budget for new components, the trade-off between the number of reused and ordered parts, the whole production and the minimum and maximum quantity for each produced model and ordered part type. Real instances are solved with different values of parameters and best solutions are presented.

Accurate modeling of the microwave treatment in reverberating chamber. sanitation of agro food material

The microwave heating is useful for drying of foodstuff, disinfestation of works of art, phitosanitary treatment and disinfection of packaging according to current international guidelines. The computer simulation allows predicting and monitoring the heating process. The microwave treatment can nevertheless present some problems such as the presence of highly heated areas (hot spots) or areas with poor radiation due to particular shapes. Simulation of complex systems has evolved into a research discovery tool: such models and simulations, drawing upon the dramatic scale up of computational power and associated architectures and algorithmic innovation, can address complex systems with many degrees of freedom and with multiple length and time scales of interest. Using specific programs, the distribution of heating power in objects to be treated, even if complex shapes, can be predicted so as to be able to define the possibility, the time necessary to the processing, the power to be transmitted in the chamber and any repair or protection to cover the most sensitive areas. It can also predict the behavior of irradiation in the presence of other entities such as nails or pests. In order to perform simulation, important data are the geometry of the object or objects in the case of multiple loading and their dielectric characteristics. As a result we obtain the distribution of heating power.

Automatic Classification for Anti Mixup Events in Advanced Manufacturing System

The mix-up is a phenomenon in which a tablet/capsule gets into a different package. It is an annoying problem because mixing different products in the same package could result dangerous for consumers that take the incorrect product or receive an unintended ingredient. So, the consequences could be very dangerous: overdose, interaction with other medications a consumer may be taking, or an allergic reaction. The manufacturers are not able to guarantee the contents of the packages and so for this reason they are very exposed to the risk in which users rightly want to obtain compensation for possible damages caused by the mix-up. The aim of this work is the identification of mix-up events, through machine learning approach based on data, coming from different embedded systems installed in the manufacturing facilities and from the information system, in order to implement integrated policies for data analysis and sensor fusion that leads to waste and detection of pieces that do not comply. In this field, two types of approaches from the point of view of embedded sensors (optical and NIR vision and interferometry) will be analyzed focusing in particular on data processing and their classification on advanced manufacturing scenarios. Results are presented considering a simulated scenario that uses pre-recorded real data to test, in a preliminary stage, the effectiveness and the novelty of the proposed approach.Copyright

Person Re-Identification with RGB-D Camera in Top-View Configuration through Multiple Nearest Neighbor Classifiers and Neighborhood Component Features Selection

Person re-identification is an important topic in retail, scene monitoring, human-computer interaction, people counting, ambient assisted living and many other application fields. A dataset for person re-identification TVPR (Top View Person Re-Identification) based on a number of significant features derived from both depth and color images has been previously built. This dataset uses an RGB-D camera in a top-view configuration to extract anthropometric features for the recognition of people in view of the camera, reducing the problem of occlusions while being privacy preserving. In this paper, we introduce a machine learning method for person re-identification using the TVPR dataset. In particular, we propose the combination of multiple k-nearest neighbor classifiers based on different distance functions and feature subsets derived from depth and color images. Moreover, the neighborhood component feature selection is used to learn the depth features’ weighting vector by minimizing the leave-one-out regularized training error. The classification process is performed by selecting the first passage under the camera for training and using the others as the testing set. Experimental results show that the proposed methodology outperforms standard supervised classifiers widely used for the re-identification task. This improvement encourages the application of this approach in the retail context in order to improve retail analytics, customer service and shopping space management.

User-Centered Predictive Model for Improving Cultural Heritage Augmented Reality Applications: An HMM-Based Approach for Eye-Tracking Data

Today, museum visits are perceived as an opportunity for individuals to explore and make up their own minds. The increasing technical capabilities of Augmented Reality (AR) technology have raised audience expectations, advancing the use of mobile AR in cultural heritage (CH) settings. Hence, there is the need to define a criteria, based on users’ preference, able to drive developers and insiders toward a more conscious development of AR-based applications. Starting from previous research (performed to define a protocol for understanding the visual behaviour of subjects looking at paintings), this paper introduces a truly predictive model of the museum visitor’s visual behaviour, measured by an eye tracker. A Hidden Markov Model (HMM) approach is presented, able to predict users’ attention in front of a painting. Furthermore, this research compares users’ behaviour between adults and children, expanding the results to different kind of users, thus providing a reliable approach to eye trajectories. Tests have been conducted defining areas of interest (AOI) and observing the most visited ones, attempting the prediction of subsequent transitions between AOIs. The results demonstrate the effectiveness and suitability of our approach, with performance evaluation values that exceed 90%.