Judith Redi

Digital image forensics: a booklet for beginners

Digital visual media represent nowadays one of the principal means for communication. Lately, the reliability of digital visual information has been questioned, due to the ease in counterfeiting both its origin and content. Digital image forensics is a brand new research field which aims at validating the authenticity of images by recovering information about their history. Two main problems are addressed: the identification of the imaging device that captured the image, and the detection of traces of forgeries. Nowadays, thanks to the promising results attained by early studies and to the always growing number of applications, digital image forensics represents an appealing investigation domain for many researchers. This survey is designed for scholars and IT professionals approaching this field, reviewing existing tools and providing a view on the past, the present and the future of digital image forensics.

Circular-ELM for the reduced-reference assessment of perceived image quality

Providing a satisfactory visual experience is one of the main goals for present-day electronic multimedia devices. All the enabling technologies for storage, transmission, compression, rendering should preserve, and possibly enhance, the quality of the video signal; to do so, quality control mechanisms are required. These mechanisms rely on systems that can assess the visual quality of the incoming signal consistently with human perception. Computational Intelligence (CI) paradigms represent a suitable technology to tackle this challenging problem. The present research introduces an augmented version of the basic Extreme Learning Machine (ELM), the Circular-ELM (C-ELM), which proves effective in addressing the visual quality assessment problem. The C-ELM model derives from the original Circular BackPropagation (CBP) architecture, in which the input vector of a conventional MultiLayer Perceptron (MLP) is augmented by one additional dimension, the circular input; this paper shows that C-ELM can actually benefit from the enhancement provided by the circular input without losing any of the fruitful properties that characterize the basic ELM framework. In the proposed framework, C-ELM handles the actual mapping of visual signals into quality scores, successfully reproducing perceptual mechanisms. Its effectiveness is proved on recognized benchmarks and for four different types of distortions.

Color Distribution Information for the Reduced-Reference Assessment of Perceived Image Quality

Reduced-reference systems can predict in real-time the perceived quality of images for digital broadcasting, only requiring that a limited set of features, extracted from the original undistorted signals, is transmitted together with the image data. This paper uses descriptors based on the color correlogram, analyzing the alterations in the color distribution of an image as a consequence of the occurrence of distortions, for the reduced reference data. The processing architecture relies on a double layer at the receiver end. The first layer identifies the kind of distortion that may affect the received signal. The second layer deploys a dedicated prediction module for each type of distortion; every predictor yields an objective quality score, thus completing the estimation process. Computational-intelligence models are used extensively to support both layers with empirical training. The double-layer architecture implements a general purpose image quality assessment system, not being tied up to specific distortions and, at the same time, it allows us to benefit from the accuracy of specific, distortion-targeted metrics. Experimental results based on subjective quality data confirm the general validity of the approach.

Interactions of visual attention and quality perception

Several attempts to integrate visual saliency information in quality metrics are described in literature, albeit with contradictory results. The way saliency is integrated in quality metrics should reflect the mechanisms underlying the interaction between image quality assessment and visual attention. This interaction is actually two-fold: (1) image distortions can attract attention away from the Natural Scene Saliency (NSS), and (2) the quality assessment task in itself can affect the way people look at an image. A subjective study was performed to analyze the deviation in attention from NSS as a consequence of being asked to assess the quality of distorted images, and, in particular, whether, and if so how, this deviation depended on the distortion kind and/or amount. Saliency maps were derived from eye-tracking data obtained during scoring distorted images, and they were compared to the corresponding NSS, derived from eye-tracking data obtained during freely looking at high quality images. The study revealed some structural differences between the NSS maps and the ones obtained during quality assessment of the distorted images. These differences were related to the quality level of the images; the lower the quality, the higher the deviation from the NSS was. The main change was identified as a shrinking of the region of interest, being most evident at low quality. No evident role for the kind of distortion in the change in saliency was found. Especially at low quality, the quality assessment task seemed to prevail on the natural attention, forcing it to deviate in order to better evaluate the impact of artifacts.

Supporting visual quality assessment with machine learning

Objective metrics for visual quality assessment often base their reliability on the explicit modeling of the highly non-linear behavior of human perception; as a result, they may be complex and computationally expensive. Conversely, machine learning (ML) paradigms allow to tackle the quality assessment task from a different perspective, as the eventual goal is to mimic quality perception instead of designing an explicit model the human visual system. Several studies already proved the ability of ML-based approaches to address visual quality assessment; nevertheless, these paradigms are highly prone to overfitting, and their overall reliability may be questionable. In fact, a prerequisite for successfully using ML in modeling perceptual mechanisms is a profound understanding of the advantages and limitations that characterize learning machines. This paper illustrates and exemplifies the good practices to be followed.

Text Clustering for Digital Forensics Analysis

In the last decades digital forensics have become a prominent activity in modern investigations. Indeed, an important data source is often constituted by information contained in devices on which investigational activity is performed. Due to the complexity of this inquiring activity, the digital tools used for investigation constitute a central concern. In this paper a clustering-based text mining technique is introduced for investigational purposes. The proposed methodology is experimentally applied to the publicly available Enron dataset that well fits a plausible forensics analysis context.

Understanding the role of social context and user factors in video Quality of Experience

We examine the impact of social context on compressed video Quality of Experience.We analyze the effect on QoE of user factors such as interest and demographics.Co-viewing videos increases users enjoyment and endurability of the experience.Low bitrate does not decrease video enjoyment, yet lower video quality is perceived.User interest increases QoE; this effect is suppressed by the presence of co-viewers. Quality of Experience is a concept to reflect the level of satisfaction of a user with a multimedia content, service or system. So far, the objective (i.e., computational) approaches to measure QoE have been mostly based on the analysis of the media technical properties. However, recent studies have shown that this approach cannot sufficiently estimate user satisfaction, and that QoE depends on multiple factors, besides the media technical properties. This paper aims to identify the role of social context and user factors (such as interest and demographics) in determining quality of viewing experience. We also investigate the relationships between social context, user factors and some media technical properties, the effect of which on image quality is already known (i.e., bitrate level and video genre). Our results show that the presence of co-viewers increases the users level of enjoyment and enhances the endurability of the experience, and so does interest in the video content. Furthermore, although participants can clearly distinguish the various levels of video quality used in our study, these do not affect any of the other aspects of QoE. Finally, we report an impact of both gender and cultural background on QoE. Our results provide a first step toward building an accurate model of user QoE appreciation, to be deployed in future multimedia systems to optimize the user experience.

How to apply spatial saliency into objective metrics for JPEG compressed images

This paper investigates how saliency obtained from eye-tracking data can be integrated into objective metrics for JPEG compressed images. The objective metrics used in this paper are both based on features, locally extracted from the images and serving as input to a neural network for the overall quality prediction. We compare various weighting functions to combine saliency with these objective metrics, taking into account the possible distraction due to artifacts that might affect the quality judgment. Experimental results indicate that including saliency into objective metrics in an appropriate way can further enhance their performance.

How Passive Image Viewers Became Active Multimedia Users

Subjective assessment of quality of experience (QoE) is key to understanding user preferences with respect to multimedia fruition. As such, it is a necessary step to multimedia delivery optimization, since QoE needs to take into account technology limitations as well as user satisfaction. The study of QoE appreciation dates back to the twentieth century, when it exploded with the advent of CRT first and LCD displays later. For a long time, this branch of research was targeted at determining user sensitivity to impairments induced in the media by suboptimal delivery. The media recipient was considered a passive observer, whose appreciation of the video material was determined primarily by the degree of annoyance due to the impairments affecting it. With the advent of mobile technology and Internet-based media delivery, this impairment-centric concept of QoE has shown to be incomplete. The media recipient became an active user who creates content, interacts with the system, and selects the media he/she wants to have delivered. As a result, elements such as visual semantics, user personality, preferences and intent, social and environmental context of media fruition also concur to the final experience assessment. The role played by these elements in QoE and the cognitive/affective processes that underlie them are still to be understood, although several models of QoE appreciation have already been proposed. In this paper, we review the evolution of subjective QoE assessment and models from the impairment-centric approach to a more user-centric approach. We analyze relevant features and factors influencing QoE, and point out future directions for subjective QoE assessment research.

Crowdsourcing for Rating Image Aesthetic Appeal: Better a Paid or a Volunteer Crowd?

Crowdsourcing has the potential to become a preferred tool to study image aesthetic appeal preferences of users. Nevertheless, some reliability issues still exist, partially due to the sometimes doubtful commitment of paid workers to perform the rating task properly. In this paper we compare the reliability in scoring image aesthetic appeal of both a paid and a volunteer crowd. We recruit our volunteers through Facebook and our paid users via Microworkers. We conclude that, whereas volunteer participants are more likely to leave the rating task unfinished, when they complete it they do so more reliably than paid users.