Milan Mirkovic

Salient Motion Features for Video Quality Assessment

Design of algorithms that are able to estimate video quality as perceived by human observers is of interest for a number of applications. Depending on the video content, the artifacts introduced by the coding process can be more or less pronounced and diversely affect the quality of videos, as estimated by humans. While it is well understood that motion affects both human attention and coding quality, this relationship has only recently started gaining attention among the research community, when video quality assessment (VQA) is concerned. In this paper, the effect of calculating several objective measure features, related to video coding artifacts, separately for salient motion and other regions of the frames of the sequence is examined. In addition, we propose a new scheme for quality assessment of coded video streams, which takes into account salient motion. Standardized procedure has been used to calculate the Mean Opinion Score (MOS), based on experiments conducted with a group of non-expert observers viewing standard definition (SD) sequences. MOS measurements were taken for nine different SD sequences, coded using MPEG-2 at five different bit-rates. Eighteen different published approaches related to measuring the amount of coding artifacts objectively on a single-frame basis were implemented. Additional features describing the intensity of salient motion in the frames, as well as the intensity of coding artifacts in the salient motion regions were proposed. Automatic feature selection was performed to determine the subset of features most correlated to video quality. The results show that salient-motion-related features enhance prediction and indicate that the presence of blocking effect artifacts and blurring in the salient regions and variance and intensity of temporal changes in non-salient regions influence the perceived video quality.

Mining web videos for video quality assessment

Correlating estimates of objective measures related to the presence of different coding artifacts with the quality of video as perceived by human observers is a non-trivial task. There is no shortage of data to learn from, thanks to the Internet and web-sites such as YouTubetm. There has, however, been little done in the research community to try to use such resources to advance our understanding of perceived video quality. The problem is the fact that it is not easy to obtain the Mean Opinion Score (MOS), a standard measure of the perceived video quality, for more than a handful of videos. The paper presents an approach to determining the quality of a relatively large number of videos obtained randomly from YouTubetm. Several measures related to motion, saliency and coding artifacts are calculated for the frames of the video. Programmable graphics hardware is used to perform clustering: first, to create an artifacts-related signature of each video; then, to cluster the videos according to their signatures. To obtain an estimate for the video quality, MOS is obtained for representative videos, closest to the cluster centers. This is then used as an estimate of the quality of all other videos in the cluster. Results based on 2,107 videos containing some 90,000,000 frames are presented in the paper.

Evaluating the role of content in subjective video quality assessment.

Video quality as perceived by human observers is the ground truth when Video Quality Assessment (VQA) is in question. It is dependent on many variables, one of them being the content of the video that is being evaluated. Despite the evidence that content has an impact on the quality score the sequence receives from human evaluators, currently available VQA databases mostly comprise of sequences which fail to take this into account. In this paper, we aim to identify and analyze differences between human cognitive, affective, and conative responses to a set of videos commonly used for VQA and a set of videos specifically chosen to include video content which might affect the judgment of evaluators when perceived video quality is in question. Our findings indicate that considerable differences exist between the two sets on selected factors, which leads us to conclude that videos starring a different type of content than the currently employed ones might be more appropriate for VQA.

Logging real packet reception patterns for end-to-end quality of experience assessment in wireless multimedia transmission

While a fairly large number of databases exist that provide impaired video sequences designed for the development and evaluation of Quality of Experience (QoE) approaches, the impairments due to transmission errors resulting in packet loss in these databases are based on simulation of small number of scenarios and not representative of real transmission scenarios that arise in end-to-end transmission of multimedia to mobile devices. This paper proposes a solution to this problem in the form of a framework for recording real packet drops as they occur in different situations of wireless multimedia transmission to mobile devices. The logs can be used to generate realistic impairments, as well as design new highly-efficient quality assessment approaches based on monitoring network performance in real time. An Android-based mobile device is used to receive streamed H.264 videos and real Real-time Transport Protocol (RTP) packet reception sequences (traces) are recorded. To evaluate the approach and the impact of different transmission scenarios on perceived quality, a study of the quality experienced by the users in 3 different real wireless transmission scenarios was conducted and the results are presented in the paper, showing that they diverge significantly from the packet-loss sequences usually considered in quality of experience studies.

A comparative study of spatial, temporal and content-based patterns emerging in YouTube and Flickr

Due to the recent advances and wide adoption of Web 2.0 technologies, there is an abundance of publicly available user generated content, which can be a valuable resource for researchers, enabling them to apply sophisticated analysis methods on data of unprecedented scale. analysis methods on data of unprecedented scale.

MR Image Based Approach for Metal Artifact Reduction in X-Ray CT

For decades, computed tomography (CT) images have been widely used to discover valuable anatomical information. Metallic implants such as dental fillings cause severe streaking artifacts which significantly degrade the quality of CT images. In this paper, we propose a new method for metal-artifact reduction using complementary magnetic resonance (MR) images. The method exploits the possibilities which arise from the use of emergent trimodality systems. The proposed algorithm corrects reconstructed CT images. The projected data which is affected by dental fillings is detected and the missing projections are replaced with data obtained from a corresponding MR image. A simulation study was conducted in order to compare the reconstructed images with images reconstructed through linear interpolation, which is a common metal-artifact reduction technique. The results show that the proposed method is successful in reducing severe metal artifacts without introducing significant amount of secondary artifacts.

Mining Geo-Referenced Community-Contributed Multimedia Data

Besides connecting users and allowing interactions between them, social networks are becoming an increasingly popular medium for sharing multimedia content, such as images and videos. Due to technological advances it has become extremely simple to create and share such content in (near) real-time, and even associate it with a location where it was made (i.e. geo-reference it). All of this has caused tremendous amounts of geo-referenced multimedia content to become publicly available, which made it suitable for analysis by employing different visualization and data-mining techniques. This chapter presents some of the techniques and methods for mining geo-referenced multimedia content in order to discover patterns and trends in it, which can lead to better understanding of the phenomena driving the data generation in the first place.

Suppression of Metal Artefacts in CT Using Virtual Singorams and Corresponding MR Images

Medical imaging is invaluable when it comes to gaining insight into the human body. As is well known, medical images need to deal with artefacts. This article presents a modern procedure for metal artifact reduction in computed tomography, which relies on additional information extracted from corresponding magnetic resonance images. We conducted a simulation study so as to compare the resulting images with those corrected, using the baseline linear interpolation method. The outcome indicates that the proposed method incomparably outperforms the baseline and reduces metal artefacts, improving the quality of images, which can be later used in a clinical setting.

Video Quality Assessment for Mobile Devices on Mobile Devices

Pervasiveness of mobile devices and ubiquitous broadband Internet access have laid foundations for video content to be consumed increasingly on smart phones or tablets. As over 85% of the global consumer traffic by 2016 is estimated to be generated by streaming video content, video quality as perceived by end-users of such devices is becoming an important issue. Most of the studies concerned with Video Quality Assessment (VQA) for mobile devices have been carried out in a carefully controlled environment, thus potentially failing to take into account variables or effects present in real-world conditions. In this paper, we compare the results of traditional approach to VQA for mobile devices to those obtained in real-world conditions by using a physical mobile device, for the same video test-set. Results indicate that a difference in perceived video quality between the two settings exists, thus laying foundations for further research to explain the reasons behind it.

Using YouTube data to analyze human continent-level mobility

An increasing amount of publicly available geo-referenced data enables the identification of patterns of behavior, habits and movements of people. This paper presents results of a case study analysis based on data ser containing publicly available geo-referenced videos downloaded from YouTube, tagged as recorded in Africa. Our goal was to determine major routes of movement across the continent, to determine when people start their trip and the basic means of transportation used. The paper presents results of the analysis conducted on 113.157 unique YouTube records. We were able to identify major travel routes, directions, carriers and even flights favored by people traveling across Africa, information that is potentially valuable to disease outbreak management, airline industry, etc.