H. Vicky Zhao

Block-based adaptive compressed sensing for video

Compressed sensing is a novel technology to acquire and reconstruct signals below the Nyquist rate, and has great potential in image and video acquisition to explore the data redundancy and to significantly reduce the number of sampled data. In this paper, we explore the temporal redundancy in videos, and propose a block-based adaptive framework for compressed video sampling. It addresses the independent movement of different regions in a video, classifies blocks into different types depending on their inter-frame correlation, and adjusts the sampling and reconstruction strategies accordingly. Our framework also considers the diverse texture complexity of different regions, and adaptively adjusts the number of measurements collected for each region based on their sparsity. Our simulation results show that the proposed framework reduces the number of sampled measurements by 52% to 80% while still satisfying the quality constraint on the reconstructed frames. Compared to prior works, our proposed scheme improves the quality of the reconstructed frames and achieves a 0.8dB to 5.4dB gain in the average PSNR.

Behavior Dynamics in Media-Sharing Social Networks

In large-scale media-sharing social networks, where millions of users create, share, link, and reuse media content, there are clear challenges in protecting content security and intellectual property, and in designing scalable and reliable networks capable of handling high levels of traffic. This comprehensive resource demonstrates how game theory can be used to model user dynamics and optimize design of media-sharing networks. It reviews the fundamental methodologies used to model and analyze human behavior, using examples from realworld multimedia social networks. With a thorough investigation of the impact of human factors on multimedia system design, this accessible book shows how an understanding of human behavior can be used to improve system performance. Bringing together mathematical tools and engineering concepts with ideas from sociology and human behavior analysis, this one-stop guide will enable researchers to explore this emerging field further and ultimately design media-sharing systems with more efficient, secure, and personalized services.

Feature based classification of computer graphics and real images

Photorealistic images can now be created using advanced techniques in computer graphics (CG). Synthesized elements could easily be mistaken for photographic (real) images. Therefore we need to differentiate between CG and real images. In our work, we propose and develop a new framework based on an aggregate of existing features. Our framework has a classification accuracy of 90% when tested on the de facto standard Columbia dataset, which is 4% better than the best results obtained by other prominent methods in this area. We further show that using feature selection it is possible to reduce the feature dimension of our framework from 557 to 80 without a significant loss in performance (≪ 1%). We also investigate different approaches that attackers can use to fool the classification system, including creation of hybrid images and histogram manipulations. We then propose and develop filters to effectively detect such attacks, thereby limiting the effect of such attacks to our classification system.

Pollution-resistant peer-to-peer live streaming using trust management

In the emerging peer-to-peer (P2P) live streaming, users cooperate with each other to support efficient delivery of video over networks in live streaming applications. Pollution attack is an effective attack against P2P live streaming, where attackers upload bogus multimedia data to their peers. The polluted data can spread over the entire network, and cause severe quality degradation of the videos. To resist pollution attacks in P2P live streaming, this paper proposes a trust management system that identifies attackers and excludes them from further sharing of multimedia data. We investigate possible attacks against the trust management system and analyze the attack resistance of the proposed system. Our simulation results show that the proposed trust management system can efficiently detect attackers and stimulate user cooperation even under attacks. It helps users receive more clean data and improves the performance of P2P live streaming.

Anchor View Allocation for Collaborative Free Viewpoint Video Streaming

In free viewpoint video, a viewer can choose at will any camera angle or the so-called “virtual view” to observe a dynamic 3-D scene, enhancing his/her depth perception. The virtual view is synthesized using texture and depth videos of two anchor camera views via depth-image-based rendering (DIBR). We consider, for the first time, collaborative live streaming of a free viewpoint video, where a group of users may interactively pull and cooperatively share streams of different anchor views. There is a cost to access the anchor views from the live source, a cost to “reconfigure” the peer network due to a change in selected anchors during view switching, and a distortion cost due to the distance of the virtual views to the received anchor views at users. We optimize the anchor views allocated to users so as to minimize the overall streaming cost given by the access cost, reconfiguration cost, and view distortion cost. We first show that, if the reconfiguration cost due to view switching is negligible, the view allocation problem can be optimally and efficiently solved in polynomial time using dynamic programming. For the case of non-negligible reconfiguration cost, the problem becomes NP-hard. We thus present a locally optimal and centralized algorithm inspired by Lloyds algorithm used in non-uniform scalar quantization. We further propose a distributed algorithm with convergence guarantee, where each peer group independently makes merge-and-split decisions with a well-defined fairness criteria. Simulation results show that our algorithms achieve low streaming cost due to its excellent anchor view allocation.

Emotionally Representative Image Discovery for Social Events

With the emerging social networks, images have become a major medium for emotion delivery in social events due to their infectious and vivid characteristics. Discovering the emotionally representative images can help people intuitively understand the emotional aspects of social events. Prior works focus on finding the most visually representative images for the target queries or social events. However, the emotionally representative image should not only visually relevant with the social event, but also has a strong emotional appeal among people. In this paper, we propose an emotionally representative image discovery framework by jointly considering textual, visual and social factors. In particular, we build a hybrid link graph for images of each social event, where the weight of each link is measured by textual emotion information, visual similarity and social similarity. Then we propose the Visual-Social-Textual Rank (VSTRank) algorithm to calculate the importance score for each image, so that the emotionally representative images can be discovered under the constraint of textual, visual and social representativeness. To evaluate the effectiveness of our approach, we conduct a series of experiments with 15 social events extracted from real social media dataset, and evaluate the proposed method with both quantitative criterions and user study.

Joint pollution detection and attacker identification in peer-to-peer live streaming

In the emerging peer-to-peer (P2P) live streaming, users cooperate with each other to support efficient delivery of video over networks. Pollution attack is an effective attack against P2P live streaming, where attackers upload useless data to their peers, which may cause distrust among users. To resist pollution attacks and stimulate user cooperation in P2P live streaming, this paper proposes a joint pollution detection and attacker identification system, where polluted chunks are detected as early as possible and trust management is used to identify polluters. We analyze its performance and propose different schemes to address the tradeoff between pollution resistance and system overhead. Our simulation results show that the proposed system can effectively resist pollution attacks while minimizing the users computation overhead.

Fairness dynamics in multimedia colluders’ social networks

Multimedia social network analysis is a research area with growing importance, in which the social network members share multimedia contents with all different purposes and analyzing their behavior help design more secured and efficient multimedia and networking systems. In this paper, we focus on multimedia fingerprinting social network, in which multi-user collusion being a powerful attack, where a group of attackers collectively undermine the traitor tracing capability. During collusion, different colluders have different objective thus, the colluders form a social network and an how to achieve agreement on distributing the risk/profit among colluders and ensure fairness of the attack is a crucial question. This paper models the dynamics among colluders as a non-cooperative game, propose a general model of utility functions and study four different bargaining solutions of this game.

Optimizing peer grouping for live free viewpoint video streaming

In free viewpoint video, a user can pull texture and depth videos captured from two nearby reference viewpoints to synthesize his chosen intermediate virtual view for observation via depth-image-based rendering (DIBR). For users who are observing the same video at the same time but not necessarily from the same virtual viewpoint, they have incentive to pull the same reference views so that the streaming cost can be shared. On the other hand, in general distortion of a synthesized virtual view increases with its distance to the reference views, and so a user also has incentive to select reference views that tightly “sandwich” his chosen virtual view, minimizing distortion. In a previous work, reference view sharing strategies-ones that optimally trade off shared streaming costs with synthesized view distortions-were investigated for the case when users are first divided into groups, and each user group independently pulls two reference views and shares the resulting streaming cost. In this paper, we generalize the previous notion of user group, so that a user can simultaneously belong to two groups, and each group shares the streaming cost of a single view. We also aim to find a Nash Equilibrium (NE) solution of reference view selection, which is stable and from which no one has incentive to unilaterally deviate. Specifically, we first derive a lemma based on known properties of synthesized view distortion functions. We then design a search algorithm to find a NE solution, leveraging on the derived lemma to reduce search complexity. Experimental results show that the stable NE solution increases the overall cost only slightly when compared to the unstable optimal reference selection that gives the lowest overall cost. Further, a larger network will give a lower average cost for each user, and thus, users tend to join large networks for cooperation.

Distributed cooperative multicast in wireless networks: Performance analysis and optimal power allocation

For wireless multicast applications where a group of users subscribe to the same service and receive the same data, a promising solution to combat channel fading is to explore the cooperative diversity and let users help each other forward packets. This paper investigates a distributed cooperative multicast scheme that uses a maximal ratio combiner to enhance the received signal-to-noise ratio (SNR), and provides a thorough performance analysis. We derive a close-form formulation of the average outage probability, examine its asymptotic behavior in the high SNR regime, and investigate the optimal power allocation. Our analytical and simulation results show that cooperative multicast performs better in denser networks with more relays helping, and user cooperation can significantly reduce the outage probability, especially in the high SNR region.