Chunjia Hu

Information security display system based on Spatial Psychovisual Modulation

Privacy protection is of increasing importance in this era of information explosion. This paper introduces an information security display system based on the idea of Spatial Psycho-visual Modulation (SPVM). With the rapid advance of modern manufacturing techniques, display devices now support very high pixel density (e.g. the retina display of Apple). Meanwhile the human visual system (HVS) cannot distinguish image signals with spatial frequency above a threshold, as predicted by the contrast sensitivity function (CSF). Therefore, it is now possible for us to devise a type of information security display using the mismatch between resolutions of modern display devices and the HVS. Given the desired visual stimuli for both the bystanders and the authorized users, we propose a method to design display signals accordingly. We select polarization as a way to effectively differentiate the bystanders and the authorized users. Operationally, applying complementary polarization to light emitted from different spatial sections of the display could in itself be a challenge. Fortunately, the development of stereoscopic display technologies has made available a lot of polarization based spatial multiplexing type of display devices. Hardware of the information security display system is based on a polarization based stereoscopic screen made by LG. Software of the information security display system is written in C++ with SDKs of DirectX and etc. Kinect is also included into our system to enhance the experience of human-computer interaction. Extended experimental results will be given in this paper to justify the effectiveness and robustness of the system. The developed system serves both as a proof-of-concept of the SPVM method, as well as a test bed for future research of SPVM based display technology.

Influence of compression artifacts on visual attention

Visual attention is an important function of the human visual system (HVS). In the long term research of visual attention, various computational models have been proposed with encouraging results. However, most of those work were conducted on images with ideal visual quality. In practice, outputs of most visual communication systems contain different levels of artifacts, e.g. noise, blurring, blockiness and etc. Therefore, it is interesting to investigate the impacts of artifacts on visual attention. In this paper, we question into the problem of how the widely encountered JPEG compression artifacts affect visual attention. We designed eye-tracking experiments on images with different levels of compression and viewing time and quantitatively compared the recorded eye movement data. We found that compression level does have impacts on visual attention, and yet this influence can be negligible for low levels of compression. For high levels of compression, the visual artifacts alter visual attention in a systematic way. Dependence of the influence on viewing duration was also analyzed and it was observed that too short or too long viewing time reduces the impact of compression artifacts on visual attention.

Dual-view medical image visualization based on spatial-temporal psychovisual modulation

Medical imaging technologies such as magnetic resonance imaging (MRI) and computerized tomographic (CT) are used to diagnose a wide range of medical diseases. Medical images are generated by detecting density differences between different tissues in the body. Multiple medical image visualization is of critical importance to diagnosis. This paper introduces a dual-view medical image visualization prototype based on spatial-temporal psychovisual modulation (STPVM). Temporal psychovisual modulation (TPVM) enables a single display to generate multiple visual content for different viewers. Spatial psychovisual modulation (SPVM) extends the idea of TPVM to spatial domain. STPVM combines TPVM and SPVM by exploiting both temporal and spatial redundancy of modern displays. Based on STPVM technology, one display can present even more images simultaneously. In this demo, two kinds of medical images e.g. T1 and T2 weighted MRI images, are presented simultaneously. Physicians can switch between either image by just moving the eye fixations. Since T1 and T2 are shown simultaneously and are aligned on the screen, it is more convenient for the physicians to get different information of the same spot from the T1 and T2 images. The developed demo is useful for physicians during surgery navigation and effectively reduces the burden of mental transfer.

Sound influences visual attention discriminately in videos

In the long term research of visual attention, various computational models have been proposed. However, most of those works do not take audio into consideration. In practice, visual signals often come along with audio. Therefore, it is natural to investigate the influence of audio on visual attention. In this paper, we focus on the problem of when will audio influence visual attention during video watching. We perform eye-tracking experiments on a set of 60 videos in audio-visual (AV) and visual (V) conditions. Based on the eye movement data, we find that the influence of audio on visual attention depends on the consistency between the visual and audio signals. If the salient objects from the visual perspective are not consistent with the salient objects from the audio perspective, audio will influence visual attention. Otherwise, audio has little influence on visual attention.

Simultaneous triple subtitles exhibition via Temporal Psychovisual Modulation

With the quick acceleration of cultural exchange and globalization, multi-language annotation for in-theatre movie exhibition is gaining more attention. The existing solution overlays a stack of subtitles in different languages over the movie, for example, the familiar combination of English + local tongue as seen in the cinema today. However, a major drawback of this straightforward piling-up type of solution is that as the required number of languages increases, the subtitle area grows and it may eventually span over the movie contents. An effective multi-language subtitling system without compromising the movie watching experience is therefore highly desired. In this paper, we propose a new Temporal Psychovisual Modulation (TPVM) based solution to the multi-language subtitling problem. TPVM is a new paradigm of information display exploiting the mismatch between high refresh rate of the modern optoelectronic displays and limited temporal resolution of the human visual system (HVS). In this work, we design a simultaneous triple subtitle exhibition system using 2 120 Hz stereoscopic DLP projectors with linear polarization filters and LCD shutter glasses. Wearing different pairs of glasses, the audience can enjoy the movie with 3 optional subtitles without interfering with the audience without the glasses and see the movie directly. Extended experimental results will be given in this paper to justify the effectiveness and robustness of the proposed multiple subtitling system.

Fixation prediction through multimodal analysis

In this paper, we propose to predict human fixations by incorporating both audio and visual cues. Traditional visual attention models generally make the utmost of stimulis visual features, while discarding all audio information. But in the real world, we human beings not only direct our gaze according to visual saliency but also may be attracted by some salient audio. Psychological experiments show that audio may have some influence on visual attention, and subjects tend to be attracted the sound sources. Therefore, we propose to fuse both audio and visual information to predict fixations. In our framework, we first localize the moving-sounding objects through multimodal analysis and generate an audio attention map, in which greater value denotes higher possibility of a position being the sound source. Then we calculate the spatial and temporal attention maps using only the visual modality. At last, the audio, spatial and temporal attention maps are fused, generating our final audio-visual saliency map. We gather a set of videos and collect eye-tracking data under audio-visual test conditions. Experiment results show that we can achieve better performance when considering both audio and visual cues.

An Augmented-Reality night vision enhancement application for see-through glasses

Nyctalopia, millions of people all over the world suffering from which, brings much trouble to the patients. The available night vision systems are poor in user experience but cost much. In this paper we design and implement an Augmented-Reality night vision enhancement application for see-through glasses. According to our model proposed for the night blindness, fast and efficient algorithms are used for both night vision enhancement and display calibration. The processed images are much brighter and aligned to the real world. The results in different daily life scenes are presented, indicating huge convenience the application will bring to the nyctalopia patients.

Visible Light Communication via Temporal Psycho-Visual Modulation

In this paper we propose a new paradigm for visible light communication (VLC) using the emerging display technology of Temporal Psycho-Visual Modulation (TPVM) that exploits the interaction between human visual system and modern electro-optical display devices. Unlike traditional VLC, no specifically designed light emitter and receiver are required. In the proposed system, light projector is used as the information source and digital cameras act as information decoder. The emitted light is designed in a specific way such that it can carry meaningful information (or simply works as an illumination source) for human eyes while other message can be decoded by the digital camera due to the fundamental difference in the imaging mechanism of the human eye and digital devices. We further describe two applications of this new type of VLC in ubiquitous augmented reality and illegal camcorder-recording prevention with extensive experimental results.

Simultaneous dual-subtitles exhibition via Spatial Psychovisual Modulation

Dual-language annotation for in-theatre movie exhibition is a useful technique to facilitate the understanding of the movie for audiences from different nations and with different cultural backgrounds. Currently, the most popular solution is the direct superimposition of subtitles in a pair of different languages over the movie, e.g. English + Chinese which can often be seen in cinema in China today. An obvious drawback of this straightforward solution is that the subtitle area often occludes the movie contents and even becomes annoying for the audience. In this paper, we propose a new Spatial Psychovisual Modulation (SPVM) based solution to the dual-language subtitling problem. SPVM is a new paradigm of information display exploiting the mismatch between high resolution of the modern optoelectronic displays and limited spatial resolution of the human visual system (HVS). In this work, we design a simultaneous dual-subtitle exhibition system using 2 synchronized projectors with linear polarization filters and polarization glasses. Most audiences can enjoy the movie as usual with a default subtitle language, say English, while others have the option of only seeing subtitle in the other language, e.g. Chinese using the polarization glasses. We have implemented the system and experimental results will be demonstrated to justify the effectiveness and robustness of the proposed dual-subtitle exhibition system.

Uncrowded window inspired information security display

With the booming of visual media, people pay more and more attention to privacy protection in public environments. Most existing research on information security such as cryptography and steganography is mainly concerned about transmission and yet little has been done to prevent the information displayed on screens from reaching eyes of the bystanders. To deal with the problem, for the application of text-reading, we proposed an eye tracking based solution using the newly revealed concept of uncrowded window from vision research. The theory of uncrowded window suggests that human vision can only effectively recognize objects inside a small window. Object features outside the window may still be detectable but the feature detection results cannot be efficiently combined properly and therefore those objects will not be recognizable. We use eye-tracker to locate fixation points of the authorized reader in real time, and only the area inside the uncrowded window displays the private information we want to protect. A number of dummy windows with fake messages are displayed around the real uncrowded window as diversions. And without the precise knowledge about the fixations of the authorized reader, the chance for bystanders to capture the private message from those surrounding area and the dummy windows is very low. Meanwhile, since the authorized reader can only read within the uncrowded window, detrimental impact of those dummy windows is almost negligible. The proposed prototype system was written in C++ with SDKs of Direct3D, Tobii Gaze SDK, CEGUI, MuPDF, OpenCV and etc. Extended demonstration of the system will be provided to show that the proposed method is an effective solution to the problem of information security and display.