Seungji Yang

Semantic Home Photo Categorization

A semantic categorization method for generic home photo is proposed. The main contribution of this paper is to exploit a two-layered classification model incorporating camera metadata with low-level features for multilabel detection. The two-layered support vector machine (SVM) classifiers operate to detect local and global photo semantics in a feed-forward way. The first layer aims to predict likelihood of predefined local photo semantics based on camera metadata and regional low-level visual features. In the second layer, one or more global photo semantics is detected based on the likelihood. To construct classifiers producing a posterior probability, we use a parametric model to fit the output of SVM classifiers to posterior probability. A concept merging process based on a set of semantic-confidence maps is also presented to cope with selecting more likelihood photo semantics on spatially overlapping local regions. Experiment was performed with 3086 photos that come from MPEG-7 visual core experiment two official databases. Results showed that the proposed method would much better capture multiple semantic meanings of home photos, compared to other similar technologies

Face annotation for personal photos using context-assisted face recognition

Face annotation for personal photos has a number of potential applications in Multimedia Information Retrieval (MIR). We propose a novel face annotation method that systematically combines contextual information of photos with traditional Face Recognition (FR) technologies. To overcome the limitations of conventional FR-based annotation approaches, situation and person clustering methods are proposed based on context of personal photos. By relying on the proposed clustering methods, multiple facial images that belong to the same person are applied to subspace FR methods. This allows traditional FR methods to take the advantages of feature- or measurement-level fusion strategies for automatic annotation of faces on personal photos. The effectiveness of the proposed method was successfully tested on 1,120 personal photos selected from standard MPEG-7 Visual Core Experiment 3 (VCE-3). Extensive experimental results showed that the proposed method improves annotation accuracy by at least an order of magnitude compared with conventional FR-based approaches.

Visual contents adaptation for color vision deficiency

In this paper, we propose methods to adapt colors on the visual content for people with color vision deficiency. The proposed adaptation consists of two parts: adaptations for dichromat and anomalous trichromat. The adaptation for dichromats aims to give them better color information, while the adaptation for anomalous trichromats aims to give them original color. To verify the proposed methods, we used both quantitative and qualitative measurements. Experimental results showed that the proposed adaptation enhanced color information readability of the people with color vision deficiency.

Quantification and standardized description of color vision deficiency caused by anomalous trichromats-part I: simulation and measurement

The MPEG-21 Multimedia Framework allows visually impaired users to have an improved access to visual content by enabling content adaptation techniques such as color compensation. However, one important issue is the method to create and interpret the standardized CVD descriptions when making the use of generic color vision tests. In Part I of our study to tackle the issue, we present a novel computerized hue test (CHT) to examine and quantify CVD, which allows reproducing and manipulating test colors for the purposes of computer simulation and analysis of CVD. Both objective evaluation via color difference measurement and subjective evaluation via clinical experiment showed that the CHT works well as a color vision test: it is highly correlated with the Farnsworth-Munsell 100 Hue (FM100H) test and allows for a more elaborate and correct color reproduction than the FM100H test.

User-centric digital home photo album

In this paper, we propose a provision of user-centric home photo album system and albuming functions to support it. The main idea of the user-centric photo album is to enable users to organize and browse their photos along the semantically meaningful axes that are situation, person-identity, and category. Experiment results showed that the proposed method would be useful to album photos based on human perception.

Color adaptation for anomalous trichromats

In this article, we propose a content‐based color adaptation method for color vision impairment, especially for anomalous trichromats, to improve color information accessibility. Color degradation caused by anomalous trichromacy is compensated in digital color content with a range of the visible spectrum corresponding to color deficiency characteristics. To verify the usefulness of the proposed method, we performed clinical experiments as well as computer simulations. The results of both the experiments and the simulations show that the proposed adaptation could convey better color information to anomalous trichromats.

Semantic categorization of digital home photo using photographic region templates

In this paper, a semantic categorization method in generic home photos is proposed. In recent years, the semantic categorization of image has been a challenge due to the proliferation of digital home photos. Our approach is to detect semantically meaningful concepts contained in a photo. The proposed method incorporates an intermediate level of concepts, called local concept, so that it catches well semantic meaning of local regions of image as bridging the semantic gap of the low-level features and high-level category concepts. To detect the local concepts from the home photo, region segmentation by photographic region template and concept merging is also proposed. The efficacy of the proposed semantic categorization method was demonstrated with 3828 general home photos. The experiment results showed the proposed categorization method would be useful to detect multiple semantic meaning of the home photos.

Home photo categorization based on photographic region templates

In this paper, we propose new photo categorization which is suitable for a home photo album. To enhance the categorization, both local and global concepts of the photos are modeled and their combined concept learning method for the photo categorization is proposed. The local and global concepts are trained by individual support vector machines. Region templates for the local concepts of generic home photos are proposed. Further, local concepts are merged with confidence to lead to the global concept to achieve reliable categorization. Experiment results show that the proposed method is useful to detect multi-category concepts for the home photo album.

Quantification and standardized description of color vision deficiency caused by anomalous trichromats-part II: modeling and color compensation

A color compensation scheme has been developed to enhance the perception of people with color vision deficiency (CVD) and for people suffering from anomalous trichromacy. It is operated within the MPEG-21 Multimedia Framework, which provides a standardized description of CVD. The basic idea behind the proposed color compensation consists of simulating the path of human color perception. As such, compensated color is realized by relying on the spectral cone sensitivities of the human eye and the spectral emission functions of the display device. For quantified color compensation, the spectral sensitivity of anomalous cones has been modeled according to the deficiency degree of the standardized CVD description. The latter is based on the error score of a computerized hue test (CHT), developed in Part I of our study. Given the anomalous cone spectra, the reduction of error score on the CHT after color compensation was measured in each deficiency degree. The quantitative relationship of color compensation with the error score is linearly regressed, based on the deficiency degree with the least error score after color compensation as well as the error score before color compensation.

Semantic Photo Album Based on MPEG-4 Compatible Application Format

In this paper, we propose a promising photo album system that enables augmented use of digital photos over a wide range of multimedia devices and semantic consumption of photos as well. The proposed system contains an album engine to create metadata and a photo player to consume the photos. It operates based on a standardized application format to encode a group of photos and associated metadata. Experiments showed that the proposed system achieved reasonable albuming performance.