Gunhee Kim

Distributed cosegmentation via submodular optimization on anisotropic diffusion

The saliency of regions or objects in an image can be significantly boosted if they recur in multiple images. Leveraging this idea, cosegmentation jointly segments common regions from multiple images. In this paper, we propose CoSand, a distributed cosegmentation approach for a highly variable large-scale image collection. The segmentation task is modeled by temperature maximization on anisotropic heat diffusion, of which the temperature maximization with finite K heat sources corresponds to a K-way segmentation that maximizes the segmentation confidence of every pixel in an image. We show that our method takes advantage of a strong theoretic property in that the temperature under linear anisotropic diffusion is a submodular function; therefore, a greedy algorithm guarantees at least a constant factor approximation to the optimal solution for temperature maximization. Our theoretic result is successfully applied to scalable cosegmentation as well as diversity ranking and single-image segmentation. We evaluate CoSand on MSRC and ImageNet datasets, and show its competence both in competitive performance over previous work, and in much superior scalability.

On multiple foreground cosegmentation

In this paper, we address a challenging image segmentation problem called multiple foreground cosegmentation (MFC), which concerns a realistic scenario in general Webuser photo sets where a finite number of K foregrounds of interest repeatedly occur cross the entire photo set, but only an unknown subset of them is presented in each image. This contrasts the classical cosegmentation problem dealt with by most existing algorithms, which assume a much simpler but less realistic setting where the same set of foregrounds recurs in every image. We propose a novel optimization method for MFC, which makes no assumption on foreground configurations and does not suffer from the aforementioned limitation, while still leverages all the benefits of having co-occurring or (partially) recurring contents across images. Our method builds on an iterative scheme that alternates between a foreground modeling module and a region assignment module, both highly efficient and scalable. In particular, our approach is flexible enough to integrate any advanced region classifiers for foreground modeling, and our region assignment employs a combinatorial auction framework that enjoys several intuitively good properties such as optimality guarantee and linear complexity. We show the superior performance of our method in both segmentation quality and scalability in comparison with other state-of-the-art techniques on a newly introduced FlickrMFC dataset and the standard ImageNet dataset.

Unsupervised modeling of object categories using link analysis techniques

We propose an approach for learning visual models of object categories in an unsupervised manner in which we first build a large-scale complex network which captures the interactions of all unit visual features across the entire training set and we infer information, such as which features are in which categories, directly from the graph by using link analysis techniques. The link analysis techniques are based on well-established graph mining techniques used in diverse applications such as WWW, bioinformatics, and social networks. The techniques operate directly on the patterns of connections between features in the graph rather than on statistical properties, e.g., from clustering in feature space. We argue that the resulting techniques are simpler, and we show that they perform similarly or better compared to state of the art techniques on common data sets. We also show results on more challenging data sets than those that have been used in prior work on unsupervised modeling.

Joint Summarization of Large-Scale Collections of Web Images and Videos for Storyline Reconstruction

In this paper, we address the problem of jointly summarizing large sets of Flickr images and YouTube videos. Starting from the intuition that the characteristics of the two media types are different yet complementary, we develop a fast and easily-parallelizable approach for creating not only high-quality video summaries but also novel structural summaries of online images as storyline graphs. The storyline graphs can illustrate various events or activities associated with the topic in a form of a branching network. The video summarization is achieved by diversity ranking on the similarity graphs between images and video frames. The reconstruction of storyline graphs is formulated as the inference of sparse time-varying directed graphs from a set of photo streams with assistance of videos. For evaluation, we collect the datasets of 20 outdoor activities, consisting of 2.7M Flickr images and 16K YouTube videos. Due to the large-scale nature of our problem, we evaluate our algorithm via crowdsourcing using Amazon Mechanical Turk. In our experiments, we demonstrate that the proposed joint summarization approach outperforms other baselines and our own methods using videos or images only.

The autonomous tour-guide robot Jinny

This paper explains a new tour-guide robot Jinny. The Jinny is developed by focusing on human robot interaction and autonomous navigation. In order to achieve reliable and safe navigation performance, an integrated navigation strategy is established based on the analysis of a robots states and the decision making process of robot behaviors. According to the condition of environments, the robot can select its motion algorithm among four types of navigation strategy. Also, we emphasized the manageability of a robots knowledge base for human friendly interactions. The robots knowledge base can be extended or modified intuitively enough to be managed by non-experts. In order to show the feasibility and effectiveness of our system, we also present experimental results of the navigation system and some experiences on practical installations.

Tripodal schematic design of the control architecture for the Service Robot PSR

This paper describes a control architecture design and a system integration strategy for the autonomous service robot PSR (Public Service Robot). The PSR is under development at the KIST (Korea Institute of Science and Technology) for service tasks in public spaces such as office buildings and hospitals. The proposed control architecture is designed by tripodal frameworks, which are layered functionality diagram, class diagram, and configuration diagram. The tripodal schematic design clearly points out the way of integrating various hardware and software components. The developed strategy is implemented on the PSR and successfully tested.

Jointly Aligning and Segmenting Multiple Web Photo Streams for the Inference of Collective Photo Storylines

With an explosion of popularity of online photo sharing, we can trivially collect a huge number of photo streams for any interesting topics such as scuba diving as an outdoor recreational activity class. Obviously, the retrieved photo streams are neither aligned nor calibrated since they are taken in different temporal, spatial, and personal perspectives. However, at the same time, they are likely to share common storylines that consist of sequences of events and activities frequently recurred within the topic. In this paper, as a first technical step to detect such collective storylines, we propose an approach to jointly aligning and segmenting uncalibrated multiple photo streams. The alignment task discovers the matched images between different photo streams, and the image segmentation task parses each image into multiple meaningful regions to facilitate the image understanding. We close a loop between the two tasks so that solving one task helps enhance the performance of the other in a mutually rewarding way. To this end, we design a scalable message-passing based optimization framework to jointly achieve both tasks for the whole input image set at once. With evaluation on the new Flickr dataset of 15 outdoor activities that consist of 1.5 millions of images of 13 thousands of photo streams, our empirical results show that the proposed algorithms are more successful than other candidate methods for both tasks.

Reconstructing Storyline Graphs for Image Recommendation from Web Community Photos

In this paper, we investigate an approach for reconstructing storyline graphs from large-scale collections of Internet images, and optionally other side information such as friendship graphs. The storyline graphs can be an effective summary that visualizes various branching narrative structure of events or activities recurring across the input photo sets of a topic class. In order to explore further the usefulness of the storyline graphs, we leverage them to perform the image sequential prediction tasks, from which photo recommendation applications can benefit. We formulate the storyline reconstruction problem as an inference of sparse time-varying directed graphs, and develop an optimization algorithm that successfully addresses a number of key challenges of Web-scale problems, including global optimality, linear complexity, and easy parallelization. With experiments on more than 3.3 millions of images of 24 classes and user studies via Amazon Mechanical Turk, we show that the proposed algorithm improves other candidate methods for both storyline reconstruction and image prediction tasks.

Development of the multi-functional indoor service robot PSR systems

This paper discusses the development of the multi-functional indoor service robot PSR (Public Service Robots) systems. We have built three versions of PSR systems, which are the mobile manipulator PSR-1 and PSR-2, and the guide robot Jinny. The PSR robots successfully accomplished four target service tasks including a delivery, a patrol, a guide, and a floor cleaning task. These applications were defined from our investigation on service requirements of various indoor public environments. This paper shows how mobile-manipulator typed service robots were developed towards intelligent agents in a real environment. We identified system integration, multi-functionality, and autonomy considering environmental uncertainties as key research issues. Our research focused on solving these issues, and the solutions can be considered as the distinct features of our systems. Several key technologies were developed to satisfy technological consistency through the proposed integration scheme.

Object recognition with 3d models

Intro: We propose several new ideas of how to use 3D models for viewbased object recognition. In an initial experiment we show that even the simple task of distinguishing between two objects requires large training sets if high accuracy and pose invariance are to be achieved. Using synthetic image data, we propose a method for quantifying the degree of difficulty of detecting objects across views and a novel alignment algorithm for pose-based clustering on the view sphere. Finally, we introduce an active learning algorithm that searches for local minima of a classifier’s output in a low-dimensional space of rendering parameters. Experimental setup: Synthetic training and test images were rendered from five textureless 3D models (see fig. 1) by moving a virtual camera on a sphere around each model. The models were illuminated by ambient light and a point light source. The six free rendering parameters were the camera’s location in azimuth and elevation, its rotation around its optical axis, the location of the point light source in azimuth and elevation, and the intensity ratio between ambient light and the point light. The rendered images were converted to 23×23 grayvalue images. From those we extracted 640 dimensional vectors of histograms of gradients. Our main classifier was an SVM with a Gaussian kernel.