Jong-Chul Yoon

Automated music video generation using multi-level feature-based segmentation

We show how to create a music video automatically, using computable characteristics of the video and music to promote coherent matching. We analyze the flow of both music and video, and then segment them into sequences of near-uniform flow. We extract features from the both video and music segments, and then find matching pairs. The granularity of the matching process can be adapted by extending the segmentation process to several levels. Our approach drastically reduces the skill required to make simple music videos.

A Hidden-picture Puzzles Generator

A hidden‐picture puzzle contains objects hidden in a background image, in such a way that each object fits closely into a local region of the background. Our system converts image of the background and objects into line drawing, and then finds places in which to hide transformed versions of the objects using rotation‐invariant shape context matching. During the hiding process, each object is subjected to a slight deformation to enhance its similarity to the background. The results were assessed by a panel of puzzle‐solvers.

Video Painting Based on a Stabilized Time-Varying Flow Field

We present a method for constructing 3D feature flow from video and its application to video stylization. Our method extracts smoothly aligned 3D vectors that describe the smallest variation of colors within a spatiotemporal video cube, and thus effectively preserves both spatial and temporal coherence in a relatively inexpensive manner. As an application of this flow field we present a particle-based video stylization technique to rerender the video in a feature enhancing, painterly style. Our method consists of per-pixel operations and is suitable for GPU implementation, which enables real-time video stylization.

Traffigram: distortion for clarification via isochronal cartography

Most geographic maps visually represent physical distance; however, travel time can in some cases be more important than distance because it directly indicates availability. The technique of creating maps from temporal data is known as isochronal cartography, and is a form of distortion for clarification. In an isochronal map, congestion expands areas, while ideal travel conditions make the map shrink in comparison to the actual distance scale of a traditional map. Although there have been many applications of this technique, detailed user studies of its efficacy remain scarce, and there are conflicting views on its practical value. To attempt to settle this issue, we utilized a user-centered design process to determine which features of isochronal cartography might be most usable in practice. We developed an interactive cartographic visualization system, Traffigram, that features a novel combination of efficient isochronal map algorithms and an interface designed to give map users a quick and seamless experience while preserving geospatial integrity and aesthetics. We validated our design choices with multiple usability studies. We present our results and discuss implications for design.

Stable and controllable noise

We introduce a stable noise function with controllable properties. The well-known Perlin noise function is generated by interpolation of a pre-defined random number table. This table must be modified if user-defined constraints are to be satisfied, but modification can destroy the stability of the table. We integrate statistical tools for measuring the stability of a random number table with user constraints within an optimization procedure, so as to create a controlled random number table which nevertheless has a uniform random distribution, no periodicity, and a band-limited property.

Optimized image resizing using flow-guided seam carving and an interactive genetic algorithm

In this paper, we introduce a novel method for content-aware image resizing based on flow-guided seam carving. It extends the existing seam carving framework by replacing the conventional energy field with a “structure-aware” energy field that takes into account the feature orientations in the image. Guided by this new energy field, our approach excels in preserving (i.e., avoiding the distortion of) important structures in the image, such as shape boundaries. We also present a simple user interface to further optimize the resizing result based on the genetic selection process among multiple resizing operators such as scaling, cropping, and flow-guided seam carving. We show that such simple user interaction, coupled with the genetic algorithm, dramatically increases the chances of producing the user-desired outcome.

Synchronized background music generation for video

We synchronize background music with a video by changing the timing of music, an approach that minimizes the damage to music data. Starting from a MIDI file and video data, feature points are extracted from both sources, paired, and then synchronized using dynamic programming to time-scale the music. We also introduce the music graph, a directed graph that encapsulates connections between many short music sequences. By traversing a music graph, we cangenerate large amounts of new background music, in which we expect to find a sequence which matches the video features better than the original music.

Feature-based synchronization of video and background music

We synchronize background music with a video by changing the timing of music, an approach that minimizes the damage to music data. Starting from a MIDI file and video data, feature points are extracted from both sources, paired, and then synchronized using dynamic programming to time-scale the music. We also introduce the music graph, a directed graph that encapsulates connections between many short music sequences. By traversing a music graph, we can generate large amounts of new background music, in which we expect to find a sequence which matches the video features better than the original music.

CartoonModes: Cartoon stylization of video objects through modal analysis

We transform the motion and shape of a video object into cartoon style using a set of representative cartooning deformations which we relate to a modal analysis of the object. Results can be obtained at interactive rate and a user survey confirmed their plausibility.

Image-based dress-up system

In this paper, we present an image-based virtual dress-up system according to user input model and garment image. A previous 3D scanner-based virtual dress-up system has expensive cost and under locational limitation issues, but our system integrates various image processing techniques and introduces an easy-to-use system for general users. We present that our system produces a visually plausible and well-fitted virtual dress-up results in a practical and usable way.