Sheng-Jie Luo

Ambiguity-Free Edge-Bundling for Interactive Graph Visualization

Graph visualization has been widely used to understand and present both global structural and local adjacency information in relational data sets (e.g., transportation networks, citation networks, or social networks). Graphs with dense edges, however, are difficult to visualize because fast layout and good clarity are not always easily achieved. When the number of edges is large, edge bundling can be used to improve the clarity, but in many cases, the edges could be still too cluttered to permit correct interpretation of the relations between nodes. In this paper, we present an ambiguity-free edge-bundling method especially for improving local detailed view of a complex graph. Our method makes more efficient use of display space and supports detail-on-demand viewing through an interactive interface. We demonstrate the effectiveness of our method with public coauthorship network data.

Perspective-aware warping for seamless stereoscopic image cloning

This paper presents a novel technique for seamless stereoscopic image cloning, which performs both shape adjustment and color blending such that the stereoscopic composite is seamless in both the perceived depth and color appearance. The core of the proposed method is an iterative disparity adaptation process which alternates between two steps: disparity estimation, which re-estimates the disparities in the gradient domain so that the disparities are continuous across the boundary of the cloned region; and perspective-aware warping, which locally re-adjusts the shape and size of the cloned region according to the estimated disparities. This process guarantees not only depth continuity across the boundary but also models local perspective projection in accordance with the disparities, leading to more natural stereoscopic composites. The proposed method allows for easy cloning of objects with intricate silhouettes and vague boundaries because it does not require precise segmentation of the objects. Several challenging cases are demonstrated to show that our method generates more compelling results compared to methods with only global shape adjustment.

Artistic QR Code Embellishment

A QR code is a two-dimensional barcode that encodes information. A standard QR code contains only regular black and white squares, and thus is unattractive. This paper proposes a novel framework for embellishing a standard QR code, to make it both attractive and recognizable by any human while maintaining its scanability. The proposed method is inspired by artistic methods. A QR code is typically embellished by stylizing the squares and embedding images into it. In the proposed framework, the regular squares are reshaped using a binary examplar, to make their local appearances resemble the example shape. Additionally, an error-aware warping technique for deforming the embedded image is proposed; it minimizes the error in the QR code that is generated by the embedding of the image to optimize the readability of the code. The proposed algorithm yields lower data error than previous global transformation techniques because the warping can locally deform the embedded image to conform to the squares that surround it. The proposed framework was examined by using it to embellish an extensive set of QR codes and to test the readability with various commercial QR code readers.

Legolization: optimizing LEGO designs

Building LEGO sculptures requires accounting for the target objects shape, colors, and stability. In particular, finding a good layout of LEGO bricks that prevents the sculpture from collapsing (due to its own weight) is usually challenging, and it becomes increasingly difficult as the target object becomes larger or more complex. We devise a force-based analysis for estimating physical stability of a given sculpture. Unlike previous techniques for Legolization, which typically use heuristic-based metrics for stability estimation, our force-based metric gives 1) an ordering in the strength so that we know which structure is more stable, and 2) a threshold for stability so that we know which one is stable enough. In addition, our stability analysis tells us the weak portion of the sculpture. Building atop our stability analysis, we present a layout refinement algorithm that iteratively improves the structure around the weak portion, allowing for automatic generation of a LEGO brick layout from a given 3D model, accounting for color information, required workload (in terms of the number of bricks) and physical stability. We demonstrate the success of our method with real LEGO sculptures built up from a wide variety of 3D models, and compare against previous methods.

Geometrically Consistent Stereoscopic Image Editing Using Patch-Based Synthesis

This paper presents a patch-based synthesis framework for stereoscopic image editing. The core of the proposed method builds upon a patch-based optimization framework with two key contributions: First, we introduce a depth-dependent patch-pair similarity measure for distinguishing and better utilizing image contents with different depth structures. Second, a joint patch-pair search is proposed for properly handling the correlation between two views. The proposed method successfully overcomes two main challenges of editing stereoscopic 3D media: (1) maintaining the depth interpretation, and (2) providing controllability of the scene depth. The method offers patch-based solutions to a wide variety of stereoscopic image editing problems, including depth-guided texture synthesis, stereoscopic NPR, paint by depth, content adaptation, and 2D to 3D conversion. Several challenging cases are demonstrated to show the effectiveness of the proposed method. The results of user studies also show that the proposed method produces stereoscopic images with good stereoscopics and visual quality.

SD Models: Super-Deformed Character Models

Super‐deformed, SD, is a specific artistic style for Japanese manga and anime which exaggerates characters in the goal of appearing cute and funny. The SD style characters are widely used, and can be seen in many anime, CG movies, or games. However, to create an SD model often requires professional skills and considerable time and effort. In this paper, we present a novel technique to generate an SD style counterpart of a normal 3D character model. Our approach uses an optimization guided by a number of constraints that can capture the properties of the SD style. Users can also customize the results by specifying a small set of parameters related to the body proportions and the emphasis of the signature characteristics. With our technique, even a novel user can generate visually pleasing SD models in seconds.

Rephotography Using Image Collections

This paper proposes a novel system that “rephotographs” a historical photograph with a collection of images. Rather than finding the accurate viewpoint of the historical photo, users only need to take a number of photographs around the target scene. We adopt the structure from motion technique to estimate the spatial relationship among these photographs, and construct a set of 3D point cloud. Based on the user‐specified correspondences between the projected 3D point cloud and historical photograph, the camera parameters of the historical photograph are estimated. We then combine forward and backward warping images to render the result. Finally, inpainting and content‐preserving warping are used to refine it, and the photograph at the same viewpoint of the historical one is produced by this photo collection.

Painting photolization

Before the widespread of modern capture devices, painting served an important role in recording and depicting the real world for human. These painting artworks not only preserve the immediate depiction but also with good aesthetic sense. However, most of the the depictions in the painting are impossible to be reproduced in modern devices. In this extend abstract, we present a method that photolizes an input painting artwork. Our method generates an image which resembles the scene of the painting and has a photo-realistic appearance. A user provides a number of photos which are partly similar to the painting, and specifies a number of corresponding edge strokes in the painting and one of the photos indicating corresponding object edges. Our method automatically deforms these corresponding edges and composites these photos together.

Dynamic Media Assemblage

This paper presents dynamic media assemblage, which is a new presentation and summarization method for images and videos on a 2-D canvas. Instead of using the keyframes of the videos to generate a still image summarization, our method allows the videos to play simultaneously on the canvas while utilizing the limited space efficiently. This technique uses an efficient iterative packing algorithm, and as a result is well suited for interactive manipulations of media files within the assemblages in real time, such as insertion, deletion, and rearrangement. Our method starts by detecting shot boundaries and dividing longer input videos into individual shots. Within each shot, its temporal-spatial salient regions are extracted and used to recover camera motions. These saliency information further defines important regions within individual videos or images, which allows us to preserve visually important regions while packing the media files more efficiently into media assemblages. Our algorithm is iterative and can therefore quickly adjust to new canvas sizes or other user intentions. We demonstrate the effectiveness of our techniques by applying our methods to several applications, including media collection presentation, single video dynamic summary, personal media file browser, and interactive video wall.

Stroke-guided Image Synthesis for Skeletal Structure Editing

Creating variations of an image object is an important task, which usually requires manipulating the skeletal structure of the object. However, most existing methods (such as image deformation) only allow for stretching the skeletal structure of an object: modifying skeletal topology remains a challenge. This paper presents a technique for synthesizing image objects with different skeletal structures while respecting to an input image object. To apply this technique, a user firstly annotates the skeletal structure of the input object by specifying a number of strokes in the input image, and draws corresponding strokes in an output domain to generate new skeletal structures. Then, a number of the example texture pieces are sampled along the strokes in the input image and pasted along the strokes in the output domain with their orientations. The result is obtained by optimizing the texture sampling and seam computation. The proposed method is successfully used to synthesize challenging skeletal structures, such as skeletal branches, and a wide range of image objects with various skeletal structures, to demonstrate its effectiveness.