Jinki Jung

Hybrid Visual Tracking for Augmented Books

The augmented book is the system augmenting multimedia elements onto a book to bring additional education effects or amusement. A book includes many pages and many duplicated designs so that tracking a book is quite difficult. For the augmented book, we propose the hybrid visual tracking which merges the merits of two traditional approaches: fiducial marker tracking and markerless tracking. The new method does not cause visual discomfort and can stabilizes camera pose estimation in real-time.

Real-time robust body part tracking for augmented reality interface

We proposed real-time robust body part tracking for augmented reality interface that does not limit the users freedom. The generality of the system was upgraded relative to body part tracking by establishing an ability to recognize details, such as, whether the user wears long sleeves or short sleeves. For precise body part tracking, we obtained images of hands, head, and feet separately via a single camera, and when detecting each body part, we separately chose appropriate features for specific parts. Using a calibrated camera, we transferred 2D detected body parts into an approximate 3D posture. In experiments conducted to evaluate the body part tracking module, the application with the proposed interface showed advanced hand tracking performance in real time(43.5fps).

Real-time recognition and tracking for augmented reality books

An augmented reality book (AR book) is an application in which such multimedia elements as virtual 3D objects, movie clips, or sound clips are augmented to a conventional book using augmented reality technology. It can provide better understanding about the contents and visual impressions for users. For AR books, this paper presents a markerless tracking method, which recognizes and tracks a large number of pages in real‐time, even on PCs with low computation power. For fast recognition with respect to a large number of pages, we propose a generic randomized forest that is an extension of a randomized forest. In addition, we define the spatial locality of the subregions in an image to resolve the problem of a dropping recognition rate under a complex background. For tracking with minimal jittering, we also propose the adaptive keyframe‐based tracking method, which automatically updates the current frame as a keyframe when it describes the page better than the existing one. Copyright

Smartphone as an augmented reality authoring tool via multi-touch based 3D interaction method

In this paper we present an Augmented Reality (AR) authoring tool for smartphones which facilitates intuitive interactions that manipulate the augmented virtual objects in real-time. A novel 3D interaction method using multi-touch interface and camera pose is proposed for intuitive authoring. With the gestures of two fingers on the touch screen, the user can adjust 3 DOF translation and 3 DOF rotation to a selected virtual object. The capabilities of the authoring tool are demonstrated on a smartphone.

Novel Applications of VR: Efficient mobile AR technology using scalable recognition and tracking based on server-client model

Advancements in mobile devices and vision technology have enabled mobile Augmented Reality (AR) to be serviced in real-time using natural features. However, in viewing AR while moving around in the real world, users often encounter new and diverse target objects. Whether the AR system is scalable to the number of target objects is a very crucial issue for mobile AR services in the real world. This scalability, however, has been severely limited because of the small internal storage capacity and memory of the mobile devices. In this paper, a new framework is proposed that achieves scalability for mobile AR. The scalability is achieved with a bag-of-visual-words based recognition module on the server side that is connected to the clients, which are mobile devices, through a conventional Wi-Fi network. On the client side, the coarse-to-fine tracking module enables robust tracking performance with natural features in real-time. In this study, we optimized modules in mobile devices for expediting pose-tracking processing and simultaneously enabled 3D rendering and animation in real-time. We also propose an efficient recognition method in which metadata are provided by the sensors of mobile devices. In the experiment, it takes approximately 0.2s for the cold start of an AR service initiated on a 10K object database with a recognition accuracy of 99.87%, which should be acceptable for a variety of real-world mobile AR applications.

AR postcard: the augmented reality system with a postcard

This paper proposes the postcard system which recognizes a card using a camera within a device (PC/Smart TV/Tablet PC) and augments the video of the sender fit to the card size. The postcards or the picture cards used in our work are trained in advance. The proposed system consists of two parts, the video management module and the augmented reality module. The video management module manages playable videos and makes ID as a link between the video and the card. The augmented reality module recognizes the postcard or the picture card using the trained data which is mentioned above and estimates ID of the card and also its three-dimensional position. The ID is sent to the video management module to get the video according to the ID of the card. After that, the augmented reality module synthesizes the video to the three-dimensional position of the card. As a result, the sender shows the video to the receiver fit to the card. All process in the proposed paper is performed in real-time.

Multiple page recognition and tracking for augmented books

An augmented book is an application that augments virtual 3D objects to a real book via AR technology. For augmented books, some markerless methods have been proposed so far. However, they can only recognize one page at a time. This leads to restrictions on the utilization of augmented books. In this paper, we present a novel markerless tracking method capable of recognizing and tracking multiple pages in real-time. The proposed method builds on our previous work using the generic randomized forest (GRF). The previous work finds out one page in the entire image using the GRF, whereas the proposed method detects multiple pages by dividing an image into subregions, applying the GRF to each subregion and discovering spatial locality from the GRF results.

AR paint: a fusion system of a paint tool and AR

In this paper, we present AR Paint, a fusion system of a paint tool and Augmented Reality (AR) that can augment in-place hand-drawn paintings onto an object. The system allows users to draw anything using the paint tool and then augment it onto an object whenever they want while providing a new experience and enjoyment during the process. It also has potential value if used in industry or for medical purposes. The hand drawing and visualization for AR is done through the screen of a mobile device, and all processes of the system are performed in real time.

Mobile Augmented Reality using scalable recognition and tracking

In this paper, a new mobile Augmented Reality (AR) framework which is scalable to the number of objects being augmented is proposed. The scalability is achieved by a visual word recognition module on the remote server and a mobile phone which detects, tracks, and augments target objects with the received information from the server. The server and the mobile phone are connected through a conventional Wi-Fi. In the experiment, it takes 0.2 seconds for the cold start of an AR service initiation on a 10k object database, which is fairly acceptable in a real-world AR application.

Real-time sensor-fusion based indoor localization for mobile Augmented Reality

In this paper we propose a sensor fusion based indoor localization method for mobile Augmented Reality (MAR). The aim of this research is to provide fine-tuning of the line feature based localization to accuracy of centimeter-level by exploring multi-modality of a mobile device. In order to match with line features from the captured scene and the given floor map, a line-based indoor scene analysis is proposed with Manhattan world assumption. An efficient pairwise line matching method using corresponding compass sensor data is presented to yield accurate localization and registration for MAR. Experimental results demonstrated that the proposed method is able to provide real-time performance and robustness in indoor environment.