Shingo Uchihashi

The beat spectrum: a new approach to rhythm analysis

We introduce the beat spectrum, a new method of automatically characterizing the rhythm and tempo of music and audio. The beat spectrum is a measure of acoustic self-similarity as a function of time lag. Highly structured or repetitive music will have strong beat spectrum peaks at the repetition times. This reveals both tempo and the relative strength of particular beats, and therefore can distinguish between different kinds of rhythms at the same tempo. We also introduce the beat spectrogram which graphically illustrates rhythm variation over time. Unlike previous approaches to tempo analysis, the beat spectrum does not depend on particular attributes such as energy or frequency, and thus will work for any music or audio in any genre. We present tempo estimation results which are accurate to within 1% for a variety of musical genres. This approach has a variety of applications, including music retrieval by similarity and automatically generating music videos. Anyone who has ever tapped a foot in time to music has performed rhythm analysis. Though simple for humans, this task is considerably more difficult to automate. We introduce a new measure of tempo analysis called the beat spectrum. This is a measure of acoustic self-similarity versus lag time, computed from a representation of spectrally similarity. Peaks in the beat spectrum correspond to major rhythmic components of the source audio. The repetition time of each component can be determined by the lag time of the corresponding peak, while the relative amplitudes of different peaks reflects the strengths of their corresponding rhythmic components. We also present the beat spectrogram which graphically illustrates rhythmic variation over time. The beat spectrogram is an image formed from the beat spectrum over successive windows. Strong rhythmic components are visible as bright bars in the beat spectrogram, making changes in tempo or time signature visible. In addition, a measure of audio novelty can be computed that measures how novel the source audio is at any time [2]. Instances when this measure is large correspond to significant audio changes. Periodic peaks correspond to rhythmic periodicity in the music. In the final section, we present various applications of the beat spectrum, including music retrieval by rhythmic similarity, an “automatic DJ” that can smoothly sequence music with similar tempos and automatic music video generation.

Summarizing video using a shot importance measure and a frame-packing algorithm

This paper presents methods of generating compact pictorial summarizations of video. By calculating a measure of shot importance video can be summarized by de-emphasizing or discarding less important information, such as repeated or common scenes. In contrast to other approaches that present keyframes for each shot, this measure allows summarization by presenting only the most important shots. Selected keyframes can also be resized depending on their relative importance. We present an efficient packing algorithm that constructs a pictorial representation from differently-sized keyframes. This results in a compact and visually pleasing summary reminiscent of a comic book.

Automatic index creation for handwritten notes

This paper describes a technique for automatically creating an index for handwritten notes captured as digital ink. No text recognition is performed. Rather, a dictionary of possible index terms is built by clustering groups of ink strokes corresponding roughly to words. Terms whose distribution varies significantly across note pages are selected for the index. An index page containing the index terms is created, and terms are hyper-linked back to their original location in the notes. Further, index terms occurring in a note page are highlighted to aid browsing.

Improvising camera control for capturing meeting activities using a floor plan

This paper describes camera control interfaces for capturing meetings and presentations into multimedia documents. While technologies are maturing to deliver multimedia documents over network, skilled human hands are still required to create the contents. We had dug into the problem and found that some portion of it derives from current camera control systems, which only provide interfaces for incremental navigations. Presets are provided for some systems to avoid cumbersome manipulations, but the difficulty to improvise controls remains untouched. We introduced an interface using floor plan for selecting an arbitrary area to be captured. We also conducted a study to compare our method with other two typical camera control interfaces. The results revealed that our method was significantly better than a typical joystick-metaphor interface. Although an interface with presets resulted superior for completing tasks in limited conditions, the participants judge the use of a floor plan to be equally good in respect to ease of camera navigation as they intended. They also indicated possible improvement for our interface to close in on the one with presets.

LightCollabo: distant collaboration support system for manufacturers

This paper introduces our LightCollabo system which is a distant collaboration tool designed for manufacturers. For distant collaboration between factory workers and members of a product design team, it is important to support interactive discussion over actual objects. LightCollabo enables remotely located people to share a view of actual objects and to annotate on the objects by using a camera and a projector whose optic axes and view angles are precisely aligned. Light-Collabo has been experimentally deployed in several locations in China and Japan and proven to be effective. Initial results of our use case study are also described.

Mixing remote locations using shared screen as virtual stage

This paper introduces a visual communication system that integrates images displayed on a shared screen and people in front of the screen using depth information obtained from a stereo camera. Conventional video conferencing systems convey PC screen contents and video of local and remote participants in different channels, making interactions among them difficult. The proposed system maps persons in the foreground onto the screen preserving relative position and size. A series of preliminary experiments is conducted to compare the performance of the proposed system against conventional systems.

Direct camera control for capturing meetings into multimedia documents

Currently, only experts can manage to capture activities in a conference room using multiple cameras because of the complexity of the task. We conducted interviews with experienced operators as well as video analysis on their outputs. Typical design of user interface for camera control was found out to be demanding users a lot of internal computations. A new interface for direct camera control is introduced to overcome the problem and tested in a real environment. The results well supported our usage model and proved our interface is effective.

Proposal and Evaluation of a Document Reader that Supports Pointing and Finger Bookmarking

Pointing and finger bookmarking effectively support reading from paper. However, current electronic media do not support these operations. Readers are discouraged to trace or point text with fingers on popular touchscreen tablet devices because the gestures may cause undesired view changes. Also, bookmarking with the current interface does not provide the ease of finger bookmarking. For solving the problems mentioned above, we proposed a document reader that provides seamless switching between pointing/tracing and touch operations, and integrate features that simulate finger bookmarking by using simple gestures. The results of two experiments (proof reading and cross reference reading between pages) show that participants performed the tasks faster with the proposed system than with the conventional touchscreen systems.

Image-based lighting adjustment method for browsing object images

In this paper, we describe an automatic lighting adjustment method for browsing object images. From a set of images of an object, taken under different lighting conditions, we generate two types of illuminated images: a textural image which eliminates unwanted specular reflections of the object, and a highlight image in which specularities of the object are highly preserved. Our user interface allows viewers to digitally zoom into any region of the image, and the lighting adjusted images are automatically generated for the selected region and displayed. Switching between the textural and the highlight images helps viewers to understand characteristics of the object surface.