Masanori Takehara

Improving Service Processes Based on Visualization of Human-Behavior and POS Data: A Case Study in a Japanese Restaurant

A case study of service process improvement based on visualization of human-behavior and POS (point-of-sales) data in a Japanese restaurant is described. We developed a human-behavior sensing and visualization suite for supporting managers and employees in actual service fields to understand and improve their service processes by visualizing of both behavior and POS data. We had an experiment using the suite in the restaurant and confirmed that managers and employees were able to understand their ordinary processes, make plans for improving their processes by using the suite, and improve their processes which are observed as the increase of the stay ratio of a waiting staff in dining areas and the number of additional orders. An overview of the suite and experiment results is described.

Measuring and evaluating real service operations with human-behavior sensing: A case study in a Japanese cuisine restaurant

A case study of human-behavior sensing in a Japanese cuisine restaurant is described. The aim of this study is to create a suite of human-behavior sensing that facilitate people who are in real service fields to understand current operations, and make plans for improving operations. We created a prototype of the suite, which consists of several component technologies such as pedestrian dead-reckoning (PDR), sensor data fusion (SDF), service operation estimation (SOE), service field simulator (SFS), and data visualization. We had anexperiment in a Japanese cuisine restaurant, and found that the suite assisted managers and waiting staff to understand current state of operations, and to make a plan for improving operations. An overview of the suite, and results of experiment are described.

Audio-visual voice conversion using noise-robust features

Voice Conversion (VC) is a technique to convert speech data of source speaker into ones of target speaker. VC has been investigated and statistical VC is used for various purposes. Conventional VC uses acoustic features, however, the audio-only VC has suffered from the degradation in noisy or real environments. This paper proposes an AudioVisual VC (AVVC) method using not only audio features but also visual information, i.e. lip images. Eigenlip feature is employed in our scheme as visual feature. We also propose a feature selection approach for audio-visual features. Experiments were conducted to evaluate our AVVC scheme comparing with audio-only VC, using noisy data. The results show that AVVC can improve the performance even in noisy environments, by properly selecting audio and visual parameters. It is also found that visual VC is also successful. Furthermore, it is observed that visual dynamic features are more effective than visual static information.

Improvement of utterance clustering by using employees' sound and area data

In this paper, we propose to use staying area data toward the estimation of serving time for customers. To classify utterances enables us to estimate conversation types between speakers. However, its performance becomes lower in real environments. We propose a method using area data with sound data to solve this problem. We also propose a method to estimate the conversation types using the decision trees. They were tested with the data recorded in a Japanese restaurant. In the experiment to classify utterances, the proposed method performed better than the method using only sound data. In the experiment to estimate the conversation types, we succeeded to recover 70% of the mis-classified conversations using both of sound and area data.

Analysis of customer communication by employee in restaurant and lead time estimation

Human behavior sensing and their analysis are great role to improve service quality and education of employees. This paper shows novel frameworks of detection of customer communication and lead time estimation(LTE) by using multi-sensored data, sound data and accounting data in the restaurant. They are useful for management about work environments and problems for employees. Lead time from order to delivery shows the quality of the service for customers. We found sound data of an employees speech is useful for these techniques by speech ratio smoothing and POS sound detection.

Measurement and analysis of speech data toward improving service in restaurant

In this paper, we introduce human behavior sensing and data collection tequniques in a real environment. In order to improve workers skill and service quality, it is desired to analyse and visualize workers behavior scientifically. We acquired multi-sensored data and speech data of wait staffs in a restaurant for several months. Our goal is to combine these data, to analyse objectively and finally to improve their behavior and service. The speech data is useful to detect their speech events and spoken contents. To collect speech data in real-environment is a difficult task because of recording environment and load to the subjects. This paper focuses on collection of speech and discusses the issue of its measurement and analysis. We also apply VAD(Voice activity detection) for speech data and calculate their speech ratio. The speech ratio has a certain inclination related to their operations. The change of speech ratio expresses the efficiency of QC(Quality control) activity.

Multi-modal service operation estimation using DNN-based acoustic bag-of-features

In service engineering it is important to estimate when and what a worker did, because they include crucial evidences to improve service quality and working environments. For Service Operation Estimation (SOE), acoustic information is one of useful and key modalities; particularly environmental or background sounds include effective cues. This paper focuses on two aspects: (1) extracting powerful and robust acoustic features by using stacked-denoising-autoencoder and bag-of-feature techniques, and (2) investigating a multi-modal SOE scheme by combining the audio features and the other sensor data as well as non-sensor information. We conducted evaluation experiments using multi-modal data recorded in a restaurant. We improved SOE performance in comparison to conventional acoustic features, and effectiveness of our multimodal SOE scheme is also clarified.