Yunhee Shin

Vision based interface system for hands free control of an intelligent wheelchair

BackgroundDue to the shift of the age structure in todays populations, the necessities for developing the devices or technologies to support them have been increasing. Traditionally, the wheelchair, including powered and manual ones, is the most popular and important rehabilitation/assistive device for the disabled and the elderly. However, it is still highly restricted especially for severely disabled. As a solution to this, the Intelligent Wheelchairs (IWs) have received considerable attention as mobility aids. The purpose of this work is to develop the IW interface for providing more convenient and efficient interface to the people the disability in their limbs.MethodsThis paper proposes an intelligent wheelchair (IW) control system for the people with various disabilities. To facilitate a wide variety of user abilities, the proposed system involves the use of face-inclination and mouth-shape information, where the direction of an IW is determined by the inclination of the users face, while proceeding and stopping are determined by the shapes of the users mouth. Our system is composed of electric powered wheelchair, data acquisition board, ultrasonic/infra-red sensors, a PC camera, and vision system. Then the vision system to analyze users gestures is performed by three stages: detector, recognizer, and converter. In the detector, the facial region of the intended user is first obtained using Adaboost, thereafter the mouth region is detected based on edge information. The extracted features are sent to the recognizer, which recognizes the face inclination and mouth shape using statistical analysis and K-means clustering, respectively. These recognition results are then delivered to the converter to control the wheelchair.Result & conclusionThe advantages of the proposed system include 1) accurate recognition of users intention with minimal user motion and 2) robustness to a cluttered background and the time-varying illumination. To prove these advantages, the proposed system was tested with 34 users in indoor and outdoor environments and the results were compared with those of other systems, then the results showed that the proposed system has superior performance to other systems in terms of speed and accuracy. Therefore, it is proved that proposed system provided a friendly and convenient interface to the severely disabled people.

Automatic textile image annotation by predicting emotional concepts from visual features

This paper presents an emotion prediction system that can automatically predict certain human emotional concepts from a given textile. The main application motivating this study is textile image annotation, which has recently rapidly expanded in relation to the Web. In the proposed method, color and pattern are used as cues to predict the emotional semantics associated with an image, where these features are extracted using a color quantization and a multi-level wavelet transform, respectively. The extracted features are then applied to three representative classifiers: K-means clustering, Naive Bayesian, and a multi-layered perceptron (MLP), all of which are widely used in data mining. When evaluating the proposed emotion prediction method using 3600 textile images, the MLP produces the best performance. Thereafter, the proposed MLP-based method is compared with other methods that only use color or pattern, and the proposed method shows the best performance with an accuracy of above 92%. Therefore, the results confirm that the proposed method can be effectively applied to the commercial textile industry and image retrieval.

Intelligent wheelchair (IW) interface using face and mouth recognition

between the user and the wheelchair. To facilitate a wide variety of user abilities, the proposed system uses faceinclination and mouth-shape information as users intention, where the direction of an IW is determined by the inclination of the users face, while proceeding and stopping are determined by the shape of the users mouth. This mechanism requires minimal motion, thereby making the system more comfortable and adaptable for the severely disabled. Furthermore, to fully guarantee users safety, the 10 range-sensors are used to detect obstacles in environment and avoid them. To assess the effectiveness of the proposed IW, it was tested with 34 users and the results show that it can provide a user unable to drive a standard joystick with friendly and convenient system

Emotion-based Textile Indexing usinig Neural Networks

This paper proposes a neural network based approach for emotion-based textile indexing. Generally, the human emotion can be affected by some physical features such as color, texture, pattern, and so on. In the previous work, we investigated the correlation between the human emotion and color or texture. Here, we aim at investigating the correlation between the emotion and pattern, and developing the textile indexing system using the pattern information. Therefore, the survey is first conducted to investigate the correlation between the emotion and the pattern. The result shows that a human emotion is deeply affected by the certain pattern. Based on that result, an automatic indexing system is developed. The proposed system is composed of feature extraction and classification. To describe the pattern information in the textiles, the wavelet transform is used. And the neural network is used as the classifier. To assess the validity of the proposed method, it was applied to recognize the human emotions in 100 textiles, and then our system produced the accuracy of 90%. This result confirmed that our system has the potential to be applied for various applications such as textile industry and e-business.

EBIR: Emotion-based image retrieval

For content-based image retrieval, human emotions as well as object information provide important clue to search images. Accordingly, this paper presents a new retrieval system that indexes images using human emotion and searches them. Our system was tested with 1,300 textile images, then the results demonstrate the effectiveness of our system.

Welfare interface using multiple facial features tracking

We propose a welfare interface using multiple facial features tracking, which can efficiently implement various mouse operations. The proposed system consist of five modules: face detection, eye detection, mouth detection, facial features tracking, and mouse control. The facial region is first obtained using skin-color model and connected-component analysis (CCs). Thereafter the eye regions are localized using neural network (NN)-based texture classifier that discriminates the facial region into eye class and non-eye class, and then mouth region is localized using edge detector. Once eye and mouth regions are localized, they are continuously and correctly tracking by mean-shift algorithm and template matching, respectively. Based on the tracking results, mouse operations such as movement or click are implemented. To assess the validity of the proposed system, it was applied to the interface system for web browser and was tested on a group of 25 users. The results show that our system have the accuracy of 99% and process more than 12 frames/sec on PC for the 320(240 size input image, as such it can supply a user-friendly and convenient access to a computer in real-time operation.

Emotion recognition using color and pattern in textile images

In this paper, a novel method is proposed using color and pattern information for recognizing some emotions included in a textile. Here we use 10 Kobayashi emotion keywords. Our method is composed of feature extraction and classification. For accurate emotion recognition, both color and pattern are extracted from a textile. Then the representative color prototypes of a textile are extracted using color quantization method and pattern is described by wavelet transform followed by some statistical terms. The extracted features are given to the neural network (NN)-based classifiers. The advantages of our method include the following: (1) it is a generalized method to accurately recognize emotions in textile images which used in various application domains; (2) it is a fully automatic method with no manual interaction. To prove these advantages, the experiments are performed on 389 textiles obtained from various application domains such as interior, fashion, and artificial ones. Our method shows the precision of 100% and the recall of 99%, regardless of the specific domain.

shooting a bird: game system using facial feature for the handicapped people

This paper presents a novel computer game system that controls a game using only the movement of humans facial features. Our system is specially designated for the handicapped people with severe disabilities and the people without experience of using the computer. Using a usual PC camera, the proposed game system detects the users eye movement and mouse movement, and then interprets the communication intent to play a game. The game system is tested with 42 numbers of people, and then the result shows that our game system should be efficiently and effectively used as the interface for the disabled people.

Finding Relationships between Human Affects and Colors Using SVD and pLSA

In this paper, a new method is presented to automatically find relationships between human affects and colors. For this, the probabilistic latent semantic model analysis (pLSA) and singular value decomposition (SVD) is applied. The proposed method is composed of three modules: feature extraction, feature transform and pLSA training. We first segment the image using mean-shift clustering, then extract color compositions by analyzing the colors from one region and its adjacent regions. Next, for the occurrence matrix, the SVD and pLSA are used. Using SVD, the occurrence matrix is decomposed into rank and null space matrix, where the null space is discarded and only the space corresponding to the singular values is used for further processing. For the reconstructed matrix, the pLSA is applied to obtain the correlation between affective classes and color compositions. To assess the effectiveness of the proposed system, it was applied to index the images using human affects. Then the results showed the effectiveness of the proposed method.

Affect-Based Retrieval of Landscape Images Using Probabilistic Affective Model

We consider the problem of ranking the web image search using human affects. For this, a Probabilistic Affective Model (PAM) is presented for predicting the affects from color compositions (CCs) of images, then the retrieval system is developed using them. The PAM first segments an image into seed regions, then extracts CCs among seed regions and their neighbors, finally infer the numerical ratings of certain affects by comparing the extracted CCs with pre-defined human-devised color triplets. The performance of the proposed system has been studied at an online demonstration site where 52 users search 16,276 landscape images using affects, then the results demonstrated its effectiveness in affect-based image annotation and retrieval.