Shaharyar Kamal

A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments

Recent advancements in depth video sensors technologies have made human activity recognition (HAR) realizable for elderly monitoring applications. Although conventional HAR utilizes RGB video sensors, HAR could be greatly improved with depth video sensors which produce depth or distance information. In this paper, a depth-based life logging HAR system is designed to recognize the daily activities of elderly people and turn these environments into an intelligent living space. Initially, a depth imaging sensor is used to capture depth silhouettes. Based on these silhouettes, human skeletons with joint information are produced which are further used for activity recognition and generating their life logs. The life-logging system is divided into two processes. Firstly, the training system includes data collection using a depth camera, feature extraction and training for each activity via Hidden Markov Models. Secondly, after training, the recognition engine starts to recognize the learned activities and produces life logs. The system was evaluated using life logging features against principal component and independent component features and achieved satisfactory recognition rates against the conventional approaches. Experiments conducted on the smart indoor activity datasets and the MSRDailyActivity3D dataset show promising results. The proposed system is directly applicable to any elderly monitoring system, such as monitoring healthcare problems for elderly people, or examining the indoor activities of people at home, office or hospital.

Real-time life logging via a depth silhouette-based human activity recognition system for smart home services

A real-time life logging system that provide monitoring, recording and recognition of daily human activities using video cameras offers life-care or health-care services at smart homes. Such a vision-based life logging system can provide continuous monitoring and recording of a residents daily activities from which one can obtain behavior patterns of daily life events and improve the quality of life especially for the elderly. This paper presents a real-time life logging system via depth imaging-based human activity recognition. A depth imaging device is utilized to obtain depth silhouettes of human activities. Then from the silhouettes, human body points information gets extracted and used in activity recognition, producing life logs. The system is composed of two key processes; one is training of the life logging system, and the other is running the trained life-logging system to record life logs. In the training process, the system includes the data collection from a depth camera, extraction of body points features from each depth silhouette and finally training of the activity recognizer (i.e., Hidden Markov Models). Then, after training, one can run the trained system which recognizes learned activities and store life logs in real-time. The proposed approach is evaluated against the life logging system that uses the conventional principal components (PC) and Radon transform features of depth silhouettes and achieves superior recognition rate. Real-time experimental results show the feasibility and functionality of the implemented system which could be used to generate life logs of human activities at smart homes.

Robust human activity recognition from depth video using spatiotemporal multi-fused features

Abstract The recently developed depth imaging technologies have provided new directions for human activity recognition (HAR) without attaching optical markers or any other motion sensors to human body parts. In this paper, we propose novel multi-fused features for online human activity recognition (HAR) system that recognizes human activities from continuous sequences of depth map. The proposed online HAR system segments human depth silhouettes using temporal human motion information as well as it obtains human skeleton joints using spatiotemporal human body information. Then, it extracts the spatiotemporal multi-fused features that concatenate four skeleton joint features and one body shape feature. Skeleton joint features include the torso-based distance feature (DT), the key joint-based distance feature (DK), the spatiotemporal magnitude feature (M) and the spatiotemporal directional angle feature (θ). The body shape feature called HOG-DDS represents the projections of the depth differential silhouettes (DDS) between two consecutive frames onto three orthogonal planes by the histogram of oriented gradients (HOG) format. The size of the proposed spatiotemporal multi-fused feature is reduced by a code vector in the code book which is generated by vector quantization method. Then, it trains the hidden Markov model (HMM) with the code vectors of the multi-fused features and recognizes the segmented human activity by the forward spotting scheme using the trained HMM-based human activity classifiers. The experimental results on three challenging depth video datasets such as IM-DailyDepthActivity, MSRAction3D and MSRDailyActivity3D demonstrate that the proposed online HAR method using the proposed multi-fused features outperforms the state-of-the-art HAR methods in terms of recognition accuracy.

Dense RGB-D Map-Based Human Tracking and Activity Recognition using Skin Joints Features and Self-Organizing Map

This paper addresses the issues of 3D human activity detection, tracking and recognition from RGB-D video sequences using a feature structured framework. During human tracking and activity recognition, initially, dense depth images are captured using depth camera. In order to track human silhouettes, we considered spatial/temporal continuity, constraints of human motion information and compute centroids of each activity based on chain coding mechanism and centroids point extraction. In body skin joints features, we estimate human body skin color to identify human body parts (i.e., head, hands, and feet) likely to extract joint points information. These joints points are further processed as feature extraction process including distance position features and centroid distance features. Lastly, self-organized maps are used to recognize different activities. Experimental results demonstrate that the proposed method is reliable and efficient in recognizing human poses at different realistic scenes. The proposed system should be applicable to different consumer application systems such as healthcare system, video surveillance system and indoor monitoring systems which track and recognize different activities of multiple users.

Depth map-based human activity tracking and recognition using body joints features and Self-Organized Map

In this paper, we implement human activity tracking and recognition system utilizing body joints features using depth maps. During HAR settings, depth maps are processed to track human silhouettes by considering temporal continuity constraints of human motion information and compute centroids for each activity based on contour generation. In body joints features, depth silhouettes are computed first through geodesic distance to identify anatomical landmarks which produce joint points information from specific body parts. Then, body joints are processed to produce centroid distance features and key joints distance features. Finally, Self-Organized Map (SOM) is employed to train and recognize different human activities from the features. Experimental results show that body joints features achieved high recognition rate over the conventional features. The proposed system should be applicable as e-healthcare systems for monitoring elderly people, surveillance systems for observing pedestrian traffic areas and indoor environment systems which recognize activities of multiple users.

Shape and Motion Features Approach for Activity Tracking and Recognition from Kinect Video Camera

Recent development in depth sensors opens up new challenging task in the field of computer vision research areas, including human-computer interaction, computer games and surveillance systems. This paper addresses shape and motion features approach to observe, track and recognize human silhouettes using a sequence of RGB-D images. Under our proposed activity recognition framework, the required procedure includes: detecting human silhouettes from the image sequence, we remove noisy effects from background and track human silhouettes using temporal continuity constraints of human motion information for each activity, extracting the shape and motion features to identify richer motion information and then these features are clustered and fed into Hidden Markov Model (HMM) to train, model and recognize human activities based on transition and emission probabilities values. During experimental results, we demonstrate this approach on two challenging depth video datasets: one based on our own annotated database and other based on public database (i.e., MSRAction3D). Our approach shows significant recognition results over the state of the art algorithms.

Family of Nyquist-I Pulses to Enhance Orthogonal Frequency Division Multiplexing System Performance

ABSTRACT A family of Nyquist-I pulses called sinc parametric linear combination pulse (SPLCP) is proposed. It is characterized by two novel design parameters that provide additional degrees of freedom to minimize the intercarrier interference (ICI) power due to frequency offset. Moreover, it reduces the high peak-to-average power ratio (PAPR) value in orthogonal frequency division multiplexing (OFDM) systems. Several Nyquist-I pulses were recently proposed to address the subject of high sensitivity to frequency offset and high PAPR in OFDM-based transmissions. In this paper, we investigate the performance of SPLCP in terms of ICI power, signal-to-interference ratio (SIR) power, bit error rate (BER), and PAPR. We additionally examine the behaviour of SPLCP with new design parameters for a certain roll-off factor, α. We compare the performance of SPLCP with other well-known pulses. Theoretical and simulation results show that the proposed SPLCP outperforms other existing pulses in terms of ICI power, SIR power, BER, and PAPR.

Depth Images-based Human Detection, Tracking and Activity Recognition Using Spatiotemporal Features and Modified HMM

Human activity recognition using depth information is an emerging and challenging technology in computer vision due to its considerable attention by many practical applications such as smart home/office system, personal health care and 3D video games. This paper presents a novel framework of 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. To detect human silhouette, raw depth data is examined to extract human silhouette by considering spatial continuity and constraints of human motion information. While, frame differentiation is used to track human movements. Features extraction mechanism consists of spatial depth shape features and temporal joints features are used to improve classification performance. Both of these features are fused together to recognize different activities using the modified hidden Markov model (M-HMM). The proposed approach is evaluated on two challenging depth video datasets. Moreover, our system has significant abilities to handle subjects body parts rotation and body parts missing which provide major contributions in human activity recognition.

Human daily activity recognition with joints plus body features representation using Kinect sensor

Human activity recognition has been studied actively from decades using a sequence of 2D images/video. With the development of depth sensors, new opportunities arise to improve and advance this field. This study presents a depth imaging activity recognition system to monitor and recognize daily activities of the human without attaching optical markers or motion sensors. In this paper, we proposed a new feature representation and extraction method using a sequence of depth silhouettes. Particularly, we first extract the depth silhouette by removing background from noisy effects and then extract the joints plus body features as skin color detection from joint information and multi-view body shape from depth silhouettes (i.e., front and side views). We combine the joints plus body shape features to make feature vector. These features have two nice properties including invariant with respect to body shape or size and insensitive to small noise. Self-Organized Map (SOM) is then used to train and test the feature vectors. Experimental results regarding our proposed human activity dataset and publically available dataset demonstrate that our feature extraction method is more promising and outperforms the state-of-the-art feature extraction methods.

Depth silhouettes context: A new robust feature for human tracking and activity recognition based on embedded HMMs

Activity and action detection, tracking and recognition are very demanding research area in computer vision and human computer interaction. In this paper, a video-based novel approach for human activity recognition is presented using robust hybrid features and embedded Hidden Markov Models. In the proposed HAR framework, depth maps are analyzed by temporal motion identification method to segment human silhouettes from noisy background and compute depth silhouette area for each activity to track human movements in a scene. Several representative features, including invariant, depth sequential silhouettes and spatiotemporal body joints features were fused together to explore gradient orientation change, intensity differentiation, temporal variation and local motion of specific body parts. Then, these features are processed by the dynamics of their respective class and learned, trained and recognized with specific embedded HMM having active feature values. Our experiments on two depth datasets demonstrate that the proposed features are efficient and robust over the state of the arts features for human activity recognition especially when there are similar postures of different activities.