Jingyuan Cheng

From Smart Clothing to Smart Table Cloth: Design and Implementation of a Large Scale, Textile Pressure Matrix Sensor

We describe the design and implementation of an unobtrusive, cheap, large scale, pressure sensor matrix that can be used for a variety of applications ranging from smart clothing, through smart furniture, to an intelligent table cloth or carpet. The specific functionality and with it most of the complexity lies in the electronics and the processing software. We propose a scalable, modular architecture for such electronics, describe a prototype implementation, and present the results of its application to three different scenarios.

Designing Sensitive Wearable Capacitive Sensors for Activity Recognition

We investigate the design space of flexible, textile capacitive sensors for applications in human activity recognition. In a previous paper, we showed that conductive textile patches can be used to measure capacitance of the human body and could reveal information about a broad range of activities. In this paper, we systematically investigate how different design parameters such as electrode size, electric field frequency, and the concrete analog circuit design influence sensor performance. To this end, we combine FEM electric field simulations, circuit analysis, and measurements. We illustrate the performance of sensor systems that implemented according to the design guidelines that we derived. Results from four typical activity recognition scenarios were considered, including heart rate and breathing rate monitoring, hand gesture recognition, swallowing monitoring, and gait analysis.

Smart table surface: A novel approach to pervasive dining monitoring

We present a novel sensor system for the support of nutrition monitoring. The system is based on smart table cloth equipped with a fine grained pressure textile matrix and a weight sensitive tablet. Unlike many other nutrition monitoring approaches, our system is unobtrusive, non privacy invasive and easily deployable in every day life. It allows the spotting and recognition of food intake related actions, such as cutting, scooping, stirring, etc., the identification of the plate/container on which the action is executed, and the tracking of the weight change in the containers. In other words, we can determine how many pieces are cut on the main dish plate, how many are taken from the side dish, how many sips are taken from the drink, how fast the food is being consumed and how much weight is taken overall. In addition, the distinction between different eating actions, such as cutting, scooping, poking, provides clues to the type of food taken and the way the meal is consumed. We have evaluated our system on 40 meals (5 subjects) in a real life living environment: for seven eating related actions (cutting, scooping, stirring, etc.), resulting in above 90% average recognition rate for person dependent cases, and spotting each action out of continuous data streams (average F1 score 87%).

Smart Textiles: From Niche to Mainstream

Current technology supports only special-purpose, low-volume textiles, garments, and electronics. Moreover, the textile, electronic, and software industries have different product cycles, cultures, and price models, creating scores of practical problems for smart textiles. Mass producing smart cloth will require decoupling the textile production from concrete sensing apps and moving the complexity to generic electronics and software--creating wearable sensing as an app.

Smart-surface

In this paper we present textile-based surface pressure mapping as a novel, unobtrusive information source for activity recognition. The concept is motivated by the observation that the vast majority of human activities are associated with certain types of surface contact (walking, running, etc. on the floor; sitting on a chair/sofa; eating, writing, etc. at a table; exercising on a fitness mat, and many others). A key hypothesis which we validate in this paper is: by analysing subtle features of such interaction, various complex activities, often ones that are difficult to distinguish using other unobtrusive sensors, can be well recognised. A core contribution of our work is a sensing and recognition system based on cheap, easy-to-produce textile components. These components can be integrated into matrices with tens of thousands of elements, a spatial pitch as fine as 1źcm2, temporal granularity of up to 40 Hz and pressure dynamic range from 0.25?105 to 5?105 źPa. We present the evaluation of the concept and the technology in five scenarios, through matrix monitoring driver motions at a car seat (32?32 sensors on 32?32źcm2), a Smart-YogaMat (80?80 sensors on 80?80źcm2) detecting and counting exercises, to a Smart-Tablecloth (30?42 sensors on 30?42źcm2) recognising various types of food being eaten.

Never skip leg day: A novel wearable approach to monitoring gym leg exercises

We present a wearable textile sensor system for monitoring muscle activity, leveraging surface pressure changes between the skin and an elastic sport support band. The sensor is based on an 8×16 element fabric resistive pressure sensing matrix of 1cm spatial resolution, which can be read out with 50fps refresh rate. We evaluate the system by monitoring leg muscles during leg workouts in a gym out of the lab. The sensor covers the lower part of quadriceps of the user. The shape and movement of the two major muscles (vastus lateralis and medialis) are visible from the data during various exercises. The system registers the activity of the user for every second, including which machine he/she is using, walking, relaxing and adjusting the machines; it also counts the repetitions from each set and evaluate the force consistency which is related to the workout quality. 6 people participated in the experiment of overall 24 leg workout sessions. Each session includes cross-trainer warm-up and cool-down, 3 different leg machines, 4 sets on each machine. Plus relaxing, adjusting machines, and walking, we perform activity recognition and quality evaluation through 2-dimensional mapping and the time sequence of the average force. We have reached 81.7% average recognition accuracy on a 2s sliding window basis, 93.3% on an event basis, and 85.6% spotting F1-score. We further demonstrate how to evaluate the workout quality through counting, force pattern variation and consistency.

Hands-free gesture control with a capacitive textile neckband

We present a novel sensing modality for hands-free gesture controlled user interfaces, based on active capacitive sensing. Four capacitive electrodes are integrated into a textile neckband, allowing continuous unobtrusive head movement monitoring. We explore the capability of the proposed system for recognising head gestures and postures. A study involving 12 subjects was carried out, recording data from 15 head gestures and 19 different postures. We present a quantitative evaluation based on this dataset, achieving an overall accuracy of 79.1% for head gesture recognition and 40.4% for distinguishing between head postures (69.9% when merging the most adjacent positions), respectively. These results indicate that our approach is promising for hands-free control interfaces. An example application scenario of this technology is the control of an electric wheelchair for people with motor impairments, where recognised gestures or postures can be mapped to control commands.

Recognizing Subtle User Activities and Person Identity with Cheap Resistive Pressure Sensing Carpet

We demonstrate through a pressure sensor matrix, that weight distribution on feet is influenced by body posture. A small cheap carpet equipped with low precision pressure sensor matrix is already sufficient to detect subtle activities and identity of the person on the carpet. By a 0.4 m2 matrix of 32 × 32, 12 bit pressure sensors, we achieve 78.7% accuracy for 11 test subjects performing 7 subtle activities (open 7 different drawers or cabinet doors) and 88.6% accuracy in recognizing who has performed the activities. We thus see the potential of using a single carpet as a unified approach in houses to detect how inhabitants interact with the furniture without attaching different sensors onto each single furniture.

SimpleSkin: towards multipurpose smart garments

Smart textiles have been researched in the lab over the last 20 years. However, the gap between research and available mass-market products is huge. We identify challenges that are the core reasons for this gap. To tackle these challenges, we present our work towards a multipurpose smart textile with different sensing modalities. It separates the concern of developing textiles, electronics, infrastructure, and applications. Furthermore, it uses a similar application model as current smart-phones allowing developers to create applications for the smart textiles. We believe that this approach is capable of moving smart textiles from niche to mainstream.

On the tip of my tongue: a non-invasive pressure-based tongue interface

Mobile and wearable devices became pervasive in daily life. The dominant input techniques for mobile and wearable technology are touch and speech. Both approaches are not appropriate in all settings. Therefore, we propose a novel interface that is controlled through the tongue. It is based on an array of textile pressure sensors attached to the users cheek. It can be easily integrated into helmets or face masks in a non-invasive way. In an initial study, we investigate gestures for tongue-based interface. Six participants repeatedly performed five simple tongue gestures. We show that gestures can be recognized with 98% accuracy. Based on feedback from participants, we discuss potential use cases and provide an outlook on further improvement of the system.