MGait: Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors
Sizhe An, Yigit Tuncel, Toygun Basaklar, Gokul Krishna Krishnakumar, Ganapati Bhat, Umit Ogras
1111
MGait : Model-Based Gait Analysis Using Wearable Bend andInertial Sensors
SIZHE AN,
University of Wisconsin-Madison, USA
YIGIT TUNCEL,
University of Wisconsin-Madison, USA
TOYGUN BASAKLAR,
University of Wisconsin-Madison, USA
GOKUL KRISHNA KRISHNAKUMAR,
Arizona State University, USA
GANAPATI BHAT,
Washington State University, USA
UMIT OGRAS,
University of Wisconsin-Madison, USAMovement disorders, such as Parkinson’s disease, affect more than 10 million people worldwide. Gait analysisis a critical step in the diagnosis and rehabilitation of these disorders. Specifically, step length provides valuableinsights into the gait quality and rehabilitation process. However, traditional approaches for estimatingstep length are not suitable for continuous daily monitoring since they rely on special mats and clinicalenvironments. To address this limitation, we present a novel and practical step-length estimation techniqueusing low-power wearable bend and inertial sensors. Experimental results show that the proposed modelestimates step length with 5.49% mean absolute percentage error and provides accurate real-time feedback tothe user.Additional Key Words and Phrases: Gait analysis, step length estimation, wearable devices, bend sensor, IMU,health monitoring, low-power design, online estimation.
ACM Reference Format:
Sizhe An, Yigit Tuncel, Toygun Basaklar, Gokul Krishna Krishnakumar, Ganapati Bhat, and Umit Ogras. 2021.
MGait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors.
ACM Trans. Internet Things
Movement disorders are one of the leading causes of functional disability in the elderly population.More than 10 million people worldwide suffer from movement disorders, such as Parkinson’sdisease (PD); more than 900,000 patients are expected to be diagnosed with PD in the United Statesby the year 2020 [15]. Gait impairment and instability are among the most common symptoms inmovement disorder patients and stroke survivors [11]. Therefore, gait function analysis plays animportant role in treatment and rehabilitation.Gait function analysis provides valuable insight into a patient’s symptoms and rehabilitation, byevaluating metrics, such as step length, stride length, and gait velocity [2, 22]. In particular, step
Authors’ addresses: Sizhe An, University of Wisconsin-Madison, 1415 Engineering Dr, Madison, Wisconsin, USA, 53706,[email protected]; Yigit Tuncel, University of Wisconsin-Madison, 1415 Engineering Dr, Madison, Wisconsin, USA, 53706,[email protected]; Toygun Basaklar, University of Wisconsin-Madison, 1415 Engineering Dr, Madison, Wisconsin, USA,53706, [email protected]; Gokul Krishna Krishnakumar, Arizona State University, 781 S Terrace Rd, Tempe, Arizona, USA,85287, [email protected]; Ganapati Bhat, Washington State University, 355 NE Spokane St, Pullman, Washington, USA,99164, [email protected]; Umit Ogras, University of Wisconsin-Madison, 1415 Engineering Dr, Madison, Wisconsin,USA, 53706, [email protected] to make digital or hard copies of all or part of this work for personal or classroom use is granted without feeprovided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice andthe full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requiresprior specific permission and/or a fee. Request permissions from [email protected].© 2021 Association for Computing Machinery.2577-6207/2021/1-ART111 $15.00https://doi.org/10.1111/11.11 ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021. a r X i v : . [ ee ss . SP ] F e b IMU Sensor & MCU B end S en s o r d l l l l Hip Angle (a) (b)
Knee Angle
Fig. 1. (a) Wearable setup used for gait analysis (b) Magnified view of the sensors length is used to analyze the symmetry of gait. While step length remains nearly constant for ahealthy person, it varies as the left and right feet alternate for a patient with gait asymmetry. Thisdifference is used as an important feature to evaluate the symmetry of gait and monitor the progressof rehabilitation [22]. Several clinical studies use the GAITRite system [5], a pressure-sensitivewalking mat, to analyze the gait parameters [8, 16, 29]. While GAITRite can provide a 98%-99%accuracy, it cannot be used to continuously monitor the patient’s gait after they leave the clinic. Toaddress this limitation, recent work employs wearable sensors for gait analysis [2, 28, 30]. Mostof these studies mount a number of inertial motion units (IMU), which typically incorporate a3-axis accelerometer and a 3-axis gyroscope, on the leg to collect acceleration and rotation datawhen a person is walking. However, these approaches need to employ either high-cost (over a fewthousand dollars) [13] or a large number of sensors [2, 27], which is not practical for daily use.Therefore, there is a critical need for simple and intuitive models to estimate gait parameters usingrelatively few sensors.This paper presents a wearable cyber-physical system (CPS), called
MGait , that combines phys-iological sensors, energy-efficient local processing, and real-time user feedback. We implement
MGait on a knee sleeve, as shown in Figure 1 and demonstrate that it achieves the following goals:
Functionality:
Enable continuous daily monitoring and real-time feedback with 95% averageaccuracy without relying on any clinical environment and experimental infrastructure.
Power-performance:
Achieve real-time operation and mW-range operation using only one pairof IMUs and low-power bend sensors for the first time in literature .We meet these goals with two sensors per leg and commercial-off-the-shelf components thatcan be integrated with a total cost of less than $160, in contrast to techniques that use sensorsand equipment with over one order of magnitude higher cost. A stretchable bend sensor mountedon the back of the knee measures the knee angle, while an IMU sensor above the knee measuresthe swing of the hip on each leg, as shown in Figure 1. Data from the bend and IMU sensors areprocessed locally in real-time to obtain these angles. We propose a novel biomechanical model andderive a closed-form expression that computes the step length using the angle data. Using only ourclosed-form expression and sensor data leads to over 10% estimation error due to sensor offset andmeasurement noise. We reduce this error significantly using a novel two-step nonlinear regressiontechnique. Thorough experimental evaluations with seven subjects show that the proposed approachachieves a 5% mean absolute percentage error (MAPE) with this optimization. In addition, we also
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:3 present a recursive least square estimation technique that can tune the proposed model in the field.Finally, we employ the proposed model to derive other commonly used clinical metrics, includinggait velocity and stride length. The proposed system can be used to provide real-time feedback ifany outlier behavior is observed, as illustrated in Section 5.3. We will release our empirical datasetto the public to enable further research in the gait analysis domain. In addition to the empiricaldataset, we also implement a novel method to generate synthetic data. Specifically, we train aConditional Generative Adversarial Nets (CGAN) using the empirical data to generate additionalsynthetic data. The synthetic data generation methodology will allow researchers to minimize thetime and labor-consuming step of data collection. The data generated using the CGAN closelyfollows the patterns of the empirical with a mean deviation of 1 ◦ .In summary, this paper makes the following major contributions: • An energy-efficient wearable cyber-physical system for real-time gait analysis and step lengthestimation. • A novel analytical modeling approach and a closed-form expression for estimating step length,stride length, and gait velocity. • A novel method to generate synthetic gait data using conditional generative adversarial nets. • A thorough experimental evaluation, including offline and online estimation, with seven subjectsand a new dataset, will be released to the public.In the following sections, Section 2 reviews the related work. Section 3 introduces the gaitanalysis and the proposed approach. Section 4 and 5 present angle and step length estimationapproaches, respectively. Finally, Section 6 presents the experimental results, Section 7 proposes atechnique to generate synthetic data based on CGAN, and Section 8 concludes the paper.
Gait analysis is vital for enabling a variety of CPS applications, such as feedback for patients undertreatment, prediction of possible movement disorders [6, 20], athlete assessment, fall detection [21],and numerous other augmented and virtual reality applications. For this reason recent work hasfocused on using wearable sensors to estimate step length, which is a very important parameter ingait analysis [2, 13, 19, 27, 28, 30].Table 1 summarizes state-of-the-art step length estimation techniques that are closest to ourwork. Wang et al. propose a biomechanical model that takes knee bending into account and usesdata obtained from a total of four low power IMUs placed on both legs to estimate step length andgait asymmetry [28]. The proposed double pendulum model estimates the step length with a 5.5 cmroot mean square error (RMSE). This model considers the hip as a single hinge joint but ignoringthe displacement of the hip joints in the direction of the movement thus, results in errors in steplength estimation. Wu et al. place two IMU dev-kits on the ankles of the subjects to estimate steplength, using an inverted pendulum model with 3.69% MAPE [30]. However, the error reportedin their study is for total walking distance, not for individual step lengths. Their model does notconsider knee and hip joints, which can result in low step length estimation accuracy. Since themodel assumes a single segment for each leg, it is able to calculate the step length using only twoIMUs. Consequently, the power consumption of this approach is lower, compared to other morecomplicated approaches. Their approach also requires a Kinect V2 setup to estimate leg lengthsof the subjects in their study. We do not include it in the sensor type, since it does not contributeto the model calculation. Pepa et al. utilize accelerometers in smartphones and develop an app tocollect motion data. They also use an inverted pendulum model and estimate step length with lessthan 10% MAPE [19]. None of these studies provide stride length and gait velocity estimates, whichare two other main parameters used for gait function analysis [22].
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
A recent work addresses the limitations in the previous studies, by using four high-cost ultra-wideband (UWB) distance sensors and four IMUs on the back and front of both feet [2]. It employsa geometrical trapezoid distance model and estimates step length with 4.1 cm RMSE and 4.75%MAPE. However, the high processing requirements of the data from eight sensors increase thecomplexity and the power consumption of the system. Another recent study places two IMUs onthe feet, two IMUs on the shanks, two IMUs on the thighs, and one IMU on the pelvis, using a totalof seven IMUs [27]. It uses a Kalman filter framework to calculate the orientation of each sensor.Since they use seven IMUs, the proposed approach is not practical, and the power consumption ofthe system is high, due to a large number of sensors and their processing cost.In summary, none of these studies targets real-time feedback to the user and considers the systempower consumption. In contrast to all of these approaches,
MGait estimates the key gait parameterswith the minimum number of IMUs and low-power bend sensors (one on each leg), and a lowpower micro-controller using a novel closed-form expression. It achieves accurate step length,stride length, and gait velocity. Low power consumption and processing requirements of
MGait can enable the first low-cost, low power, and wearable CPS for gait analysis with user-feedbackfunctionality.
MGAIT
APPROACH3.1 Gait Cycle Analysis
Accurate step length modeling requires a clear understanding of the periodic gait cycle and twokey angles (
All the angles and the length of the limbs in the following text are consideredon the sagittal plane unless otherwise specified. ): Definition 1.
Knee Angle ( 𝛽 ) is the angle formed at the joint of the thighs and legs, as shown inFigure 1(a). It ranges between 0° (when the leg is straight) and ≈
50° (during the swing).
Definition 2.
Hip Angle ( 𝛼 ) is the angle formed by the inner thigh and the vertical axis, as shownin Figure 1(a). It is positive ( ≈ 𝛼 𝑓 and 𝛽 𝑓 , respectively. Similarly, the hip and knee angles of the backlimb are denoted using 𝛼 𝑏 and 𝛽 𝑏 , respectively. The details of how to find front and back limbusing key events are explained in Section 4.4. There are four key events and two stances [4, 24], asmarked in Figure 2(a): Table 1. A compilation of previous related work on gait analysis (“–” means the results are not reported)
Ref Step Length Err. Stride Length Err. Velocity Err. Sensor Type SensorCount UserFeed. WearableForm-factor Power(mW)RMSE MAPE RMSE MAPE RMSE MAPE[28] 5.5 cm - - - - - MPU-6050 IMU 4 ✗ ✓ ✗ ✗ ✗ ✗ -[2] 4.1 cm 4.75% 6.3 cm 3.70% 0.07 m/s 4.23%
High Cost
UWB,MPU-6050 IMU 4+4 ✗ ✓ ✗ ✓ ✗ ✗ - MGait ✓ ✓
GR*: GAITRite [5]ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:5 A ng l e ( D e g r ees ) Knee angle Hip angle
Percentage of a gait cycle
0% OppositeFoot-off OppositeFoot-strike Foot-off Initial contact(Foot-strike)
B HA GF EDC
Kneeangle Hipangle
Initial contact(Foot-strike) 100%Left LegRightLeg (a)
Find hip ( 𝛼 ) and knee ( 𝛽 ) angles Bend sensorsIMU sensors
Identify front and back limbs1.Find the minimum of the front knee angle (point A in the plot to the left) time instance 𝑡 !, at which the knee angle minimum occurs3.Find 𝛽 ! ( point A) and 𝛼 ! ( point B) at 𝑡 !, minimum of the back hipangle (point D) that occurs after 𝑡 !, time instance 𝑡 &, at which the hip angle minimum occurs6.Find 𝛽 & ( point C) and 𝛼 & ( point D) at 𝑡 &, 𝑑 ! , 𝑑 " , 𝑑 , 𝑑 $ using Equations 3-62.Compute step length, stride length & gait velocity (c) User Feedback 𝒍 𝒍 𝒍 𝒍 𝜶 𝒃 𝜶 𝒇 𝜶 𝒃 + 𝜷 𝒃 𝜶 𝒇 − 𝜷 𝒇 𝜷 𝒇 𝒅 𝒅 𝒅 𝒅 𝒅 Front limb Back limb (b)
Fig. 2. (a) Overview of the gait cycle. This figure is generated using the wearable setup shown in Figure 1 (b)Illustration of the step length components used in our model. These components, defined in Equations 2–4are added to get the total step length. (c) Overview of the proposed approach. Using the hip and knee angles,we first identify the points of interest shown in part (a). Using the angles at the points of interest, we calculatethe step length components using Equations 2–4 to obtain the total step length. (1)
Initial Contact : At the beginning of the gait cycle, the foot just touches the ground. The kneeangle of the front leg (the left leg shaded in Figure 2(a)) reaches its first minimum and the hipangle is close to its maximum, as shown with markers A and B, respectively.(2)
Opposite foot-off : The opposite leg (the right leg in Figure 2(a)) starts lifting from the ground.After this point, the right leg is in a forward swing, until it strikes the ground. The angles inthis stance are denoted by markers C and D.(3)
Opposite foot-strike : At this point, the right leg completes the swing (i.e., one complete step)and touches the ground. The right leg is in front of the torso while the left leg is behind thetorso. Thus, the knee angle of the right leg is at a minimum, as shown with the marker E inFigure 2(a).(4)
Foot-off : The hip angle of the left leg reaches its minimum value. The left also starts its swingforward for the next step. This stance is shown using markers G and H.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
These four events of one leg belong to its stance phase. The period from foot-off to the nextinitial contact is the swing phase.
The goal of the
MGait approach is to model and estimate the step length and relevant parametersusing the setup shown in Figure 1(a). We place a bend sensor behind the knee to measure the kneeangle. Accelerometer and gyroscope data from the IMU located just above the knee are used tocalculate the hip angle. We process the angle data to find the major events in the gait cycle, asoutlined in Figure 2(c). Specifically, we find the points {A, B, C, D} for the first step and {E, F, G,H} for the second step plotted in Figure 2(a). The angles at these points are used to calculate thestep length components 𝑑 – 𝑑 shown in Figure 2(b). These components are used by the MGait approach to compute the step length, stride length, and gait velocity. The details of the analyticalmodel to compute the step length, stride length, and gait velocity are presented in Section 5.
Due to changing environment, sensor data is noise and drift prone. For instance, angle measuredby the bend sensor for the resting state can vary with time and IMU sensors may have offsets. Tocalibrate the sensors, we instruct the users to stand still with a straight leg for a few seconds beforethe experiment. During this period, we record the sensor readings and find their median values asthe offsets. We then subtract the offsets from the sensor data during actual use. For example, theself-calibration ensures that the IMU measures only the gravity when the user stands in a straightposition.After calibrating the sensors, we apply a downsampling and smoothing filter to reduce noise inthe data. For each sensor, we obtain the downsampled and smoothed samples as: 𝑠 𝑠 [ 𝑀𝑘 ] = 𝑀 𝑀 ( 𝑘 + ) ∑︁ 𝑖 = 𝑀 ( 𝑘 − )+ 𝑠 [ 𝑖 ] (1)where 𝑠 𝑠 is the smoothened data stream, 𝑀 is the downsampling factor, 𝑘 is the sample index, and 𝑠 is the data stream before filtering. The IMU and bend sensor sampling rates are 250 Hz and 100 Hz,respectively. Since the fastest gait is less than 10 Hz, we downsample the data to 25 Hz, which issufficient, considering the Nyquist rate. The specifications of sensors before and after filtering areshown in Table 2. Table 2. Sensors specifications
Sensorrange Samplingrate (Hz) Sampling rate aftermedian filter (Hz)Accelerometer ±8g 250 25Gyrometer ±250dps 250 25Bending sensor ±180° 100 25
The output of the bend sensor (located at the back of the knee) is the angle displacement experiencedby the sensor, as illustrated in Figure 1(a). Sample knee angle data from our experiments is visualizedin Figure 3(a). A comparison between the raw and filtered data shows that the averaging filter
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:7 smooths out the variations in the data. The knee angles reach a maximum of about 50–55° as theleg swings. Another key observation is that the angles of the two legs exhibit approximately 50%phase shift when walking. This is expected for healthy subjects, since their steps are of similarlength. A ng l e ( d e g r ee ) Raw angle (Left leg) Filtered angle (Left leg)Raw angle (Right leg) Filtered angle (Right leg)
Time (s) A ng l e ( d e g r ee ) Hip angleKnee angle(a)(b)
Fig. 3. Visualization of raw and median filtered (a) knee angle and (b) hip angle of both legs during walking
We find the hip angle using the IMU sensor located right above the knee, as shown in Figure 1(a).There are several approaches to filter the accelerometer and gyroscope samples from the IMU andobtain the hip angle. Among these, we compared the performance of the complementary filter [10]and the Madgwick filter [14], which are the most commonly used techniques. Since the Madgwickfilter outperforms the complementary filter in terms of the ability to converge and the smoothness,we used it in the final implementation. The Madgwick filter uses a quaternion representation ofthe accelerometer samples and applies a gradient descent algorithm to calculate the error in thedirection of the gyroscope samples. By compensating for the error, it accurately estimates theorientation of the IMU even during motion, without being affected by gyroscopic drift [14].Figure 3(b) plots left and right hip angles during walking. The hip angles vary between a minimumof about -10° to a maximum of 35° and exhibit a 50% phase difference, as expected. We also showinputs and outputs of the Madgwick filter in Figure 4. Specifically, the Madgwick filter takes theaccelerometer and gyroscope data shown in Figures 4(a) and 4(b) as inputs. Using these inputs, itgenerates the smoothed hip angles, as shown in 4(c).
We need to identify the key gait cycle events, such as initial contact and foot-off, to convert theknee and hip angles into step length. From our gait cycle analysis, we know that these events occurwhen the knee angle is at a minimum for one leg (point A in Figure 2(a)), while the hip angle is ata minimum for the other (point D in Figure 2(a)). To obtain the minimum values, we continuouslymonitor the five-point derivative of the two angles. A minimum is marked whenever the derivativechanges from negative to positive or zero to positive. After detecting the minimum of the kneeangle, we designate the corresponding leg as the front limb for the current step. We then call theknee angle 𝛽 𝑓 and hip angle 𝛼 𝑓 for the front limb. For example, the knee angle 𝛽 𝑓 and hip angle 𝛼 𝑓 in Figure 2(a) correspond to points A and B . We then wait for the minumum of the hip angle tooccur in the other leg, which is referred to as the “back limb”. At this point, the knee angle 𝛽 𝑏 and ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Time (s) -2-10123 A cce l e r a t i on ( g ) Accx Accy Accz
Time (s) -200-1000100200 A ngu l a r ve l o c i t y ( ° / s ) Gyrox Gyroy Gyroz
Time (s) -2002040 A ng l e ( d e g r ee ) hip angle after Madgwick filter(a)(b)(c) Fig. 4. Visualization of acceleration values (a) and angular velocity values (b) forming hip angle (c) after theMadgwick filter the hip angle 𝛼 𝑏 for the back limb corresponds to points C and D in Figure 2(a). The angle valuesare then plugged into our analytical model to estimate the step length, as detailed in Section 5. Thisprocess is repeated continuously, where front and back limbs alternate. MGait estimates the step length, stride length, and gait velocity defined below in real-time.
Definition 3.
Step Length is the distance between the front and back feet, when the front limb isin the initial contact stance and the back limb is in the foot-off stance.
Definition 4.
Stride Length is the distance between two foot strike stances of the same leg.
Definition 5.
Gait Velocity is the ratio of the stride length and the time taken to complete onestride.
We represent the stances during a step by the geometric stick diagram shown in Figure 5. The leftside of each drawing shows the front limb in the initial contact stance, while the right side showsthe back limb in the foot off position. There are two cases of the stick diagram, depending on the
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:9 position of the back limb. In the first case shown in Figure 5(a), the back limb is extended behindthe torso of the subject. In contrast, the upper part of the back limb aligns with the torso of thesubject in the second case shown in Figure 5(b). We consider these two cases separately, since theychange how the various components of the step length contribute to the model.The step length is a function of the length of the leg of the subject. The length of the leg betweenthe gluteus (hip) and popliteus point (knee joint) is denoted by 𝑙 , as shown in Figure 5(a). Thelength of the leg from the popliteus to the calcaneus point (ankle) is given by 𝑙 . We then considerthe knee and hip angles formed by both legs in the stick model. Using these definitions, we canexpress the step length as a sum of five components shown in Figure 5. Next, we describe thedefinitions of each of these components. 𝑑 : The 𝑑 component is the projection of the front leg on the ground. Figure 5(a) shows that theangle formed by the vertical line and the leg is 𝛼 𝑓 − 𝛽 𝑓 . Using this angle, the value of 𝑑 is writtenas: 𝑑 = 𝑙 sin ( 𝛼 𝑓 − 𝛽 𝑓 ) (2) 𝑑 : 𝑑 is the projection of the front thigh to the ground. Since the angle formed by the thigh andthe vertical axis is given by 𝛼 𝑓 , the projection is: 𝑑 = 𝑙 sin ( 𝛼 𝑓 ) (3) 𝑑 : 𝑑 is the projection of the back thigh to the ground. Since the angle formed by the thigh andthe vertical axis is given by 𝛼 𝑏 , the projection is: 𝑑 = 𝑙 sin (− 𝛼 𝑏 ) (4)There is a negative sign in Equation 4 to account for the two cases shown in Figure 5(a) andFigure 5(b). In Figure 5(a), the 𝑑 has to be added to obtain the total step length. Therefore, wemust ensure that the sign of 𝑑 is positive. By our definition of the hip angle, 𝛼 𝑏 is negative inFigure 5(a) because it is behind the torso. The inclusion of a negative sign in Equation 4 ensuresthat 𝑑 is positive. Conversely, in Figure 5(b) 𝑑 overlaps with 𝑑 and 𝑑 . Consequently, it has to besubtracted from the step length. In this case, 𝛼 𝑏 is positive because it is in front of the torso. Hence,we obtain a negative sign for 𝑑 , ensuring its subtraction from the total step length. 𝑑 : 𝑑 is the projection of the back leg to the ground. Similar to 𝑑 , we first need to calculate theangle between the back leg and the vertical line from the knee to the ground. In case of Figure 5(a),the magnitude of this angle is given by the sum of angles 𝛼 𝑏 and 𝛽 𝑏 , whereas for Figure 5(b) themagnitude is given by the difference of 𝛽 𝑏 and 𝛼 𝑏 . The sign of 𝛽 𝑏 is always positive, while the sign l l α f β f d d β b l l d α f -β f β b - α b d d l l α f α f -β f d d β b -α b d l d d Front limb Back limb l β f α b Front limb Back limb (a) (b) α b Fig. 5. Stick diagram for step length calculation. The stick model has two cases, depending on the positionof the back limb. (a) upper part of the back limb is behind the torso and (b) upper part of the back limb isparallel or slightly in front of the torso. The figures show the magnitude of all the angles in the model.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021. of 𝛼 𝑏 is negative in Figure 5(a) and positive in Figure 5(b). Therefore, when we consider the signsof the respective angles, the resulting angle between the back leg and the vertical line becomes ( 𝛽 𝑏 − 𝛼 𝑏 ) . Hence, the projection is expressed as: 𝑑 = 𝑙 sin ( 𝛽 𝑏 − 𝛼 𝑏 ) (5) 𝑑 : 𝑑 is given by the diameter of the subject’s thigh below the gluteus. It is included, since the stickdiagram does not cover the width of the user’s leg. It is measured at the beginning to personalizethe model to each user.Adding 𝑑 – 𝑑 the total step length is obtained as: 𝐷 = 𝑙 sin ( 𝛼 𝑓 − 𝛽 𝑓 ) + 𝑙 sin ( 𝛼 𝑓 ) + 𝑙 sin (− 𝛼 𝑏 ) + 𝑙 sin ( 𝛽 𝑏 − 𝛼 𝑏 ) + 𝑑 (6)After obtaining the step length, we calculate the stride length by adding the lengths of two consec-utive steps. We compute the gait velocity by dividing the length of a sequence of five strides bythe time. The stride time is obtained by subtracting two consecutive timestamps obtained whenfinding the minimums, as described in Figure 2(b). Gait analysis is commonly used in patient rehabilitation and health monitoring. Therefore, providingfeedback to the users about any abnormalities in gait is a crucial aspect of gait analysis. To thisend, we provide the following feedback to the user in real-time, as shown in Figure 2(b).
Gait Asymmetry Detection : Patients with movement disorders often have different left and rightstep lengths [31]. Therefore, we include gait asymmetry as one of the feedbacks that we provide tothe user. We notify the user of gait asymmetry using a buzzer on our device. The buzzer sounds anaudible alarm to the user such that they can take appropriate action to correct the gait asymmetry.
Gait Velocity Reduction Detection:
Falling is among the primary causes of death in the elderlypopulation [21]. Change in the gait velocity over a long period is considered as a significant cue topredict future falls [20, 21].
MGait keeps track of the gait velocity of each patient and notifies thepatient or the healthcare provider, if there is a specific trend in the decrease of gait velocity thatcan be identified as a potential risk of fall.
Step/Stride Length Reduction Detection:
Freezing of gait (FoG) is the most common motorsymptom in patients with Parkinson’s disease [6]. Reduction in stride/step length has been used asan indicator of a FoG episode [6]. Therefore, we keep track of the user’s moving average step/stridelength and notify the user, if there is a significant reduction in step length using an LED on ourdevice.
Feedback Algorithm:
Gait asymmetry and step/stride length reduction can be detected by ob-serving the trend of step length over time. In a healthy walking pattern, left-to-right step lengthand right-to-left step length are similar in length. In contrast, when gait asymmetry and step/stridelength reduction are present, one step is shorter than the other. That is, the difference betweenleft-to-right and right-to-left step length is larger than healthy walking. Moreover, the averagewalking speed is also reduced when gait asymmetry and step/stride length reduction are present.Using these insights, we design the feedback algorithm using the moving average and the movingvariance of a window of steps (set to five in our implementation).The first step of our algorithm is to monitor the moving variance of the step length, which showsthe range of the step length fluctuation. If the variance exceeds a threshold, we then activate thesecond part of the algorithm that checks the moving average of the step lengths. The value of thevariance threshold has to be chosen carefully to avoid false positives. From our dataset, we see thatthe variation in step length when walking is less than 10 cm. Therefore, in our implementation, wechoose the threshold as 100 cm . ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:11
After detecting the high moving variance at a certain timestamp, we compare the five-stepmoving average of the step length before and after the change in variance. If the difference is largerthan 10 cm, we give feedback about gait asymmetry and step length reduction. The moving averageof the step length is not used as a trigger to detect gait asymmetry since a lower step length isa necessary but not a sufficient condition for gait asymmetry. Specifically, the moving averageof the step length only indicates the trend of walking velocity. For example, a subject can firstwalk in large steps then switch to small steps, which results in lower moving average, but this isnot necessary to be gait asymmetry. At the same time, any changes in speed are recorded by thealgorithm to analyze the trends of gait velocity.
Wearable Device:
MGait framework uses the wearable setup shown in Figure 1(a). The magnifiedview of the wearable setup is shown in Figure 1(b). It consists of a wearable bend sensor [3] and aTexas Instruments CC2650 Sensortag [26]. We sample the bend sensor and sensortag at 100 Hz and250 Hz, respectively. The proposed gait analysis technique is implemented on the TI Sensortag toenable runtime analysis and user feedback.
User Studies:
We collected data from a total of seven subjects (S1–S7), following an official protocolapproved by the IRB board of our institution. The information, including the static parameters ( 𝑙 , 𝑙 ,and 𝑑 ), height, age, and gender of 7 subjects, are shown in Table 3. Each subject participated in sixtrials. Four of the six trials were regular free walking with normal pace, whereas in the remainingtwo the subjects were asked to imitate limping, resulting in a total of 806 steps. In addition to theempirical data, we also proposed a method to generate additional data, as described in Section 7.The synthetic data generation augments the data collected from the 7 subjects to create a richerdataset for further research. Both empirical and synthetic data will be released to the public, alongwith this paper.Accurate step length reference is critical to evaluating MGait . For most of the experiment,subjects walked on a 7-meter long white paper roll. We rubbed the bottom of the subject’s shoeswith washable ink. The marks left on the paper are then used to capture the user steps. After eachtrial, the distances between the marks on the paper are recorded as step lengths. For the last twosubjects, we employed the GAITRite [5] system to obtain the step length reference.Across all the experiments, the recorded ranges for step length, stride length, and gait velocityare 25–78 cm, 69–156 cm, and 0.60–1.27 m/s respectively. The step length distribution in ourexperimental data is as follows. 80% of the step lengths are between 60 cm and 70 cm, 9% of themfall into 50 cm to 60 cm range, 6% are shorter than 50 cm, finally and 5% of them are longer than 70cm. Similarly, 76% of the stride lengths are from between 120 cm and 140 cm, 11% of them vary from100 cm to 120 cm, 7 of them are shorter than 100 cm, and 6% of them are longer than 140 cm. Wealso perform a statistical analysis on our dataset. For each subject’s step length and stride length,we calculate the mean and standard deviation. Moreover, a 95% confidence interval is computed.The detailed results are shown in Table 3.
The proposed model has three user-specific static parameters: length of the thigh ( 𝑙 ), length of theleg ( 𝑙 ), and hip diameter ( 𝑑 ). Since measuring them is subject to human error, we employ bothoffline batch processing and online least squares regression models to determine these parameters. ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Table 3. Overview of the dataset for each subject
Static param.(cm) Height(cm) Age Gender Step length (cm) Stride length (cm)Mean ± std CI Mean ± std CIS1 30/45/14 193 23 M 61.33 ± 6.65 60.31 - 62.35 122.87 ± 11.07 120.46 - 125.29S2 25/40/15 175 26 M 61.75 ± 13.39 57.67 - 65.82 123.50 ± 25.41 112.23 - 134.76S3 29/40/16 188 26 M 73.37 ± 3.83 72.10 - 74.65 146.66 ± 6.20 143.58 - 149.74S4 27/33/12 170 29 M 58.21 ± 13.56 53.98 - 62.44 116.42 ± 25.33 104.89 - 127.96S5 20/25/15 163 26 F 48.05 ± 6.58 46.42 - 49.68 96.65 ± 11.57 92.48 - 100.82S6 28/40/16 181 25 M 60.47 ± 2.89 60.19 - 60.75 121.01 ± 5.29 120.28 - 121.74S7 22/28/22 158 25 F 57.76 ± 5.35 56.14 - 59.39 115.53 ± 10.36 110.4 - 120.13 Batch processing is a common method used in the leastsquare estimation. For our step length estimation problem, batch processing is an effective solutionwhen the gait analysis is performed offline. For instance, a patient can walk for a few minutes,while measuring the golden step length reference. The collected data then can be used to reducethe measurement error by tuning the model.We employ a nonlinear least square estimation to solve the problem. First, we measure theirnominal values by hand and record them as 𝑙 𝑛𝑜𝑚 , 𝑙 𝑛𝑜𝑚 , 𝑑 𝑛𝑜𝑚 . We then use our test data and thesenominal values to formulate the following regression problem:minimize 𝑁 ∑︁ 𝑖 = || 𝐷 𝑟𝑒𝑓 ,𝑖 − 𝐷 𝑖 ( 𝑙 , 𝑙 , 𝑑 )|| subject to . × 𝑙 𝑛𝑜𝑚 ≤ 𝑙 ≤ . × 𝑙 𝑛𝑜𝑚 . × 𝑙 𝑛𝑜𝑚 ≤ 𝑙 ≤ . × 𝑙 𝑛𝑜𝑚 (7) . × 𝑑 𝑛𝑜𝑚 ≤ 𝑑 ≤ . × 𝑑 𝑛𝑜𝑚 where 𝑁 is the number of steps used for regression and 𝐷 𝑟𝑒𝑓 ,𝑖 is the step length reference for 𝑖 th step. The objective in Equation 7 minimizes the sum of squared error between 𝐷 𝑟𝑒𝑓 ,𝑖 and ourestimate 𝐷 𝑖 obtained with Equation 6 using the measured angles ( 𝛼 𝑓 , 𝛼 𝑏 , 𝛽 𝑓 , 𝛽 𝑏 ). We constrain theoptimization variables 𝑙 , 𝑙 , and 𝑑 within 10% of their nominal values to ensure that they do notoverfit to unrealistic values. As an example, the parameters of S1 are 30.0, 45.0, and 12.0 cm initially.After the regression, they are corrected as 33.0, 42.1, and 13.2 cm, respectively. This method is usedonce for each subject to find their static parameters.
It is also useful to fine-tune the model in real-time after itis deployed, since this can help the proposed system to adapt to a particular person. Batch processingis not appropriate for this purpose as it requires offline processing. Therefore, we also introduce aRecursive Least Square (RLS) estimation framework that can calibrate the static parameters for agiven user.We can represent the step length in Equation 6 as a linear equation with the input features as: h [ 𝑛 ] = 𝑠𝑖𝑛 ( 𝛼 𝑓 [ 𝑛 ]) − 𝑠𝑖𝑛 ( 𝛼 𝑏 [ 𝑛 ]) 𝑠𝑖𝑛 ( 𝛼 𝑓 [ 𝑛 ] − 𝛽 𝑓 [ 𝑛 ]) + 𝑠𝑖𝑛 ( 𝛽 𝑏 [ 𝑛 ] − 𝛼 𝑏 [ 𝑛 ]) , (8)where 𝑛 is the time index. Similarly, the model coefficients become the static parameters we aim toestimate: w = (cid:2) 𝑙 𝑙 𝑑 (cid:3) 𝑇 (9) ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:13 L e ng t h ( c m ) l1 RLS Init Batch LS L e ng t h ( c m ) l20 5 10 15 20 25 30 35 40Number of steps14151617 L e ng t h ( c m ) d5 0 5 10 15 20 25 30 35 40Number of steps81012141618 A PE ( % ) Abs percentage error (a) (b)(c) (d)
Fig. 6. Visualization of how (a) l1, (b) l2, and (c) d5 online updating and converging to the batch LS result andthe (d) absolute percentage error of step length estimation using RLS for one subject.
When the step length model needs to be updated, e.g., for a new user, the system operates ina calibration mode. In this mode, the user is asked to walk on a line with a constant step length.During this time, the RLS filter estimates the step length as a product of h [ n ] and w , which aregiven Equation 9 and Equation 8, respectively. This estimate is subtracted from the reference steplength to find the modeling error. Then, this error is then used to update the parameter estimates w using an RLS technique with stability guarantees [7].Experimental results show that the RLS method improves the step length estimation accuracyby about 3% on average. It is effective in reducing the measurement error online, especially forsubjects with a high initial error rate. For instance, S2 and S5 have 16.44% and 16.01% initial MAPE,respectively. Applying RLS estimation, the MAPE are reduced by 7.54% and 8.05%, respectively.Figure 6 shows how static parameters and the error converge online, as steps increases. The RLSmodel converges to its final values within five steps. The accuracy of RLS converges also to the batchLS when it is applied to all subjects. This shows that the RLS approach is effective for estimatingthe static parameters at runtime. Table 4. Error in step length estimation using offline and online method (without angle correction)
Initial After Batch LS After RLSMAPE(%) RMSE(cm) MAPE(%) RMSE(cm) MAPE(%) RMSE(cm)S1 6.38 4.03 5.80 3.65 6.28 3.99S2 16.44 12.15 8.55 6.34 8.90 6.60S3 7.17 4.53 6.88 4.31 6.93 4.38S4 7.65 4.99 5.89 3.75 5.95 3.79S5 16.01 8.03 7.94 3.99 7.96 3.99S6 8.70 5.25 8.07 4.86 8.66 5.22S7 7.88 4.47 7.69 4.35 7.66 4.35Avg. 10.03 6.21 7.26 4.46 7.47 4.62
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
The angle measurements are also subject to error, due to the noisy nature of the sensors. Furthermore,sensors may experience a systematic bias, due to their positioning. This difference should beaccounted for in the models to ensure an accurate estimation of the step length. To compensatefor these errors, we solve the nonlinear regression problem given in Equation 7 again, by usingthe static parameters found in Section 6.2. This time, we let the knee and hip angles in Equation 6become free variables. That is, the estimation for the 𝑖 th step is changed as 𝐷 𝑖 ( 𝛼 𝑓 , 𝛼 𝑏 , 𝛽 𝑓 , 𝛽 𝑏 ) in theproblem formulation in Equation 7. Similarly, these parameters are constrained within 10% of theirnominal values given by the average of our observations, following the same methodology as inSection 6.2.The output of the nonlinear regression gives the average knee and hip angles that provide theminimum estimation error. Hence, we use the difference between these values and our observationsas the measurement bias. For instance, the nonlinear regression finds the hip angle of the forwardleg at the initial contact point as 𝛼 𝑓 = 24.8° for Subject 5. The same value is 22.5° in our observationdataset. Consequently, we compute the bias as -2.3°. We rectify the sensor bias, by subtracting theseempirically determined values from the knee and hip angles found in real-time. After correcting thebias, the error between the best fit and our measured values become zero-mean, as demonstratedin Figure 7. Difference in Angles (degree) Difference in Angles (degree) P D FP D F Before After 𝜶 𝒃 𝜷 𝒃 𝜶 𝒃 𝜷 𝒃 𝜶 𝒇 𝜷 𝒇 𝜶 𝒇 𝜷 𝒇 Fig. 7. Illustration of the error distribution after removing sensor bias.
We use 70% of the steps collected from a subject for regression (Sections 6.2 and 6.3) and theremaining 30% for evaluating accuracy. The errors reported in this section use the corrected staticparameters and angle distributions. The average MAPE in step length estimation is about 10.03%, when we use the user measurements without any error correction . The error is reduced to about 7.26%with the fitting of user-specific parameters, as shown in Figure 8. It is reduced further to about 5.49%,when we remove the offset in angle measurements. In particular, the MAPE for Subjects 2 and 5drops by more than 10%, since they had a higher initial error due to inaccurate limb measurements.We then evaluate the accuracy for step length, stride length and gait velocity for each subject.Each row in Table 5 corresponds to a different subject, while the columns show the MAPE andRMSE in step length, stride length, and gait velocity, respectively. The MAPE and RMSE for steplength range from 3.63% to 6.91% and 2.83 cm to 5.48 cm, respectively. The error rates are lower forstride length and gait velocity estimations. Specifically, the maximum MAPE in stride length andgait velocity is 6.40% and 3.59%, respectively. We note that the RMSE value for stride length seemslarger, since each stride consists of two steps. In summary, the results show that the proposedmodel is able to accurately estimate gait parameters with lower power consumption overheads.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:15 M A PE ( % ) Initial Fit1 Fit2
Fig. 8. Mean absolute percentage error in step length estimatesTable 5. Error in step length, stride length, and velocity with angle correction. The results in this table are for30% test data.
Step Length Stride Length VelocityMAPE(%) RMSE(cm) MAPE(%) RMSE(cm) MAPE(%) RMSE(m/s)S1 5.76 4.53 4.48 6.55 3.33 0.04S2 3.63 3.35 3.93 7.56 0.77 0.01S3 6.91 5.48 3.89 6.54 1.64 0.02S4 5.94 4.51 3.71 6.48 2.06 0.03S5 4.38 2.83 1.99 2.66 0.94 0.01S6 5.31 3.61 4.79 6.61 2.73 0.05S7 6.51 4.24 6.40 7.72 3.59 0.06Avg. 5.49 4.08 4.17 6.30 2.15 0.03
User feedback on gait quality plays a critical role in patient rehabilitation and health monitoring.For instance, feedback on unequal step lengths is an important part of rehabilitation of patientswith a leg injury. Similarly, gait speed is an important indicator in movement disorders.An example of the user feedback for one subject given by
MGait is shown in Figure 9.
MGait provides user feedback as soon as the step lengths are asymmetric and the variance between steplengths grows and exceeds the threshold of 100, as shown in Figure 9. The subject starts walking,aiming for approximately 60 cm steps (Fixed-length walking) and then continues without any target(Free walking). After 150 steps, the subject starts making one regular step, followed by a short step(Gait asymmetry).
MGait keeps track of the moving averages of the step length. At each new step,
MGait also checks the moving variance. Since there are no abnormalities in the first 150 steps,
MGait does not produce any user feedback. The moving variance is larger than the threshold of 100at 𝑡ℎ steps. After this point, MGait waits for five steps and compares the average step lengthbefore and after the change in variance. Since the change in the average step length is greater than10 cm,
MGait raises feedback. Specifically, the user is notified using a buzzer and an LED.We evaluated all seven subjects using this feedback algorithm. Overall, 49 steps out of 806 stepsare obtained while the user was walking with gait asymmetry. The precision, recall, and f1 scoreof predicting limping are 1, 0.71, and 0.83, respectively. We plan to integrate the feedback with asmartphone application to perform longer-term evaluations.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Fixed length walking Free walking Gait Asymmetry
Fig. 9. Gait asymmetry and gait velocity reduction detection
Experimental data collection and labeling is both time- and labor-consuming due to three barriers.First, a reliable multi-sensor hardware infrastructure is needed to perform the experiments. Besidescalibrating and operating individual sensors, the infrastructure should synchronize data frommultiple sensors (in our case accelerometer, gyroscope, and bend sensor). Second, experimentswith human subjects require comprehensive experiments and recruitment protocols approvedby an institutional review board (IRB). Both preparation of protocols and their approval requiresignificant amount of time (a few months). Last but not least, representative datasets requiremultiple volunteers with diverse attributes, such as age, gender, and height.Throughout this study, sensor data and step length data were collected for more than 800 stepsfrom seven subjects following an IRB protocol. Although this data is sufficient for a thoroughexperimental evaluation, additional data points may catalyze future research. The synthetic datageneration can augment experimental data to create a richer dataset. Synthetic data has beenpreviously used to make significant advances in many domains. For instance, in the Network-on-Chips (NoC) domain, real-world workloads available at design time do not represent all thetraffic patterns that will be encountered by the network in the future [25]. Furthermore, runningmultiple workloads to test the network under varying traffic conditions is time-consuming. Inorder to address this, a number of synthetic workloads were developed for NoCs [12, 18, 23]. Thesesynthetic traffic patterns speed up the design and test of NoCs architectures, leading to significantadvances in the field.To this end, we present a technique to generate synthetic training and test data with the help ofour existing experimental dataset to enable gait analysis research with more data. To the best ofour knowledge, this is the first dataset that provides two sensor modalities for gait analysis. Thesynthetic data, along with the original sensor data, will aid future researchers to develop novelmethods of gait analysis while avoiding the tedious step of data collection. Both the empirical andsynthetic datasets produced by this work will be released to the public.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:17
Multiple data generation methods using deep neural networks have been proposed in recent years.Goodfellow et al. [9] presented Generative Adversarial Networks (GANs) in 2014 for the first time.GANs have been implemented for a variety number of applications in image, text, and sensordataset generation. Alzantot et al. [1] proposed an architecture for generating wearable sensordata, including accelerometer and gyroscope values using Gaussian Mixture Density Networkconnected to a recurrent neural network (RNN). Although GAN models can generate new randomplausible examples for a given dataset, they do not have labels for the data, which limits theireffectiveness for supervised learning tasks. To solve this problem, Mirza et al. [17] proposed aConditional Generative Adversarial Networks (CGAN). It includes the label information in theinput of the discriminator and the generator as an additional input layer.
We implement a CGAN to generate the step length dataset. The objective function of CGAN isshown below: min 𝐺 max 𝐷 E 𝑥 ∼ 𝑝 data [ log 𝐷 ( 𝑥 | 𝑦 )] + E 𝑧 ∼ 𝑝 𝑧 [ log ( − 𝐷 ( 𝐺 ( 𝑧 | 𝑦 )))] . (1)The idea behind GAN is a two-player minimax game. GANs consist of both generator and discrimi-nator models. The generator is used to generate new domain samples from latent space, while thediscriminator classifies whether the input samples are real or generated. In the very beginning,both performances are poor in the sense that the generator can only generate samples that arewell-distinguished from the real data distribution, while the discriminator cannot tell whether theinput data is real or synthetic. However, as more samples are given to train the discriminator, theperformance of the discriminator model is used to update both the model weights of the discrimi-nator itself and the generator model. This shows that the generator never observes examples fromthe domain. It has the ability to adapt itself, based on how well the discriminator performs. Ideally,in the end, the generator can generate samples that are hard for the discriminator to distinguishwhether they are real or synthetic. Similarly, the discriminator is working decently when it is ableto tell whether samples are from the real distribution or generated.In the implementation of CGAN, we employ an embedding layer for the labels, then concatenateit with our input data samples as the inputs. Therefore, the model knows which class the currentdata belongs to (for the discriminator) and which class the generated data should be labeled as(for the generator). In such a way, labels, which are the additional information are attached to theinput samples. The improvement shows more stable and faster training, and most importantly, thegenerated samples have labels.Figure 10 describes the flow of generating a synthetic dataset using CGAN. Algorithm 1 showsthe pseudo-code for how we train the CGAN. From the real data, we segmented windows based oneach walking step. In a specific window, there are 3-axis accelerations, 3-axis angular velocity, andknee angle data. We then gave these data to the CGAN as the input. In our case, the dimension ofdata is 7 (3+3+1) and the data length is 41. Each window is, on average, 1.64 seconds in duration,since the sampling rate is 25 Hz. Note that not every window is exactly 1.64 seconds. For theshorter width windows, we pad the sensor data with zeros. We also embedded the step length asan additional layer to the data. The CGAN takes these inputs to train the model. As mentionedabove, in each epoch, we first use real data from the input and the noise from latent space totrain the discriminator to differentiate the real data from the synthetic data. Secondly, we use thegenerator to generate the synthetic data and use this synthetic data to train the whole CGAN model,while disabling the training of the discriminator. In other words, we only train the generator here. ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
After enough epochs, as both the loss of both discriminator and generator converges, CGAN iswell-trained. We then use the generator to generate the synthetic data with labels. To get the hipangle, we need to use the Madgwick filter to process the synthetic raw IMU data again. Finally, weuse the
MGait model to estimate the step length from the synthetic data and compare the accuracy.
Knee Angle 𝑎 x , 𝑎 y , 𝑎 z g x , g y , g z Hip AngleReal Data Segmented 𝑎 x , 𝑎 y , 𝑎 z g x , g y , g z CGAN MGaitMadgwick Filter Knee Angle
Fig. 10. Overview of the synthetic data generation flow
Algorithm 1:
Training of the CGAN
Input: a batch of k samples { 𝑒 ( ) 𝑟𝑒𝑎𝑙 , . . . , 𝑒 ( 𝑘 ) 𝑟𝑒𝑎𝑙 } , where 𝑒 ( 𝑘 ) = ( 𝑥 ( 𝑘 ) 𝑟𝑒𝑎𝑙 , 𝑦 ( 𝑘 ) 𝑟𝑒𝑎𝑙 ) . 𝑥 ( 𝑘 ) 𝑟𝑒𝑎𝑙 here is the6-axis IMU data and the knee angle, 𝑦 ( 𝑘 ) 𝑟𝑒𝑎𝑙 here stands for the label, which is the steplength for this window. Output: a batch of k synthetic samples { 𝑒 ( ) 𝑠𝑦𝑛 , . . . , 𝑒 ( 𝑘 ) 𝑠𝑦𝑛 } . where 𝑒 ( 𝑘 ) = ( 𝑥 ( 𝑘 ) 𝑠𝑦𝑛 , 𝑦 ( 𝑘 ) 𝑠𝑦𝑛 ) , 𝑥 ( 𝑘 ) 𝑠𝑦𝑛 hereis the 6-axis IMU data and the knee angle, 𝑦 ( 𝑘 ) 𝑠𝑦𝑛 here stands for the label, which isthe step length for this window. for numbers of training iterations do • Sample a minibatch of 𝑠 noise samples { 𝑒 ( ) 𝑠𝑦𝑛 , . . . , 𝑒 ( 𝑠 ) 𝑠𝑦𝑛 } from noise prior 𝑝 𝑔 ( 𝑧 ) . • Sample a minibatch of 𝑠 samples { 𝑒 ( ) 𝑟𝑒𝑎𝑙 , . . . , 𝑒 ( 𝑠 ) 𝑟𝑒𝑎𝑙 } from data distribution 𝑝 𝑑𝑎𝑡𝑎 ( 𝑒 ) . • Update the discriminator by increasing its stochastic gradient using Adam optimizer. • Sample a minibatch of 𝑠 noise samples { 𝑒 ( ) 𝑠𝑦𝑛 , . . . , 𝑒 ( 𝑠 ) 𝑠𝑦𝑛 } from noise prior 𝑝 𝑔 ( 𝑧 ) . • Update the generator by decreasing its stochastic gradient using Adam optimizer. end
Figure 11(a) plots the knee angle based on our experimental data. The angles for each user areoverlayed on top of each other to show the pattern and variations. Figure 11(b) shows that thesynthetically generated data leads to a very similar pattern with the real knee angle. Similarly, weobserve a good match between the hip angle based on the experimental data and synthetic data, asshown in Figure 11(c) and Figure 11(d). We also note that the synthetic raw data also has a similarpattern with the real raw data. We do not plot all 6-axis IMU data here for the readability.The quality of the synthetic data is evaluated first using statistical metrics, including mean,standard deviation, and correlation coefficients of both measured and synthetic data over a windowof samples that correspond to a single step. The mean value of the synthetic data is within 1 ◦ degreeof the measured data for both knee and hip angles, as shown in Table 6. The synthetic data hasa slightly higher standard deviation than the real data since its range increases when the GANstry to capture all data patterns from the real data. Table 6 also compares the pairwise correlationcoefficients of all data windows. The mean correlation coefficients of measured and synthetic dataare 0.71 and 0.79, respectively, for the hip angle. Similarly, the mean correlation coefficients ofthe measured and synthetic knee angles are 0.77 and 0.90. The real data has a lower correlationcoefficient since it contains some outliers from sensors reading, resulting in a low correlation ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:19
Number of samples -200204060 A ng l e ( d e g r ee ) Number of samples -200204060 A ng l e ( d e g r ee ) (a) (b)knee angle (real) knee angle (synthetic)0 10 20 30 Number of samples -200204060 A ng l e ( d e g r ee ) Number of samples -200204060 A ng l e ( d e g r ee ) (d)(c) hip angle (real) hip angle(synthetic) Fig. 11. One window of (a) measured knee angle, (b) synthetic knee angle, (c) real hip angle, and (d) synthetichip angle for a given step length. The grey dashed lines show different instances of the data, while the redline shows a representative window. coefficient. We fine-tune the GANs such that the generated synthetic data will not learn the outlierdata patterns, which leads to a higher correlation coefficient in generated data.In addition to the statistical analysis, we evaluate the quality of the synthetic data using a“representative” window of hip and knee angles, as shown in Figure 11. Each window of datacorresponds to a different step in our data set. To find the representative window, we first computethe Euclidean distance between each window in the data set. A large distance between a givenwindow and others imply that the corresponding window is more likely to be an outlier. Therefore,the window with the smallest average Euclidian distance to other windows is chosen as the representative window for the corresponding angles, as illustrated with red lines in Figure 11. Weobserve that the representative instances of synthetic and real data are very similar in terms ofdata pattern and the values for both hip and knee angles, as shown in Figure 11(a), (b), (c), and (d),respectively.To show that our synthetic data can be used for the gait analysis, we also compare step lengthestimations using real data and the synthetic data in Table 7. Without doing any regression, the6400 steps synthetic data achieve a 9.39% MAPE in step length estimation, while the real data is10.03%. After the batch LS and RLS process, the MAPE of synthetic data reduced to 8.17%, andit remains the same for the RLS approach. For step estimation, using both initial and regressedsynthetic data give similar MAPE (within 1%) when compared to the real data. In summary, thesynthetic data has a patterns similar to the real data and it can be used to do the step estimation.
ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Table 6. Statistical metrics of real data and synthetic data
Knee Angle Hip AngleMean Std. Correlation Coefficients Mean Std. Correlation CoefficientsReal data 14.45 15.05 0.77 9.76 12.56 0.71Synthetic data 14.59 21.11 0.90 9.33 14.14 0.79
Table 7. Error in step length estimation using real data and synthetic data (without angle correction)
Initial Batch LS RLSMAPE(%) RMSE(cm) MAPE(%) RMSE(cm) MAPE(%) RMSE(cm)Real data(Avg.) 10.03 6.21 7.26 4.46 7.48 4.61Synthetic data(Avg.) 9.39 5.63 8.17 4.90 8.17 4.90
Gait analysis plays a crucial role in patient rehabilitation and health monitoring of movementdisorder patients. This paper presented
MGait , a wearable CPS for step length, stride length, andgait velocity estimation. We start with a wearable device that incorporates bend sensors and IMUsto measure the knee and hip angles of the subject. We then use the angles to determine the gaitparameters of the subject at runtime with constant complexity. Experimental evaluations usingseven subjects showed that our approach is able to estimate the step length, one of the mostwidely used gait parameters, with 5.49% error on average. We propose a synthetic data generationtechnique, based on CGAN, and we will give the access of both real data and synthetic data tothe public. We plan to use the
MGait framework for studies with movement disorder patients andenable a robust ecosystem for health monitoring and rehabilitation.
The authors would like to thank Professor Narayanan Krishnamurthi and his student Miss NivedithaMuthukrishnan at Arizona State University for the assistance in accessing the GAITRite equipment.This research was funded in part by NSF CAREER award CNS-1651624.
REFERENCES [1] Alzantot, M., Chakraborty, S., and Srivastava, M. Sensegen: A deep learning architecture for synthetic sensordata generation. In (2017), IEEE, pp. 188–193.[2] Anderson, B., et al. Mobile Gait Analysis Using Foot-Mounted UWB Sensors.
Proc. ACM IMWUT 3
Phys. Med. Reh. Clin. N. 13
Proc. BodyNets (2015).ACM Trans. Internet Things, Vol. 11, No. 1, Article 111. Publication date: January 2021.
Gait : Model-Based Gait Analysis Using Wearable Bend and Inertial Sensors 111:21 [7] Fortescue, T., Kershenbaum, L. S., and Ydstie, B. E. Implementation of self-tuning regulators with variable forgettingfactors.
Automatica 17 , 6 (1981), 831–835.[8] González, I., et al. Comparison Between Passive Vision-Based System and a Wearable Inertial-Based System forEstimating Temporal Gait Parameters Related to the GAITRite Electronic Walkway.
J. Biomed. Inform. 62 (2016),210–223.[9] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.Generative adversarial nets. In
Advances in neural information processing systems (2014), pp. 2672–2680.[10] Higgins, W. T. A comparison of complementary and kalman filtering.
IEEE Transactions on Aerospace and ElectronicSystems , 3 (1975), 321–325.[11] Hsu, A.-L., Tang, P.-F., and Jan, M.-H. Analysis of Impairments Influencing Gait Velocity and Asymmetry ofHemiplegic Patients After Mild to Moderate Stroke.
Arch. Phys. Med. Rehabil. 84 , 8 (2003), 1185–1193.[12] Kiasari, A. E., Lu, Z., and Jantsch, A. An analytical latency model for networks-on-chip.
IEEE Transactions on VeryLarge Scale Integration (VLSI) Systems 21 , 1 (2012), 113–123.[13] Kim, D., Ju, H., and Park, C. G. Comparison of Step Length Estimation Models Using Inertial Sensor on Pelvis.
ICA-SYMP (2019).[14] Madgwick, S. O., Harrison, A. J., and Vaidyanathan, R. Estimation of IMU and MARG Orientation Using a GradientDescent Algorithm. In
IEEE ICORR (2011), pp. 1–7.[15] Marras, C., et al. Prevalence of Parkinson’s Disease Across North America.
NPJ Parkinsons Dis. 4 , 1 (2018), 21.[16] McDonough, A. L., Batavia, M., Chen, F. C., Kwon, S., and Ziai, J. The validity and reliability of the gaitrite system’smeasurements: A preliminary evaluation.
Archives of physical medicine and rehabilitation 82 , 3 (2001), 419–425.[17] Mirza, M., and Osindero, S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).[18] Ogras, U. Y., and Marculescu, R.
Modeling, Analysis And Optimization Of Network-On-Chip Communication Architec-tures , vol. 184. Springer Science & Business Media, 2013.[19] Pepa, L., et al. Real Time Step Length Estimation on Smartphone.
IEEE ICCE (2016).[20] Phillips, L. J., et al. Using Embedded Sensors in Independent Living to Predict Gait Changes and Falls.
West. J. Nurs.Res. 39 , 1 (2016), 78–94.[21] Piau, A., et al. When Will My Patient Fall? Sensor-Based In-Home Walking Speed Identifies Future Falls in OlderAdults.
J. Gerontol. A Biol. Sci. Med. Sci. (2019).[22] Pirker, W., and Katzenschlager, R. Gait Disorders in Adults and the Elderly.
Wien. Klin. Wochenschr. 129 , 3-4 (2017),81–95.[23] Rahmani, A.-M., Afzali-Kusha, A., and Pedram, M. A novel synthetic traffic pattern for power/performance analysisof network-on-chips using negative exponential distribution.
Journal of Low Power Electronics 5 , 3 (2009), 396–405.[24] Shi, L., Duan, F., Yang, Y., and Sun, Z. The effect of treadmill walking on gait and upper trunk through linear andnonlinear analysis methods.
Sensors 19 , 9 (2019), 2204.[25] Soteriou, V., Wang, H., and Peh, L. A statistical traffic model for on-chip interconnection networks. In
Sensors 19 , 14 (2019), 3140.[28] Wang, L., Sun, Y., Li, Q., and Liu, T. Estimation of Step Length and Gait Asymmetry Using Wearable Inertial Sensors.
IEEE Sens. J. 18 , 9 (Jan 2018), 3844–3851.[29] Webster, K. E., Wittwer, J. E., and Feller, J. A. Validity of the gaitrite® walkway system for the measurement ofaveraged and individual step parameters of gait.
Gait & posture 22 , 4 (2005), 317–321.[30] Wu, X., et al. A Robust Step Length Estimation System for Human Gait Using Motion Sensors.
Proc. Wireless Health (2015).[31] Yogev, G., et al. Gait Asymmetry in Patients with Parkinson’s Disease and Elderly Fallers: When Does the BilateralCoordination of Gait Require Attention?