Associations between finger tapping, gait and fall risk with application to fall risk assessment
AAssociations between finger tapping, gait andfall risk with application to fall risk assessment
Jian MA ∗ Hitachi (China) Research & Development CorporationJuly 1, 2020
Abstract
As the world ages, elderly care becomes a big concern of the society.To address the elderly’s issues on dementia and fall risk, we have inves-tigated smart cognitive and fall risk assessment with machine learningmethodology based on the data collected from finger tapping test andTimed Up and Go (TUG) test. Meanwhile, we have discovered theassociations between cognition and finger motion from finger tappingdata and the association between fall risk and gait characteristics fromTUG data. In this paper, we jointly analyze the finger tapping andgait characteristics data with copula entropy. We find that the associ-ations between certain finger tapping characteristics (number of tapsof both hand of bi-inphase and bi-untiphase) and TUG score, certaingait characteristics are relatively high. According to this finding, wepropose to utilize this associations to improve the predictive models ofautomatic fall risk assessment we developed previously. Experimentalresults show that using the characteristics of both finger tapping andgait as inputs of the predictive models of predicting TUG score canconsiderably improve the prediction performance in terms of MAEcompared with using only one type of characteristics.
Keywords: copula entropy; association; finger tapping; Timed Up and Go;gait characteristics; fall risk assessment ∗ Email: [email protected] a r X i v : . [ q - b i o . Q M ] J un Introduction
As the world is continuously aging [1], elderly care becomes a high concernof the society. Delivering better care to the elderly can not only improvetheir wellbeings of all aspects, but also relieve the burden of their family andsociety. At the core of all the needs of the elderly is the need for health andmedical care. According to the WHO study [2], dementia and fall injury aretwo main diseases which suffer the elderly mainly instead of other popula-tions. How to manage these diseases for the elderly is a main challenge. Toaddress these issues, better care instruments are very needed.In our previous research, two technologies for smart elderly care weredeveloped: one for predicting dementia [3] and the other for automatic fallrisk assessment [4]. The former technology is to predict Minimal Mental StateExamination (MMSE) score from finger tapping measurement with machinelearning, in which a group of characteristics of finger tapping movement areextracted and selected for the predictive models. The latter technology isbased on the similar machine learning methodology, to predict Timed Upand Go (TUG) score from a group of gait characteristics extracted fromvideo with steoro vision and 3D pose estimation technologies.In these research, finger tapping test and TUG test are two sources ofexperimental data. The two types of data were analyzed separately in twoindependent works. In fact, cognitive impairment and fall are both commonsyndromes of the aging people. There are many evidences that cognitionimpairment is an predictor of fall risk and associated with increased fallrisk [5]. There are also several research reporting the relationship betweencognition and gait [6, 7, 8].In this paper, we will continue to investigate the relationship betweenfinger tapping, cognition, gait and fall risk by jointly analyzing the datacollected from the two research in [3, 4]. Previously, the relationship betweencharacteristics of finger tapping and cognition impairment has been studied,in which certain characteristics (number of taps, average interval of tapping,frequency of tapping, and SD of average interval of tapping) were found to beassociated with MMSE score [3]. However, the other relationships betweencognition, finger motion and gait remain to be studied.Discovering such associations is of fundamental importance for automaticfall risk assessment because they can lay scientific foundations of the predic-tive models built in the research. If, for example, the relationship betweenfinger tapping and TUG score is found, then the models for predicting TUG2core can be improved with characteristics of finger tapping. Such relation-ship will also be the evidence that finger motor ability and functional abilityare related with each other. In this paper, we will try to find such asso-ciations within our dataset and then utilize the association relationships toimprove the prediction of fall risk, i.e., TUG score.The mathematical tool used in this research is Copula Entropy (CE),which is defined by Ma and Sun [9]. It is a rigorously mathematical con-cept for statistical independence testing and enjoys several good axiomaticproperties for statistical independence measure. A simple non-parametricmethod for estimating CE was also proposed, which makes CE universallyapplicable without making any assumptions [9]. As a tool for discoveringassociation relationships, it has been applied successfully in our previous re-search to study the relationship between finger motor and cognitive ability[3] and the relationship between gait characteristics and fall risk [4].The main contributions of this paper include: • The association relationships between finger tapping, gait, and fall riskare discovered with CE. Particularly, the association between the char-acteristics of finger tapping and gait, and the association between thecharacteristics of finger tapping and fall risk are discovered; • A method for predicting TUG score (fall risk) with both the charac-teristics of finger tapping and gait is proposed and its advantage isdemonstrated on real data. The predictive models such developed areexplainable due to the association relationships discovered above.This paper is orgnized as follows: in Section 2, the theory and estimationof CE will be introduced; Section 3 will give the details on the data collectedfrom the previous research; experiments and results on association discoveryand its application on automatic TUG score prediction will be presented inSection 4 and followed by some discussion in Section 5; finally, we concludethe paper in Section 6.
Copula theory unifies representation of multivariate dependence with cop-ula function [10, 11]. According to Sklar theorem [12], multivariate density3unction can be represented as a product of its marginals and copula densityfunction which represents dependence structure among random variables.This section is to define an association measure with copula. For clarity,please refer to [9] for notations.With copula density, Copula Entropy is define as follows [9]:
Definition 1 (Copula Entropy) . Let X be random variables with marginals u and copula density c ( u ) . CE of X is defined as H c ( X ) = − (cid:90) u c ( u ) log c ( u ) d u . (1)In information theory, Mutual Information (MI) and entropy are twodifferent concepts [13]. In [9], Ma and Sun proved that MI is actually a kindof entropy, negative CE, stated as follows: Theorem 1.
MI of random variables is equivalent to negative CE: I ( X ) = − H c ( X ) . (2)Theorem 1 has simple proof [9] and an instant corollary (Corollary 1) onthe relationship between information containing in joint probability densityfunction, marginals and copula density. Corollary 1. H ( X ) = (cid:88) i H ( X i ) + H c ( X ) (3)The above results cast insight into the relationship between entropy, MI,and copula through CE, and therefore build a bridge between informationtheory and copula theory. CE itself provides a theoretical concept of statis-tical independence measure. It is widely considered that estimating MI is notoriously difficult. Under theblessing of Theorem 1, Ma and Sun [9] proposed a non-parametric methodfor estimating CE (MI) from data which composes of only two steps: ∗ ∗ The R package copent for estimating copula entropy is available on the CRAN andalso on GitHub at: https://github.com/majianthu/copent .
4. Estimating Empirical Copula Density (ECD);2. Estimating CE.For Step 1, if given data samples { x , . . . , x T } i.i.d. generated from ran-dom variables X = { x , . . . , x N } T , one can easily estimate ECD as follows: F i ( x i ) = 1 T T (cid:88) t =1 χ ( x it ≤ x i ) , (4)where i = 1 , . . . , N and χ represents for indicator function. Let u = [ F , . . . , F N ],and then one can derives a new samples set { u , . . . , u T } as data from ECD c ( u ).Once ECD is estimated, Step 2 is essentially a problem of entropy estima-tion which can be tackled by many existing methods. Among those methods,the kNN method [14] was suggested in [9], which leads to a non-parametricway of estimating CE. Rigorously defined, CE has several properties which an ideal statistical in-dependence measure should have, including multivariate, symmetric, non-negative (0 iff independent), invariant to monotonic transformations, andequivalent to correlation coefficient in Gaussian cases.Theoretically, CE has many advantages over traditional association mea-sure – Correlation Coefficient (CC). Implied by definition, CC is a bivariatemeasure with Gaussian assumption while CE has no such limitations. Moretheoretical comparisons between CC and CE are listed in Table 1. SinceCE shows clear advantages over CC, it has been proposed as a method fordiscovering association relationships [15].
The data used in this paper were collected from 40 subjects recruited atTianjin, whose age range at 45-84. All the participants signed informed con-sent. All subjects were administrated to perform four types of test, includingTinetti POMA test, MMSE test, TUG test, and finger tapping test, twice aday for several times in one month. 5able 1: Comparisons between CC and CE.CC CELinearity Linear linear/Non-linearOrder Second AllAssumption Gaussian NoneDimensions bivariate mutlivariateAssociation Type correlation dependenceThe finger tapping test is based on the finger tapping device [16] whichmeasures finger motion movement with magnetic sensing technique. In eachtest, four modes of movement are measured: bimanual in-phase, bimanualunti-phase, left hand single, right hand single. For bimanual in-phase and bi-manual unti-phase movement, 84 attributes are derived, and for single handmovement, only 40 attributes are derived, as described in [17]. In the ex-periments, each movement lasts for 15 seconds. The characteristics of fingertapping test used in the following experiments include number of taps, aver-age interval of tapping, frequency of tapping and SD of interval of tappingof both hands of bi-inphase and bi-untiphase tapping, which lead to 16 char-acteristics most associated with MMSE score [3].The data on gait include 18 characteristics (mean and SD of the 9 char-acteristics listed in Table 2) extracted from video data with the method pro-posed in [18]. In detail, the video was recorded during TUG test and then 3Dpose was derived by combining 2D pose estimated from video and 3D depthinformation from 3D cameras. The gait characteristics were calculated from3D pose series of the whole video of each test.After excluding the subjects who did not complete all the four tests,we got 38 subjects and 134 tests totally. Each sample generated from thesetests composes of the scores of four tests, the characteristics of finger tappingand gait. In summary, the data includes 134 samples with 4 scores and 34characteristics. 6able 2: Gait characteristics extracted from video [18].Name DefinitionGait speed Speed of body movementSpeed variability standard deviation of stride speedsStride time time between one peak and the second-next peakStride time variability standard deviation of stride timesStride frequency median of modal frequency for the MLand half the modal frequencies for theV and AP directionsMovement intensity standard deviation of acceleration rateLow-frequency percentage Summed power up to a threshold fre-quency divided by total powerAcceleration range Difference between minimum andmaximum accelerationStep length (Pace) Length of one step
To study the relationship between finger tapping, gait, and fall risk, we con-duct an experiment to measure the associations between four scores, thecharacteristics of finger tapping and gait with CE from the above data. CE[9] is used in this research to measure the associations between characteris-tics of finger tapping and gait. CE is an ideal tool for studying statisticaldependence in this problem. It is estimated with non-parametric two stepmethod proposed in Section 2.2. The associations are identified based on theassociation strength measured by CE.As its application, we use the identified associations to improve the workson automatic TUG test [4]. We conduct an experiment to study whether thecharacteristics of finger tapping can be used to improve the models of predict-ing TUG score that we build in [4]. The characteristics of finger tapping thatare most associated with TUG score will be integrated into the predictivemodels to predict TUG score. As contrast, we also conduct an experimentto study the performance of the predictive models for predicting TUG score7ith only certain characteristics of finger tapping as input. Comparisons onthe predictive models with three groups of inputs (finger tapping only, gaitonly, and both) will be done to check whether integrating the characteris-tics of finger tapping and gait together can improve the performance of thepredictive models.In the prediction experiments, only the characteristics which are mostlyassociated with TUG score are considered, including ‘number of taps’ ofboth hands of bi-inphase from finger tapping test and 4 gait characteristics(including gait speed, pace, speed variance, acceleration range) from theTUG test which has been identified in the previous research [3, 4].The predictive models in the experiments are Linear Regression (LR)and Support Vector Regression (SVR) [19]. The ratio between training dataand test data are (80/20)% and the data set was randomly divided for 100times. The hyper-parameters of SVR are tuned to obtain the best possibleprediction results. The performance of the predictive models are measured byMean Absolute Error (MAE) between the true TUG scores and the predictedscores.Due to the imbalance deficiency of the MMSE scores in the current data,we can not study whether the characteristics of gait can be used to improvethe models of predicting MMSE score that we built in the previous work [3].
The associations between 4 scores and characteristics of finger tapping andgait measured by CE are shown in Figure 1. It can be learned from the Fig-ure that the number of taps of both hands of bi-inphase and bi-untiphase arewidely associated with the four scores, characteristics of both finger tappingand gait. Particularly, the association between number of taps and char-acteristics of gait is biologically meaningful which means the motor abilityand functional ability are associated. It can also be learned from the Figurethat number of taps, average interval of tapping, and frequency of tappingof both hand of bi-inphase or bi-untiphase are associated with each other,which confirmed our finding in the previous research [3].The associations between MMSE score and all the characteristics areshown in Figure 2. It can be learned from the Figure that certain charac-teristics, such as pace and stride time, are also associated with MMSE scorewith relatively strong strength.The associations between TUG score and all the characteristics are shown8 haracteristics of finger tapping and motion
Figure 1: Associations between four scores, 16 characteristics of finger tap-ping and 18 gait characteristics measured by CE.9
MSE C opu l a E n t r op y . . . . . l nu m be r r nu m be r l a v g i n t r a v g i n t l f r eq1 r f r eq1 li n t s d1 r i n t s d1 l nu m be r r nu m be r l a v g i n t r a v g i n t l f r eq2 r f r eq2 li n t s d2 r i n t s d2 s peedpa c e s peed_ v a r s t r i de_ t i m e s t r i de t i m e_ v a r a cc _ r ange m o v e m en t _ i n t en s i t y l o w _ f r eq_pe r s t r i de_ f r eq s peed_ s dpa c e_ s d s peed_ v a r _ s d s t r i de_ t i m e_ s d s t r i de t i m e_ v a r _ s da cc _ r ange_ s d m o v e m en t _ i n t en s i t y _ s d l o w _ f r eq_pe r _ s d s t r i de_ f r eq_ s d Figure 2: Associations of MMSE score with 16 characteristics of finger tap-ping and 18 gait characteristics. 10able 3: Comparison on prediction performance (MAE) between finger tap-ping, gait or both as inputs of the models.LR SVRFinger tapping 1.520 1.355Gait 1.280 1.168Both 1.260 in Figure 3. It can be learned from it that number of taps of both hands ofbi-inphase and bi-untiphase are associated with TUG score and the strengthof the associations are even stronger than that of gait characteristics. Thisis an interesting finding which inspires us to use number of taps to improvethe models for predicting TUG score with characteristics of gait.The joint distribution of TUG score and number of taps is shown inFigure 4, from which it can be learned that the associations between themare highly nonlinear. This suggests that CE, as a nonlinear measure, is theright choice for measuring such associations.We conducted 6 experiments to predict TUG score with different com-binations of two models (LR and SVR) and three groups of characteristics(number of taps only, gait only, and both). The prediction results of bothmodels are shown in Figure 5(a) and 5(b). It can be learned from bothFigures that the prediction with both types of characteristics is considerablebetter than with only one. We measured the performance of 6 experimentsin terms of MAE, as listed in Table 3. It can be learned from it that: 1) SVRwith both characteristics presents the best result with the smallest MAE(=1.159); 2) the performance of SVR is better than that of LR on all thethree groups of characteristics; 3) the improvements on both LR and SVRwith both characteristics are obvious compared with those with only charac-teristics of finger tapping or gait.
Figure 1 shows that number of taps and most of gait characteristics are as-sociated, which suggests finger motor ability and gait ability are related. Tothe best of our knowledge, there is no previous research reporting this rela-tionship. Nagasaki et al examined walking patterns and rhythmic movement11 UG C opu l a E n t r op y . . . . . l nu m be r r nu m be r l a v g i n t r a v g i n t l f r eq1 r f r eq1 li n t s d1 r i n t s d1 l nu m be r r nu m be r l a v g i n t r a v g i n t l f r eq2 r f r eq2 li n t s d2 r i n t s d2 s peedpa c e s peed_ v a r s t r i de_ t i m e s t r i de t i m e_ v a r a cc _ r ange m o v e m en t _ i n t en s i t y l o w _ f r eq_pe r s t r i de_ f r eq s peed_ s dpa c e_ s d s peed_ v a r _ s d s t r i de_ t i m e_ s d s t r i de t i m e_ v a r _ s da cc _ r ange_ s d m o v e m en t _ i n t en s i t y _ s d l o w _ f r eq_pe r _ s d s t r i de_ f r eq_ s d Figure 3: Associations of TUG score with 16 characteristics of finger tappingand 18 gait characteristics. 12 number1
10 30 50 70 l l ll lllllll llll lllll l ll ll l l ll l llllll lll llll lll ll ll lll llll ll llllll lll l lll l ll l llll ll l lllll llll lll l lll lll llll lll lll l lllllll lll llllll ll l ll l l lllllll lll ll l ll lll ll llllll l llllllll l l ll ll ll lll l l llllll l ll llll ll lll llll lllllll l llll llll llll ll lllll llll ll llllllll lll ll lll l ll ll ll l llll l
10 30 50 ll lllllll lll lll ll lll ll llllll l llllllll l l ll ll ll llll l llllll l ll llll ll lll ll ll lllllll l llll lllll lll ll lllll llll llllll lllllll ll ll ll ll ll ll l llll l l ll llll llllll ll ll lllll l l lll ll lll ll lll lll ll llll lllll lll ll llllll ll lll lllll ll lll lll llllllll llll lll ll llll llllll lll ll llll lllll llll lll lllll l l l lllllllll llll lllll lllll ll lll llllll lll l l ll lll ll lllll llllll ll llll lll llll llll llll ll l lllll lllll ll llll lllllll lll lll l llll lll lll lllllllll ll rnumber1 l l lllllll lll ll l ll lll ll llllll l llllllll l l ll ll ll lll l l llllll l ll llll ll lll llll lllllll l llll llll llll ll lllll llll ll llllllll lll ll lll l ll ll ll l llll l ll lllllll lll lll ll lll ll llllll l llllllll l l ll ll ll llll l llllll l ll llll ll lll ll ll lllllll l llll lllll lll ll lllll llll llllll lllllll ll ll ll ll ll ll l llll l l ll llll llllll ll ll lllll l l lll ll lll ll lll lll ll llll lllll lll ll llllll ll lll lllll ll lll lll llllllll llll lll ll llll llllll lll ll llll lllll llll lll lllll ll l lllllllll llll lllll lllll ll lll llllll lll l l ll lll ll lllll llllll ll llll lll llll llll llll ll l lllll lllll ll llll lllllll lll lll l llll lll lll lllllllll ll l l ll lllllll llll lllll l ll ll l l ll l llllll lll llll lll ll ll lll llll ll llllll lll l lll l ll l llll ll l lllll llll lll l lll lll llll lll lll l lllllll lll llllll ll l ll lnumber2 ll lllllll lll lll ll lll ll llllll l llllllll l l ll ll ll llll l llllll l ll llll ll lll ll ll lllllll l llll lllll lll ll lllll llll llllll lllllll ll ll ll ll ll ll l llll l l ll llll llllll ll ll lllll l l lll ll lll ll lll lll ll llll lllll lll ll llllll ll lll lllll ll lll lll llllllll llll lll ll llll llllll lll ll llll lllll llll lll lllll l l l lllllllll llll lllll lllll ll lll llllll lll l l ll lll ll lllll llllll ll llll lll llll llll llll ll l lllll lllll ll llll lllllll lll lll l llll lll lll lllllllll ll l l ll lllllll llll lllll l ll ll l l ll l llllll lll llll lll ll ll lll llll ll llllll lll l lll l ll l llll ll l lllll llll lll l lll lll llll lll lll l lllllll lll llllll ll l ll l l lllllll lll ll l ll lll ll llllll l llllllll l l ll ll ll lll l l llllll l ll llll ll lll llll lllllll l llll llll llll ll lllll llll ll llllllll lll ll lll l ll ll ll l llll l rnumber2 l ll llll llllll ll ll lllll l l lll ll lll ll lll lll ll llll lllll lll ll llllll ll lll lllll ll lll lll llllllll llll lll ll llll llllll lll ll llll lllll llll lll lllll l
10 30 50 70 l l lllllllll llll lllll lllll ll lll llllll lll l l ll lll ll lllll llllll ll llll lll llll llll llll ll l lllll lllll ll llll lllllll lll lll l llll lll lll lllllllll ll l l ll lllllll llll lllll l ll ll l l ll l llllll lll llll lll ll ll lll llll ll llllll lll l lll l ll l llll ll l lllll llll lll l lll lll llll lll lll l lllllll lll llllll ll l ll
10 30 50 l l lllllll lll ll l ll lll ll llllll l llllllll l l ll ll ll lll l l llllll l ll llll ll lll llll lllllll l llll llll llll ll lllll llll ll llllllll lll ll lll l ll ll ll l llll l ll lllllll lll lll ll lll ll llllll l llllllll l l ll ll ll llll l llllll l ll llll ll lll ll ll lllllll l llll lllll lll ll lllll llll llllll lllllll ll ll ll ll ll ll l llll l tug
Figure 4: Joint distribution between TUG score and four characteristicsof finger tapping (‘number of taps’ of both hands of bi-inphase and bi-untiphase). 13 ll ll l l ll lll llll lll ll ll l lll LR True TUG P r ed i c t i on l finger tappinggaitboth (a) LR l ll ll l l ll lll llll lll ll ll l lll SVR
True TUG P r ed i c t i on l finger tappinggaitboth (b) SVR Figure 5: Performance of the predictive models (LR and SVR) with 3 groupsof characteristics as inputs. 14f fingers of older adults [20]. Hausdorff et al found that even walking andtapping are both automated, rhythmic motor task, the former is shown sur-prisingly related to catching rather than the latter [21], which provides anopinion opposite to ours. In Figure 3, number of taps is also shown strongassociation with TUG score, which indicate that number of taps is a predic-tor for fall risk. So it is reasonable to hypothesis that finger motor abilitycan help to predict fall risk, as this paper does.Figure 2 shows that MMSE is also associated with the gait characteris-tics, such as pace, speed variance and stride time, which implies the rela-tionship between gait and cognition. This result provides another evidencethat gait disorder is related to cognitive impairment [6, 7, 8]. Beauchet et al[22] reported stride variability is more specific and sensitive in subjects withdementia, which gain moderate support from our experimental results.In our experiments, the performance of two models are both improved bycombining the two characteristics together as inputs of the models. Addi-tionally, the performance of the two models in terms of MAE are different,particularly SVR presents better results than LR does. This may be becausethat the relationship between the characteristics and TUG score are nonlin-ear and hence the SVR with nonlinear models can perform well. SVR withboth types of characteristics presents the best result which suggest that bothcharacteristics are informative for this prediction task. Figure 4 supportsthis claim.Remember that the data used in the experiments are unbalanced with afew samples from patients with high fall risk. This may lead to the mod-els tending to predict more positive results rather than fall risk and hencemake the MAE unreliable to some extent. Therefore, the performance of themodels should be improved with more patients data in the future.
In this paper, we jointly analyze with CE the finger tapping data and gaitcharacteristics data collected from our previous research. The associationsbetween certain finger tapping characteristics, gait characteristics, and TUGscore are discovered. These associations are then applied to automatic fallrisk assessment to improve the performance of the predictive models on TUGscore prediction. Experimental results show that integrating the finger tap-ping characteristics into these predictive models can considerably improve15he prediction performance of the models in terms of MAE. Such associa-tions is an evidence on the relationship between gait, finger motor ability,and functional ability, which lay the scientific foundation of the predictivemodels.
Acknowledgements
The author thanks Zhang Pan and Yin Ying for providing data.