Differential Privacy for Eye Tracking with Temporal Correlations
Efe Bozkir, Onur Günlü, Wolfgang Fuhl, Rafael F. Schaefer, Enkelejda Kasneci
DDifferential Privacy for Eye Tracking with Temporal Correlations
Efe Bozkir ∗† Onur G ¨unl ¨u ∗‡ Wolfgang Fuhl † Rafael F. Schaefer ‡ Enkelejda Kasneci † Human-Computer Interaction, University of T ¨ubingen Information Theory and Applications Chair, TU Berlin
Abstract
New generation head-mounted displays, such as VR andAR glasses, are coming into the market with already inte-grated eye tracking and are expected to enable novel ways ofhuman-computer interaction in many applications. However,since eye movement properties contain biometric informa-tion, privacy concerns have to be handled properly. Privacy-preservation techniques such as differential privacy mecha-nisms have recently been applied to the eye movement dataobtained from such displays. Standard differential privacymechanisms; however, are vulnerable to temporal correla-tions in the eye movement features. In this work, we proposea novel transform-coding based differential privacy mecha-nism to further adapt it to the statistics of eye movementfeature data by comparing various low-complexity methods.We extent Fourier Perturbation Algorithm, which is a differ-ential privacy mechanism, and correct a scaling mistake inits proof. Furthermore, we illustrate significant reductions insample correlations in addition to query sensitivities, whichprovide the best utility-privacy trade-off in the eye trackingliterature. Our results show significantly high privacy withoutloss in classification accuracies as well.
Introduction
Recent advances in the field of head-mounted displays(HMDs) and eye tracking enable easy access to pervasiveeye trackers along with modern HMDs. Soon, the decreasein the cost of such devices might cause a mass consump-tion across different application domains such as gaming,entertainment, or education. Consequently, we expect a sig-nificant increase in the amount of eye movement data col-lected from the users. A large part of this data is indeeduseful for personalized experience and user-adaptive inter-action. In virtual and augmented reality (VR/AR) especially,it is possible to derive plenty of sensitive information aboutusers from the eye movement data. For instance, it has beenshown that eye tracking signals can be employed for activ-ity recognition even in challenging everyday tasks (Steil andBulling 2015; Braunagel et al. 2017; Ishimaru et al. 2014),to detect cognitive load (Appel et al. 2018; Krejtz et al. ∗ Equally contributed † efe.bozkir,wolfgang.fuhl,[email protected] ‡ guenlue,[email protected] a r X i v : . [ c s . CR ] S e p eously. For eye movement data collected from HMDs orsmart glasses, both local and global differential privacy canbe applied. Local differential privacy adds user level noiseto the data but assumes that the user sends data to a cen-tral data collector after adding local noise (Erlingsson, Pihur,and Korolova 2014; Ding, Kulkarni, and Yekhanin 2017).For this work, we consider global differential privacy, be-cause there is a central user-level data collector and pub-lisher in a VR/AR setting.To apply differential privacy to the eye movement data,we evaluate the standard Laplacian Perturbation Algorithm(LPA) (Dwork et al. 2006) and Fourier Perturbation Algo-rithm (FPA) (Rastogi and Nath 2010). The latter is suitablefor time series data such as the eye movement feature sig-nals. We propose two different methods that apply the FPAto chunks of data using original eye movement feature sig-nals or consecutive difference signals. While preserving dif-ferential privacy using parallel compositions, chunk-basedmethods decrease query sensitivity and computational com-plexity. The difference-based method further decreases thetemporal correlations between the eye movement features inaddition to the decorrelation provided by the FPA that usesthe discrete Fourier transform (DFT) as, e.g., in (G¨unl¨u andIscan 2014; G¨unl¨u et al. 2018). The difference-based methodprovides a higher level of privacy since consecutive sam-ple differences are observed to be less correlated than origi-nal consecutive data. Furthermore, we evaluate our methodsusing differentially private eye movement features in docu-ment type and gender classification, and privacy sensitivityclassification tasks by using similar configurations to previ-ous works in (Steil et al. 2019a,b). To generate differentiallyprivate eye movement data, we use the complete data insteadof applying a subsampling step, used in (Steil et al. 2019a)to reduce the sensitivity and to improve the classificationaccuracies. In addition, the previous work in (Steil et al.2019a) applies the exponential mechanism for differentialprivacy on the eye movement feature data. The exponentialmechanism is useful for situations where the best enumer-ated response needs to be chosen (Dwork and Roth 2014). Ineye movements, we are not interested in the “best” responsebut in the feature vector. Therefore, we apply the Laplacianmechanism. In summary, we are the first to propose differen-tial privacy solutions for eye movements by taking the tem-poral correlations into account, which can help provide userprivacy especially for HMD or smart glass usage.Our main contributions are as follows. (1) We proposechunk-based and difference-based differential privacy meth-ods for eye movement features to reduce query sensitivities,computational complexity, and temporal correlations. Fur-thermore, (2) we evaluate our methods on two publicly avail-able eye movement datasets, i.e., MPIIDPEye (Steil et al.2019a) and MPIIPrivacEye (Steil et al. 2019b), by compar-ing them with standard techniques such as LPA and FPA us-ing the multiplicative inverse of the normalized mean squareerror (NMSE) as the utility metric. In addition, we evaluatedocument type and gender classification, and privacy sen-sitivity classification accuracies as classification metrics us-ing differentially private eye movements in MPIIDPEye andMPIIPrivacEye datasets, respectively. Our results show sig- nificantly better performance as compared to previous worksand are capable of handling correlated data and decreasingquery sensitivities by dividing the data into smaller chunks. Related Work
There are few works that focus on privacy-preserving eyetracking. (Liebling and Preibusch 2014) provides motivationas to why privacy considerations are needed for eye track-ing data by focusing on gaze and pupillometry. Practicalsolutions are; therefore, introduced to protect user identityand sensitive stimuli based on a degraded iris authentica-tion through optical defocus (John, Koppal, and Jain 2019)and an automated disabling mechanism for the eye tracker’sego perspective camera with the help of a mechanical shut-ter depending on the detection of privacy sensitive content(Steil et al. 2019b). Furthermore, a function-specific pri-vacy model for privacy-preserving gaze estimation task andprivacy-preserving eye videos by replacing the iris texturesare proposed in (Bozkir et al. 2020) and (Chaudhary andPelz 2020), respectively. For the user identity protection,works that focus on differential privacy are more relevantfor us. Recently, standard differential privacy mechanismsare applied to eye movements in VR (Steil et al. 2019a) andheatmaps (Liu et al. 2019). These works do not address theeffects of temporal correlations in eye movements over timein the privacy context. In the privacy literature, there are pri-vacy definitions such as the Pufferfish mechanism (Kifer andMachanavajjhala 2014) or the Olympus framework (Raval,Machanavajjhala, and Pan 2019) for correlated data. Theseworks, however, have different assumptions. For example,Pufferfish requires a domain expert to specify potential se-crets and discriminative pairs, and Olympus models privacyand utility requirements as adversarial networks. As our fo-cus is to protect user identity in the eye movements, we optfor differential privacy by discussing the effects of temporalcorrelations in eye movements over time and propose meth-ods to reduce them.
Theoretical Background
Differential privacy uses a metric to measure the privacy riskfor an individual participating in a database. Considering adataset with weights of N people and a mean function, whenan adversary queries the mean function for N people, the av-erage weight over N people is obtained. After the first query,an additional query for N − people automatically leaks theweight of the remaining person. Using differential privacy,noise is added to the outcome of a function so that the out-come does not significantly change based on whether or nota randomly chosen individual participated in the dataset. Theamount of noise added should be calibrated carefully sincea high amount of noise might decrease the utility. We nextdefine differential privacy. Definition 1 (cid:15) -Differential Privacy ( (cid:15) -DP) (Dwork et al.2006).
A randomized mechanism M is (cid:15) -differentially pri-vate if for all databases D and D (cid:48) that differ at most in oneelement for every S ⊆ Range ( M ) , we have Pr[ M ( D ) ∈ S ] ≤ e (cid:15) Pr[ M ( D (cid:48) ) ∈ S ] . (1)he variance of the added noise depends on the query sen-sitivity, which is defined as follows. Definition 2
Query sensitivity (Dwork et al. 2006).
For arandom query X n and w ∈ { , } , the query sensitivity ∆ w of X n is the smallest number for all databases D and D (cid:48) that differ at most in one element such that || X n ( D ) − X n ( D (cid:48) ) || w ≤ ∆ w ( X n ) (2) where the L w -distance is defined as || X n || w = w (cid:118)(cid:117)(cid:117)(cid:116) n (cid:88) i =1 (cid:0) | X i | (cid:1) w . (3)We list theorems that are used in the proposed methods. Theorem 1
Sequential Composition Theorem (McSherry2009). Consider n independent mechanisms M i for i = 1 , , ..., n . If M , M , ..., M n are (cid:15) , (cid:15) , ..., (cid:15) n -differentially private, respectively, then their joint mecha-nism is (cid:32) n (cid:88) i =1 (cid:15) i (cid:33) -differentially private. Theorem 2
Parallel Composition Theorem (McSherry2009). Consider n mechanisms as M i for i = 1 , , ..., n that are applied to disjoint subsets of a dataset. If M , M , ..., M n are (cid:15) , (cid:15) , ..., (cid:15) n -differentially private,respectively, then their joint mechanism is (cid:18) max i ∈ [1 ,n ] (cid:15) i (cid:19) -differentially private. We define the Laplacian Perturbation Algorithm (LPA)(Dwork et al. 2006). To guarantee differential privacy, theLPA generates the noise according to a Laplace distri-bution.
Lap ( λ ) denotes a random variable drawn froma Laplace distribution with a probability density function(PDF): Pr[
Lap ( λ ) = h ] = λ e −| h | /λ , where Lap ( λ ) haszero mean and variance λ . We denote the noisy and dif-ferentially private values as (cid:101) X i = X i ( D ) + Lap ( λ ) for i = 1 , , . . . , n . Since we have a series of eye move-ment observations, the final noisy eye movement observa-tions are generated as (cid:101) X n = X n ( D ) + Lap n ( λ ) , where Lap n ( λ ) is a vector of n independent Lap ( λ ) random vari-ables and X n ( D ) is the eye movement observations with-out noise. The LPA algorithm is (cid:15) -differentially private for λ = ∆ ( X n ) /(cid:15) (Dwork et al. 2006).We define the error function that we use to measure thedifferences between original X n and noisy (cid:101) X n observa-tions. For this purpose, we use the metric normalized meansquare error (NMSE) defined asNMSE = 1 n n (cid:88) i =1 ( X i − (cid:101) X i ) X (cid:101) X (4)where X = 1 n n (cid:88) i =1 X i , (cid:101) X = 1 n n (cid:88) i =1 (cid:101) X i . (5) We define the utility metric asUtility = 1 NMSE . (6)As differential privacy is achieved by adding randomnoise to the data, there is a utility-privacy trade-off. Toomuch noise will lead to high privacy; however, it might alsoresult in poor analyses on the further tasks on eye move-ments. Therefore, it is important to find a good trade-off. Methods
Standard differential privacy mechanisms are vulnerable totemporal correlations, since the independent noise realiza-tions that are added to temporally correlated data could beuseful for adversaries. However, decorrelating the data be-fore adding the noise might remove important eye move-ment patterns and provide poor results in analyses. Many eyemovement features are extracted by using time windows, asin (Steil et al. 2019a,b), which makes the features highly cor-related. Another challenge is that the duration of eye track-ing recordings could change depending on the skills or per-sonalities of the users. The longer duration causes an in-creased query sensitivity, which means that higher amountsof noise should be added to achieve differential privacy. Inaddition, when the data is correlated, as in (Zhao, Zhang, andPoor 2017), (cid:15) (cid:48) is defined as the actual privacy metric that isobtained considering the fact that correlations can be usedto obtain more information about the differentially privatedata by filtering, instead of (cid:15) . In this work, we discuss andpropose generic low-complexity methods to keep (cid:15) (cid:48) smallfor eye movement feature signals. To deal with correlatedeye movement feature data, we propose three different meth-ods: FPA, chunk-based FPA (CFPA) for original signal, andchunk-based FPA for difference based sequences (DCFPA).The sensitivity of each eye movement feature signal is cal-culated by using the L w -distance such that ∆ fw ( X n ) = max p, q (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) X n, ( p,f ) − X n, ( q,f ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) w = max p, q w (cid:118)(cid:117)(cid:117)(cid:116) n (cid:88) t =1 (cid:16)(cid:12)(cid:12)(cid:12) X ( p,f ) t − X ( q,f ) t (cid:12)(cid:12)(cid:12)(cid:17) w (7)where X n, ( p,f ) and X n, ( q,f ) denote observation vectors fora feature f from two participants p and q , n denotes the max-imum length of the observation vectors, and w ∈ { , } . Fourier Perturbation Algorithm (FPA)
In the FPA, the signal is represented with a small num-ber of transform coefficients such that the query sensitiv-ity of the representative signal decreases. A smaller querysensitivity decreases the noise power required to make thenoisy signal differentially private. In the FPA, the signal istransformed into the frequency domain by applying DiscreteFourier Transform (DFT), which is commonly applied asa non-unitary transform. The frequency domain representa-tion of a signal consists of less correlated transform coeffi-cients as compared to the time domain signal due to the highecorrelation efficiency of the DFT. Therefore, the correla-tion between the eye movement feature signals is reduced byapplying the DFT. After the DFT, the noise sampled fromthe LPA is added to the first k elements of DF T ( X n ) thatcorrespond to k lowest frequency components, denoted as F k = DF T k ( X n ) . Once the noise is added, the remainingpart (of size n − k ) of the noisy signal (cid:101) F k is zero paddedand denoted as P AD n ( (cid:101) F k ) . Lastly, using the Inverse DFT(IDFT), the padded signal is transformed back into the timedomain. We can show that (cid:15) -differential privacy is satisfiedby the FPA for λ = √ n √ k ∆ ( X n ) (cid:15) unlike the value claimed in(Rastogi and Nath 2010), as observed independently in (Kel-laris and Papadopoulos 2013). The procedure is summarizedin Algorithm 1. Since not all coefficients are used, in addi-tion to the perturbation error caused by the added noise, areconstruction error caused by the lossy compression is in-troduced. It is important to determine the number of usedcoefficients k to minimize the total error. We discuss howwe choose k values for FPA-based methods below. Algorithm 1:
Fourier Perturbation Algorithm (FPA).
Inputs: X n , λ , k Output: (cid:101) X n F k = DF T k ( X n ) .2) (cid:101) F k = LP A ( F k , λ ) .3) (cid:101) X n = IDF T ( P AD n ( (cid:101) F k )) . Chunk-based FPA (CFPA)
One drawback of directly applying the FPA to the eye move-ment feature signals is large query sensitivities for eachfeature f due to long signal sizes. To solve this, (Steilet al. 2019a) proposes to subsample the signal using non-overlapping windows, which means removing many datapoints. While subsampling decreases the query sensitivities,it also decreases the amount of data. Instead, we propose tosplit each signal into smaller chunks and apply the FPA toeach chunk so that complete data can be used. We choosethe chunk sizes of , , and since there are divide-and-conquer type tree-based implementation algorithms forfast DFT calculations when the transform size is a power of . When the signals are split into chunks, chunk level querysensitivities are calculated and used rather than the sensi-tivity of the whole sequence. Differential privacy for thecomplete signal is preserved by Theorem 2 since the chunksare non-overlapping. As the chunk size decreases, the chunklevel sensitivity decreases as well as the computational com-plexity. However, the parameter (cid:15) (cid:48) that accounts for the sam-ple correlations might increase with smaller chunk sizes be-cause correlations between neighboring samples are largerin an eye movement dataset. Therefore, a good trade-off be-tween computational complexity and correlations is neededto determine the optimal chunk size. Difference- and chunk-based FPA (DCFPA)
To tackle temporal correlations, we convert the eye move-ment feature signals into difference signals where differ-ences between consecutive eye movement features are cal-culated as (cid:98) X ( f ) t = (cid:110) X ( f ) t − X ( f ) t − (cid:111)(cid:12)(cid:12)(cid:12) nt =2 , (cid:98) X ( f )1 = X ( f )1 . (8)Using the difference signals denoted by (cid:98) X n, ( f ) , we aimto further decrease the correlations before applying a differ-ential privacy method. We conjecture that the ratio (cid:15) (cid:48) /(cid:15) de-creases in the difference-based method as compared to theFPA method. To support this conjecture, we show that thecorrelations in the difference signals decrease significantlyas compared to the original signals. This results in lower (cid:15) (cid:48) and better privacy for the same (cid:15) . The difference basedmethod is applied together with the CFPA. Therefore, thedifferences are calculated inside chunks. The first elementof each chunk is preserved. Then, the FPA mechanism is ap-plied to the difference signals by using query sensitivitiescalculated based on differences and chunks. For each chunk,noisy difference signals are aggregated to obtain the finalnoisy signals. This mechanism is differentially private byTheorem 1. Since Theorem 1 can be applied to the DCFPAwhen consecutive differences are assumed to be indepen-dent, which is a valid assumption for eye movement featuredata as we illustrate below, there is also a trade-off betweenthe chunk sizes and utility for the DCFPA. If a large chunksize is chosen, then the total (cid:15) value could be very large,which reduces privacy. Therefore, we choose chunk sizes of , , and for the DCFPA as well for evaluation. TheDCFPA is summarized in Algorithm 2. Algorithm 2:
DCFPA
Inputs: X n , λ , k Output: (cid:101) X n (cid:98) X t = (cid:110) X t − X t − (cid:111)(cid:12)(cid:12)(cid:12) nt =2 , (cid:98) X = X .2) (cid:101)(cid:98) X n = F P A ( (cid:98) X n , λ, k ) .3) (cid:102) X t = (cid:110)(cid:102)(cid:98) X t + (cid:103)(cid:98) X t − (cid:111)(cid:12)(cid:12)(cid:12) nt =2 , (cid:101) X = (cid:102)(cid:98) X . Choice of the Number of Transform Coefficients
The proposed methods require a selection of a value for k .A small k value increases the reconstruction error, while alarge k value results in an increase in the perturbation error.Therefore, it is important to find the best k value that mini-mizes the sum of the two errors. In this work, we compare alarge set of possible k values to choose the best values.We apply the aforementioned differential privacy mecha-nisms by using noisy evaluations to find optimal k val-ues applied to features or chunks. Optimal k values havethe minimum NMSE for each chunk, eye movement fea-ture, and document or recording type. In a distributed set-ting, each user needs to know k values in advance. However,in a centralized setting, it is crucial to choose the k values in a) Correlation coefficients of original signals in MPIIDPEye. (b) Correlation coefficients of difference signals in MPIIDPEye. Figure 1: Correlation coefficients of the feature ratio large saccade in MPIIDPEye dataset for three document types over a timedifference of ∆ t (Each time step corresponds to . s) w.r.t. the samples at the fifth time instance.a differentially private manner. To evaluate the differentialprivacy in the eye tracking area while taking the temporalcorrelations into account, we focus on optimal k values forthis work. One shortcoming of this approach is that the opti-mal k value compromises some information about the data,which leaks privacy (Rastogi and Nath 2010). Our observa-tion is that the information leaked by optimizing the param-eter k is negligible as compared to the privacy reduction dueto correlated data. Thus, we illustrate the results with opti-mal k values. Evaluations
This section discusses datasets, and evaluations using utilityand classification metrics. The results are averaged over noisy evaluations with the optimal k values in MATLAB. Datasets
MPIIDPEye (Steil et al. 2019a):
A publicly available eyemovement dataset consisting of recordings dedicated toprivacy-preserving eye tracking that is collected in VR fora reading task of three document types (comic, newspaper,and textbook) from ( female, male) participants.Each recording consists of eye movement feature se-quences computed with a sliding window size of secondsand a step size of . seconds. MPIIPrivacEye (Steil et al. 2019b):
A publicly availableeye movement dataset consisting of recordings from participants with different sessions after each other withan head-mounted eye tracker and a field camera, which issimilar to an AR setup. Each recording consists of eyemovement feature sequences computed with a sliding win-dow size of seconds and a step size of second and eachobservation is annotated with binary privacy sensitivity lev-els of the scene that is being viewed. The dataset also con-sists of scene features extracted with CNNs. We do not eval-uate the last part of the recording of the participant , asthe eye movement features are not available for this region.To detect the privacy level of the scene that is being viewed,we acknowledge that information about scene is very impor-tant (Orekondy, Schiele, and Fritz 2017); however, an indi-vidual’s eye movements can improve the detection rate.We first show the data correlation using correlation co-efficients obtained from the eye movement features. Sincethere are eye movement features in both datasets, it is not feasible to show them all. Thus, in the following we il-lustrate the correlation problem based on the feature called ratio large saccade in the MPIIDPEye dataset. The correla-tion coefficients of ratio large saccade for three documenttypes over a time difference ∆ t w.r.t the signal samples at,e.g., the fifth time instance for original eye movement fea-ture signals and difference signals for all participants are de-picted in Figures 1 (a) and (b), respectively. As correlationsbetween the difference signals are significantly smaller thancorrelations between the original eye movement feature sig-nals, the DCFPA is less vulnerable to privacy reduction dueto temporal correlations, thus affecting the value of (cid:15) (cid:48) . Addi-tionally, as all minimum values of wordbook features from to are zeros in both datasets, we exclude them from theutility and total (cid:15) calculations. Utility Results
We evaluate the utility given in Equation (6) by applyingour methods separately to different document and record-ing types; therefore, we report the utility results separately.As we apply the proposed methods separately to each eyemovement feature, we first calculate the mean utility of eachfeature and then calculate the average utility over all fea-tures. The utility results for various (cid:15) values for aforemen-tioned methods on MPIIDPEye and MPIIPrivacEye datasetsare given in Figures 2 and 3, respectively.While a high NMSE, i.e., low utility, does not necessar-ily mean that the model is completely useless, higher util-ity means that the model would perform more effectivelythan low utility in various tasks. The utility results of bothevaluated datasets are similar. As the query sensitivities arelower in CFPA, utilities of CFPA are always higher than theutilities of the FPA as theoretically expected. DCFPA par-ticularly outperforms other methods in the most private set-tings, namely in the lowest (cid:15) regions. When different chunksizes are compared within the CFPA and DCFPA, differentchunk sizes perform similarly for the CFPA method. For theDCFPA, there is a significant trend for better utilities whenthe chunk sizes are decreased. Since a higher chunk size re-duces the temporal correlations better, it is ideal to use ahigher chunk size if the utilities are comparable. While theLPA, namely the standard Laplacian mechanism of the dif-ferential privacy, is vulnerable to temporal correlations, ourmethods also outperform it in terms of utilities. In additionto high utilities, the calculation complexities are decreased a) Utility of the LPA and FPA for MPIIDPEye.(b) Utility of the CFPA for MPIIDPEye.(c) Utility of the DCFPA for MPIIDPEye.
Figure 2: Utility results for MPIIDPEye dataset.with the CFPA and DCFPA which is another advantage ofchunk-based methods.
Classification Accuracy Results
We evaluate document type and gender classification resultsfor the MPIIDPEye and privacy sensitivity classification re-sults for the MPIIPrivacEye by using differentially privatedata generated by the methods which handle temporal cor-relations in the privacy context. Instead of evaluating onlySupport Vector Machines (SVM) as in previous works (Steilet al. 2019a,b), we evaluate a set of classifiers includingSVMs, decision trees (DT), random forests (RF), and k-Nearest Neighbors (k-NN). We employ a similar setup as in(Steil et al. 2019a) with radial basis function (RBF) kernel,bias parameter of C = 1 , and automatic kernel scale for theSVMs. For RFs and k-NNs, we use trees and k = 11 witha random tie breaker among tied groups, respectively. Wenormalize the training data to zero mean and unit variance,and apply the same parameters to the test data. Althoughwe do not apply subsampling while generating the differen-tially private data, which is applied in (Steil et al. 2019a),we use subsampled data for training and testing with win- (a) Utility of the LPA and FPA for MPIIPrivacEye.(b) Utility of the CFPA for MPIIPrivacEye.(c) Utility of the DCFPA for MPIIPrivacEye. Figure 3: Utility results for MPIIPrivacEye dataset.dow sizes of and for MPIIDPEye and MPIIPrivacEye,respectively, to have a fair comparison and similar amount ofdata. All the classifiers are trained and tested in a leave-one-person-out cross-validation setup, which is considered as amore challenging but generic setup. For the MPIIDPEye, weevaluate results both with majority voting by summarizingclassifications from different time instances for each partic-ipant and without majority voting. For the MPIIPrivacEye,it is not reasonable to use majority voting as each recordingcan include both privacy sensitive and non-sensitive stimuli.While classification results cannot be treated directly asthe utility, they provide insights into the usability of thedifferentially private data. We first evaluate document typeclassification task in the majority voting setting in Table 1as it is possible to compare our results with the previouswork (Steil et al. 2019a). As previous results quickly dropto the . guessing probability in high privacy regions, wesignificantly outperform them particularly with DCFPA andFPA with the accuracies over . and . , respectively. Inthe less private regions towards (cid:15) = 48 , this trend still ex-ists with the CFPA and FPA with accuracy results over . and . . Chunk-based methods perform slightly worse thanocument Type Classification Accuracies (k-NN | SVM | DT | RF)Method (cid:15) = 0 . (cid:15) = 2 . (cid:15) = 4 . (cid:15) = 24 (cid:15) = 48 FPA . | . | . | . . | . | . | . . | . | . | . . | . | . | . . | . | . | . CFPA-32 . | . | . | .
43 0 . | . | . | .
44 0 . | . | . | .
44 0 . | . | . | . . | . | . | . CFPA-64 . | . | . | .
44 0 . | . | . | .
44 0 . | . | . | .
44 0 . | . | . | .
60 0 . | . | . | . CFPA-128 . | . | . | .
45 0 . | . | . | .
45 0 . | . | . | .
45 0 . | . | . | .
56 0 . | . | . | . DCFPA-32 . | . | . | .
43 0 . | . | . | .
42 0 . | . | . | .
43 0 . | . | . | .
44 0 . | . | . | . DCFPA-64 . | . | . | .
41 0 . | . | . | .
40 0 . | . | . | . . | . | . | .
42 0 . | . | . | . DCFPA-128 . | . | . | . . | . | . | . . | . | . | .
46 0 . | . | . | .
47 0 . | . | . | . Table 1: Document type classification accuracies in MPIIDPEye using differentially private eye movement features with ma-jority voting. Gender Classification Accuracies (k-NN | SVM | DT | RF)Method (cid:15) = 0 . (cid:15) = 2 . (cid:15) = 4 . (cid:15) = 24 (cid:15) = 48 FPA . | . | . | .
37 0 . | . | . | .
39 0 . | . | . | .
38 0 . | . | . | .
40 0 . | . | . | . CFPA-32 . | . | . | .
25 0 . | . | . | .
25 0 . | . | . | .
25 0 . | . | . | .
45 0 . | . | . | . CFPA-64 . | . | . | .
26 0 . | . | . | .
26 0 . | . | . | .
27 0 . | . | . | .
45 0 . | . | . | . CFPA-128 . | . | . | .
31 0 . | . | . | .
30 0 . | . | . | .
32 0 . | . | . | .
46 0 . | . | . | . DCFPA-32 . | ≈ | . | .
33 0 . | ≈ | . | .
32 0 . | ≈ | . | .
31 0 . | ≈ | . | .
31 0 . | ≈ | . | . DCFPA-64 . | ≈ | . | .
33 0 . | ≈ | . | .
34 0 . | ≈ | . | .
34 0 . | ≈ | . | .
33 0 . | ≈ | . | . DCFPA-128 . | . | . | .
34 0 . | ≈ | . | .
33 0 . | ≈ | . | .
35 0 . | ≈ | . | .
34 0 . | ≈ | . | . Table 2: Gender classification accuracies in MPIIDPEye using differentially private eye movement features with majorityvoting. Privacy Sensitivity Classification Accuracies (k-NN | SVM | DT | RF)Method (cid:15) = 0 . (cid:15) = 2 . (cid:15) = 4 . (cid:15) = 24 (cid:15) = 48 FPA . | . | . | .
55 0 . | . | . | .
55 0 . | . | . | .
55 0 . | . | . | .
55 0 . | . | . | . CFPA-32 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
57 0 . | . | . | . CFPA-64 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
57 0 . | . | . | . CFPA-128 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
57 0 . | . | . | . DCFPA-32 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | . DCFPA-64 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | . DCFPA-128 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | .
56 0 . | . | . | . Table 3: Privacy sensitivity classification accuracies in MPIIPrivacEye using differentially private eye movement features.the FPA in the document type classifications even thoughthe utility of the FPA is lower. We observe that the read-ing patterns are hidden easier with chunk-based methods;therefore, document type classification task becomes morechallenging. This is especially validated with DCFPA meth-ods using different chunk sizes, as DCFPA-128 outperformssmaller chunk-sized DCFPAs even though the sensitivitiesare higher. Therefore, we conclude that the differential pri-vacy method should be selected for eye movements depend-ing on the further task which will be applied.Next, we analyze the gender classification results forMPIIDPEye. All methods are able to hide the gender infor-mation in the high privacy regions as it is already challeng-ing to identify it with clean data as accuracies are ≈ . inprevious work (Steil et al. 2019a). While we obtain similarresults compared to previous work for the gender classifica-tion task, the CFPA method is able to predict gender infor-mation correctly in the less private regions, namely (cid:15) = 48 ,as it also has the highest utility values in these regions. TheFPA applied to the complete signal and the DCFPA are notable to classify genders accurately. We observe that higher amount of noise that is needed by the FPA and removing thefine-grained “difference” information between eye move-ment observations with DCFPA are the reasons for hidingthe gender information successfully in all privacy regions.Overall, the CFPA provides an optimal equilibrium betweengender and document type classification success in the lessprivate regions if gender information is not considered asa feature that should be protected from adversaries. Other-wise, all proposed methods are able to hide gender informa-tion from the data in the higher privacy regions as expected.Gender classification results are depicted in Table 2. Espe-cially in some methods with k-NNs and SVMs, gender clas-sification accuracies are close to zero because of the major-ity voting and it is validated by the results without majorityvoting in the Appendix.For MPIIPrivacEye, we report privacy sensitivity classifi-cation accuracies using differentially private eye movementsin the Table 3. The FPA performs worse than our methods.The DCFPA, particularly with the chunk size of , outper-forms all other methods slightly in the higher privacy regionsas it is also the case for the utility results. In the lower pri-acy regions, the CFPA performs the best with ≈ . ac-curacy. While having ≈ . accuracy in a binary classifica-tion problem does not form the best performance, accordingto the previous work (Steil et al. 2019b), privacy sensitivityclassification using only eye movements with clean data in aperson-independent setup only performs marginally higherthan . . Therefore, we show that even though we use dif-ferentially private data in the most private settings, we obtainsimilar results to the classification results using clean data.This means that differentially private eye movements can beused along with scene features for detecting privacy sensi-tive scenes in AR setups. Conclusion
We proposed different methods to achieve differentialprivacy by correcting, extending, and adapting the FPAmethod. Since eye movement features are correlated overtime and are high dimensional, standard differential privacymethods provide low utility and are vulnerable to inferenceattacks. With this motivation, we proposed privacy solutionsfor temporally correlated eye movement data. Our methodscan easily be applied to any other human-computer interac-tion data as well since they are independent of the used data.Our methods outperform state-of-the-art methods in termsof both utility and classification accuracies while taking careof the correlations robustly. In future work, we will analyzethe actual privacy metric (cid:15) (cid:48) with k values chosen in a privatemanner for the centralized differential privacy setting. Ethics Statement
As head-mounted displays with integrated eye-trackingtechnology have found their way into many applications indaily life, it is possible to record a high amount of eye move-ment data. Apart from user assistive and comfort provid-ing tasks, machines can identify biometric information usingeye movement features. Differential privacy provides userprivacy by adding randomly generated noise to the data. Es-pecially with regard to data protection regulations, such asGeneral Data Protection Regulation (GDPR) (EUd 2018),we foresee that manufacturers and users of not only head-mounted displays, but also any device that is integrated witheye trackers or sensors that collect temporally correlated per-sonal data should benefit from this research. One disadvan-tage is that as a certain amount of noise is added to data forprotection, for purposes such as gaze guidance or contextsensitive aid, one may need more sophisticated approachesto deal with differentially private data. However, we alsothink that this would initiate new research directions in thefield of human-computer interaction.
Acknowledgments
O. G¨unl¨u and R. F. Schaefer are supported by the GermanFederal Ministry of Education and Research (BMBF) withinthe national initiative for “Post Shannon Communication(NewCom)” under the Grant 16KIS1004. O. G¨unl¨u thanksRavi Tandon for his useful suggestions. E. Bozkir thanksMartin Pawelczyk and Mete Akg¨un for useful discussions.
References
ACM Symposium on Eye Tracking Research &Applications , 4:1–4:8. New York, NY, USA. ISBN 978-1-4503-5706-7. doi:10.1145/3204493.3204531.Berkovsky, S.; Taib, R.; Koprinska, I.; Wang, E.; Zeng, Y.;Li, J.; and Kleitman, S. 2019. Detecting Personality TraitsUsing Eye-Tracking Data. In
ACM Conference on Hu-man Factors in Computing Systems , CHI ’19, 221:1–221:12.New York, NY, USA. ISBN 978-1-4503-5970-2. doi:10.1145/3290605.3300451.Bozkir, E.; Geisler, D.; and Kasneci, E. 2019. Assess-ment of Driver Attention During a Safety Critical Situa-tion in VR to Generate VR-based Training. In
ACM Sym-posium on Applied Perception 2019 , SAP ’19, 23:1–23:5.New York, NY, USA. ISBN 978-1-4503-6890-2. doi:10.1145/3343036.3343138.Bozkir, E.; ¨Unal, A. B.; Akg¨un, M.; Kasneci, E.; and Pfeifer,N. 2020. Privacy Preserving Gaze Estimation Using Syn-thetic Images via a Randomized Encoding Based Frame-work. In
ACM Symposium on Eye Tracking Researchand Applications , ETRA ’20 Short Papers. New York, NY,USA: ACM. ISBN 9781450371346. doi:10.1145/3379156.3391364.Braunagel, C.; Geisler, D.; Rosenstiel, W.; and Kasneci, E.2017. Online Recognition of Driver-Activity Based on Vi-sual Scanpath Classification.
IEEE Intelligent Transporta-tion Systems Magazine
ACMSymposium on Eye Tracking Research & Applications , 39:1–39:9. New York, NY, USA. doi:10.1145/3204493.3204550.Chaudhary, A. K.; and Pelz, J. B. 2020. Privacy-PreservingEye Videos Using Rubber Sheet Model. In
ACM Sympo-sium on Eye Tracking Research and Applications , ETRA’20 Short Papers. New York, NY, USA: ACM. ISBN9781450371346. doi:10.1145/3379156.3391375.Ding, B.; Kulkarni, J.; and Yekhanin, S. 2017. CollectingTelemetry Data Privately. In
International Conference onNeural Information Processing Systems , 3574–3583. USA:Curran Associates Inc.Dwork, C.; McSherry, F.; Nissim, K.; and Smith, A. 2006.Calibrating Noise to Sensitivity in Private Data Analysis.In Halevi, S.; and Rabin, T., eds.,
Theory of Cryptography ,265–284. Berlin, Heidelberg: Springer Berlin Heidelberg.Dwork, C.; and Roth, A. 2014. The Algorithmic Founda-tions of Differential Privacy.
Foundations and Trends inTheoretical Computer Science
ACM Trans. Priv. Secur.
ACM SIGSAC Conference on Computerand Communications Security , CCS ’14, 1054–1067. NewYork, NY, USA. ISBN 978-1-4503-2957-6. doi:10.1145/2660267.2660348.Fernndez, G.; Manes, F.; Politi, L.; Orozco, D.; Schumacher,M.; Castro, L.; Agamennoni, O.; and Rotstein, N. 2015.Patients with Mild Alzheimers Disease Fail When UsingTheir Working Memory: Evidence from the Eye TrackingTechnique.
Journal of Alzheimer’s disease: JAD
50. doi:10.3233/JAD-150265.G¨unl¨u, O. 2019.
Key Agreement with Physical UnclonableFunctions and Biometric Identifiers . Ph.D. thesis, TU Mu-nich, Germany. Published by Dr. Hut Verlag.G¨unl¨u, O.; and Iscan, O. 2014. DCT based ring oscilla-tor Physical Unclonable Functions. In , 8198–8201. ISSN 2379-190X. doi:10.1109/ICASSP.2014.6855199.G¨unl¨u, O.; Kernetzky, T.; ˙Is¸can, O.; Sidorenko, V.; Kramer,G.; and Schaefer, R. F. 2018. Secure and Reliable KeyAgreement with Physical Unclonable Functions.
Entropy
ACM Augmented Human International Conference , 15:1–15:4. New York, NY, USA. ISBN 978-1-4503-2761-9. doi:10.1145/2582051.2582066.John, B.; Koppal, S.; and Jain, E. 2019. EyeVEIL: Degrad-ing Iris Authentication in Eye Tracking Headsets. In
ACMSymposium on Eye Tracking Research & Applications , 37:1–37:5. New York, NY, USA. doi:10.1145/3314111.3319816.Kellaris, G.; and Papadopoulos, S. 2013. Practical Differ-ential Privacy via Grouping and Smoothing.
Proc. VLDBEndow.
ACMTrans. Database Syst.
ACM Symposium on Eye-TrackingResearch & Applications , 187–190. New York, NY, USA.ISBN 978-1-60558-994-7. doi:10.1145/1743666.1743712.Komogortsev, O. V.; and Holland, C. D. 2013. Biometricauthentication via complex oculomotor behavior. In , 1–8. doi:10.1109/BTAS.2013.6712725.Komogortsev, O. V.; Jayarathna, S.; Aragon, C. R.; andMahmoud, M. 2010. Biometric Identification via an Ocu-lomotor Plant Mathematical Model. In
ACM Symposium onEye-Tracking Research & Applications , 57–60. New York,NY, USA. ISBN 978-1-60558-994-7. doi:10.1145/1743666.1743679.Krejtz, K.; Duchowski, A. T.; Niedzielska, A.; Biele, C.; andKrejtz, I. 2018. Eye tracking cognitive load using pupildiameter and microsaccades with fixed gaze.
PLOS ONE
ACM Interna-tional Joint Conference on Pervasive and Ubiquitous Com-puting: Adjunct Publication , 1169–1177. New York, NY,USA. doi:10.1145/2638728.2641688.Liu, A.; Xia, L.; Duchowski, A.; Bailey, R.; Holmqvist, K.;and Jain, E. 2019. Differential Privacy for Eye-trackingData. In
ACM Symposium on Eye Tracking Research &Applications , ETRA ’19, 28:1–28:10. New York, NY, USA.ISBN 978-1-4503-6709-7. doi:10.1145/3314111.3319823.McSherry, F. D. 2009. Privacy Integrated Queries: An Ex-tensible Platform for Privacy-preserving Data Analysis. In
ACM SIGMOD International Conference on Management ofData , 19–30. New York, NY, USA. doi:10.1145/1559845.1559850.Narayanan, A.; and Shmatikov, V. 2008. Robust De-anonymization of Large Sparse Datasets. In
IEEE Sympo-sium on Security and Privacy , 111–125. doi:10.1109/SP.2008.33.Orekondy, T.; Schiele, B.; and Fritz, M. 2017. Towards a Vi-sual Privacy Advisor: Understanding and Predicting PrivacyRisks in Images. In
Proceedings of the IEEE InternationalConference on Computer Vision (ICCV) .Rastogi, V.; and Nath, S. 2010. Differentially Private Aggre-gation of Distributed Time-series with Transformation andEncryption. In
ACM SIGMOD International Conference onManagement of Data , SIGMOD ’10, 735–746. New York,NY, USA. ISBN 978-1-4503-0032-2. doi:10.1145/1807167.1807247.Raval, N.; Machanavajjhala, A.; and Pan, J. 2019. Olympus:Sensor Privacy through Utility Aware Obfuscation.
Proceed-ings on Privacy Enhancing Technologies
AAAI Conference on Artificial Intelligence , 4596–4602.Steil, J.; and Bulling, A. 2015. Discovery of EverydayHuman Activities from Long-term Visual Behaviour UsingTopic Models. In
ACM International Joint Conference onPervasive and Ubiquitous Computing , 75–85. New York,NY, USA. doi:10.1145/2750858.2807520.Steil, J.; Hagestedt, I.; Huang, M. X.; and Bulling, A. 2019a.Privacy-aware Eye Tracking Using Differential Privacy. In
CM Symposium on Eye Tracking Research & Applications ,ETRA ’19, 27:1–27:9. New York, NY, USA. ISBN 978-1-4503-6709-7. doi:10.1145/3314111.3319915.Steil, J.; Koelle, M.; Heuten, W.; Boll, S.; and Bulling, A.2019b. PrivacEye: Privacy-preserving Head-mounted EyeTracking Using Egocentric Scene Image and Eye MovementFeatures. In
ACM Symposium on Eye Tracking Research &Applications , ETRA ’19, 26:1–26:10. New York, NY, USA.ISBN 978-1-4503-6709-7. doi:10.1145/3314111.3319913.Ungrady, M. B.; Flurie, M.; Zuckerman, B. M.; Mirman,D.; and Reilly, J. 2019. Naming and Knowing Revisited:Eyetracking Correlates of Anomia in Progressive Aphasia.
Frontiers in Human Neuroscience
13: 354. ISSN 1662-5161.doi:10.3389/fnhum.2019.00354.van Leeuwen, P. M.; de Groot, S.; Happee, R.; and de Win-ter, J. C. F. 2017. Differences between racing and non-racingdrivers: A simulator study using eye-tracking.
PLOS ONE
Artificial Intelli-gence in Medicine
91: 39 – 48. ISSN 0933-3657. doi:https://doi.org/10.1016/j.artmed.2018.06.005.Zhang, Y.; Hu, W.; Xu, W.; Chou, C. T.; and Hu, J. 2018.Continuous Authentication Using Eye Movement Responseof Implicit Visual Stimuli.
ACM Interact. Mob. WearableUbiquitous Technology , 1–7. doi:10.1109/GLOCOMW.2017.8269219. upplementary Material
We report document type and gender classification results inMPIIDPEye dataset without majority voting in Tables 4 and5, respectively. Document Type Classification Accuracies (k-NN | SVM | DT | RF)Method (cid:15) = 0 . (cid:15) = 2 . (cid:15) = 4 . (cid:15) = 24 (cid:15) = 48 FPA . | . | . | .
74 0 . | . | . | .
73 0 . | . | . | .
73 0 . | . | . | .
73 0 . | . | . | . CFPA-32 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
42 0 . | . | . | . CFPA-64 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
42 0 . | . | . | . CFPA-128 . | . | . | .
39 0 . | . | . | .
39 0 . | . | . | .
39 0 . | . | . | .
42 0 . | . | . | . DCFPA-32 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | . DCFPA-64 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | .
37 0 . | . | . | . DCFPA-128 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | .
38 0 . | . | . | . Table 4: Document type classification accuracies in MPIIDPEye using differentially private eye movement features withoutmajority voting.Gender Classification Accuracies (k-NN | SVM | DT | RF)Method (cid:15) = 0 . (cid:15) = 2 . (cid:15) = 4 . (cid:15) = 24 (cid:15) = 48 FPA . | . | . | .
44 0 . | . | . | .
45 0 . | . | . | .
45 0 . | . | . | .
45 0 . | . | . | . CFPA-32 . | . | . | .
41 0 . | . | . | .
41 0 . | . | . | .
41 0 . | . | . | .
48 0 . | . | . | . CFPA-64 . | . | . | .
41 0 . | . | . | .
41 0 . | . | . | .
42 0 . | . | . | .
43 0 . | . | . | . CFPA-128 . | . | . | .
42 0 . | . | . | .
42 0 . | . | . | .
42 0 . | . | . | .
47 0 . | . | . | . DCFPA-32 . | . | . | .
42 0 . | . | . | .
42 0 . | . | . | .
42 0 . | . | . | .
42 0 . | . | . | . DCFPA-64 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | . DCFPA-128 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | .
43 0 . | . | . | .43