Mitigating Leakage in Federated Learning with Trusted Hardware
MMitigating Leakage in Federated Learningwith Trusted Hardware
Javad Ghareh Chamani
Hong Kong Universityof Science & Technology [email protected]
Dimitrios Papadopoulos
Hong Kong Universityof Science & Technology [email protected]
Abstract
In federated learning, multiple parties collaborate in order to train a global modelover their respective datasets. Even though cryptographic primitives (e.g., homo-morphic encryption) can help achieve data privacy in this setting, some partialinformation may still be leaked across parties if this is done non-judiciously. Inthis work, we study the federated learning framework of SecureBoost [Chenget al., FL@IJCAI’19] as a specific such example, demonstrate a leakage-abuseattack based on its leakage profile, and experimentally evaluate the effectivenessof our attack. We then propose two secure versions relying on trusted executionenvironments . We implement and benchmark our protocols to demonstrate thatthey are 1.2-5.4 × faster in computation and need 5-49 × less communication thanSecureBoost. Federated machine learning (FML) [33, 13, 3] is increasingly used by companies and organizationsthat want to collaboratively train a model based on the union of their separate datasets (e.g., banksand insurance companies storing their customers’ data). Typically, in FML a local model is iterativelytrained by each participant and is then merged into the global model. In this manner, it is not necessaryto transfer raw data across collaborators; the exchanged messages necessary in order to compute theglobal model consist of local model parameters, gradients, partially trained models, etc.One of the main motivations of FML is to maintain data privacy [33]. This may be necessary dueto conflicts of interest in the market or legally mandated from privacy policies such as GDPR. Toachieve data privacy, collaborating parties should at the very least not transfer raw data. However, thisdoes not guarantee privacy on its own. The transferred values (e.g., model parameters, partial models,gradients) may still leak some information that can be used to infer raw data. A growing line of workson privacy-preserving machine learning (e.g., [23, 1, 22, 28, 26, 20]) use cryptographic techniquessuch as secure multi-party computation and homomorphic encryption to ensure no information isrevealed across parties, other than the (final) global model. However, using cryptographic techniquesdoes not automatically guarantee privacy. When using such primitives in the context of a complexsystem, it is still possible that some partial information leaks which can be exploited by leakage-abuseattacks [8, 32, 12, 14, 15, 18, 7].In this work, we study the partial leakage of SecureBoost [5] (a lossless vertical federated tree-boosting system proposed by WeBank). SecureBoost allows a federated-learning process to beexecuted by multiple parties with partially common data samples but different features. It is basedon XGBoost [4] and tries to predict the classification label via regression trees. In SecureBoost,one participant (called active party ) holds sensitive labels that should not be revealed to the otherparticipants (called passive parties). On the other hand, passive parties do not want to reveal anyinformation about their data samples to each other or to the active party. Although SecureBoost
Presented at the Privacy Preserving Machine Learning Workshop (PriML/PPML Joint Edition) at the 34th Conference on Neural InformationProcessing Systems (NeurIPS 2020). Work partially supported by the HKUST-WeBank joint lab under grant WEB19EG01-E. a r X i v : . [ c s . CR ] N ov mploys additive homomorphic encryption during the tree node-splitting process, the way it is usedreveals the partial ordering of passive parties’ samples along each feature to the active party . Toevaluate the effect of this leakage, first we perform a leakage-abuse attack that tries to guess thedata sample values. We evaluate our attack (for different prior knowledge assumptions) on a publicfinancial dataset of 150K samples [6]. Under the (mild) assumption that the attacker knows someapproximate distribution of the values (e.g., via public census data) we can guess values with veryhigh accuracy (e.g., age and salary within ± ± × faster in computation time and require 5-49 × less communication sizethan SecureBoost.We believe this work can help highlight the subtle issue of leaked information in federated learning,even when cryptographic primitives are (partially) used, and how TEEs can help address it. Thereare many future directions, such as improving the performance of our schemes and combiningcryptographic primitives and trusted hardware to get secure and fast solutions for other FML tasks. Trusted Execution Environment (TEE).
Secure hardware, such as Intel-SGX [19] and AMDenclave [11], provides a trusted environment for users to execute their code in an untrusted setting,even when assuming that the operating system itself may be compromised. It provides three importantproperties: (i)
Isolated execution which is achieved by reserving a portion of the system’s memoryused to store the user’s code and data in encrypted form. (ii)
Sealing which allows secure persistentstorage of data on the untrusted area. (iii)
Remote attestation which ensures the correctness ofthe executed code in the enclave. We stress that our secure solutions can operate with any trustedhardware that realizes these properties. This is particularly important in view of the recent attacksagainst Intel-SGX [30, 17].
SecureBoost.
Cheng et al. [5] proposed SecureBoost as a lossless framework for gradient boostingdecision trees on vertically-partitioned data. In this federated machine learning setting each partyhas its own features for a common set of data samples. At a high level, SecureBoost operates asfollows. First, it uses a privacy-preserving entity alignment technique [16] to find the commonsamples among the parties’ datasets. Then, it executes an iterative computation between the activeparty that has the sensitive classification labels and the passive parties who only have samples’ valuesfor their own features. This part is based on the XGBoost [4] framework. In each iteration, the activeparty computes the gradients of samples belonging to the current tree nodes and sends their Paillierhomomorphic encryption to all the passive parties. Due to the semantic security and homomorphismof Paillier encryption [25], this allows each passive party to compute all the local possible splits(across its samples and features), while protecting the sensitive gradient values. Hence, each passiveparty iterates over its own features and samples values’ thresholds in order to compute all possiblelocal splits’ gradient summations. Then, it sends all of them back to the active party. The activeparty decrypts the received encrypted splits from all parties, computes their corresponding objectivefunction scores, and finds the best global split by comparing them. For completeness, the SecureBoostprocedures are provided in Appendix A.
As mentioned in the introduction, non-judicious use of cryptographic primitives can lead to leakageof information. In this section, we explain the data leakage profile of SecureBoost and our proposedattack. In SecureBoost, after the setup and entity alignment phase, in each round the active party2omputes the Paillier homomorphic encryption of samples’ gradients (denoted by g i and h i ) andsends them to all passive parties, as explained above. The active party then decrypts the sum of thegradients and computes each split’s score to determine the best global split. Although the activeparty only needs to learn the value of the optimal split, it actually learns all splits of all features forevery party . Clearly, this is a lot of additional information. Our main observation is that, based onthis information, an (honest-but-curious) active party can correlate the provided sums of gradients inorder to estimate the passive parties’ samples’ partial ordering across each feature. The experimentalevaluation of our attack can be found in Section 5.1.As a concrete example, assume that there is one passive party with one feature and 3 samples { x =20 , x = 30 , x = 15 }, and one active party with gradients { g = − , g = 0 . , g = 0 . }. Theinformation sent by the passive party to the active party is: { split = ( (cid:104) . (cid:105) , (cid:104)− (cid:105) + (cid:104) . (cid:105) ) , split =( (cid:104) . (cid:105) + (cid:104)− (cid:105) , (cid:104) . (cid:105) ) } where (cid:104) . (cid:105) denotes Paillier encryption. The active party decrypts the first splitand gets (0.2,-0.4). Comparing this pair with possible sum combinations of g i values reveals that0.2 is related to sample x and -0.4 is the sum of the other two samples. Likewise, the second splitreveals that x is greater than or equal to the other two samples. Therefore, the active party can inferthat x ≤ x ≤ x for this feature. We now present our solutions for securing SecureBoost, based on trusted hardware. Our protocolsare modified versions of SecureBoost that only change the split-finding procedure; other parts remainthe same. The first one assumes that each party has access to a TEE (N-TEE) while the second oneassumes that only the active party has a TEE (1-TEE). At the beginning of both protocols, code isloaded into the enclaves and interested parties get a code attestation. Then, the active party’s enclavegenerates a key for a semantically secure symmetric encryption scheme (e.g., AES) and a securechannel is established between it and the passive parties’ enclaves (for N-TEE) or the passive partiesdirectly (for 1-TEE), e.g., using [2, 27]. Secret keys are communicated (as necessary) via this securechannel. In what follows, we explain one iteration of the SecureBoost with our modified schemes.
N-TEE.
The basic idea behind N-TEE is to first find the best local split of each passive party withinits trusted hardware and then find the global best split within the active party’s trusted hardware.The split-finding procedure of N-TEE is presented in Algorithm 1. First, the active party’s enclavecomputes the symmetric encryption of gradients using the secret key for the aligned samples (denotedby I ) and sends them to the passive parties’ enclaves (line 1). Each passive party’s enclave decryptsthe gradients using the secret key (originally communicated by the active party’s enclave) and iteratesover all features and values’ thresholds to find its best local split (lines 2-18). For simplicity, weassume that all encryptions can fit in the enclave memory. Otherwise, we have to use a pagingmechanism for loading and unloading the ciphertexts. Then, each passive party’s enclave encrypts itsbest local score and sends it to the active party’s enclave (line 19). The active party’s enclave decryptsall local best splits and determines the best global split by comparing their scores (lines 20-24). Itfinally returns this output to the active party. g i and h i ).This hides the actual gradient values from passive parties while providing additive homomorphismin a very efficient way (lines 1-2). Similar to standard SecureBoost, passive parties compute allpossible splits according to their local features and threshold values (without any TEE assistance).Then, they encrypt all splits and their corresponding used gradients’ indexes using the secret key(originally communicated by the active party’s enclave) and send them back to the active party (lines3-7). The gradients’ indexes are needed for consistently removing the one-time pad randomness fromthe provided split sums at the active party. When the active party receives the encrypted splits, itpasses them along with the the random values used for one-time pad encryption to its enclave. Finally,the enclave decrypts all splits, removes their randomness based on the used gradients’ indexes (lines15,17), finds the best global split (lines 8-22), and returns it to the active party. Security and Efficiency.
In both schemes the secret key is exchanged through a secure channel anda semantically secure symmetric encryption scheme is used for encryption. Values are accessedby the TEE through sequential scans hence accesses are data-independent and leak no information.Therefore, there is no leakage and passive parties learn nothing while the active party only learns the3 lgorithm 1
N-TEEActive Party Enclave Send
Enc sk ( g i ) and Enc sk ( h i ) for all i ∈ I to all PartiesEach Passive Party Enclave Dec sk ( g i ) and Dec sk ( h i ) g ← (cid:80) i ∈ I g i , h ← (cid:80) i ∈ I h i best scor ← −∞ best f ← − ; best tr ← − //enumerate all features for k = 0 to d do g l = 0 ; h l = 0 //enumerate all threshold values for v = 0 to l k do g l ← g l + (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } g i h l ← h l + (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } h i g r ← g − g l , h r ← h − h l score = g l h l + λ + g r h r + λ + g h + λ if score > best scor then best scor ← score best f ← k best tr ← v Send
Enc sk ( best scor ) to Active PartyActive Party Enclave best scr ← −∞ , best indx ← − for i = 0 to m do (cid:46) curScore = Dec sk ( Party i .best scor ) if curScore > best scr then best scr ← curScore ; best indx ← i Algorithm 2 g rndi $ ←− [0 , ∞ ] , h rndi $ ←− [0 , ∞ ] for all i ∈ I Send g (cid:48) i ← g i + g rndi and h (cid:48) i ← h i + h rndi for all i ∈ I to all PartiesEach Passive Party for k = 0 to d do (cid:46) enumerate all features G kv ← Enc sk ( (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } g (cid:48) i ) H kv ← Enc sk ( (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } h (cid:48) i ) I kv ← Enc sk ( { i | s k,v ≥ x i,k > s k,v − } ) Send G kv , H kv , and I kv to Active PartyActive Party Enclave g ← (cid:80) i ∈ I g i , h ← (cid:80) i ∈ I h i , best i ← − best score ← −∞ , best f ← − best tr ← − for i = 0 to m do (cid:46) for k = 0 to d i do (cid:46) g l = 0 , h l = 0 for v = 0 to l k do (cid:46) t ← Dec sk ( G ikv ) − (cid:80) i ∈ I ikv g rndi g l ← g l + t t ← Dec sk ( H ikv ) − (cid:80) i ∈ I ikv h rndi h l ← h l + t scr = g l h l + λ + g r h r + λ + g h + λ if scr > best score then best i ← i ; best score ← scr best f ← k ; best tr ← v global best split outputted by its TEE. Regarding the performance, N-TEE and 1-TEE replace thecostly Paillier encryption of SecureBoost with much more light-weight symmetric encryption whichmakes them more efficient. Moreover, N-TEE has asymptotically optimal communication, muchbetter than SecureBoost, whereas 1-TEE has asymptotically the same communication as SecureBoostbut it is concretely better since it uses one-time pad instead of Paillier. In this section, we evaluate our leakage-abuse attack accuracy and measure the performance of ourconstructions. We implemented our schemes in C++ using Intel-SGX and integrated them withFATE [31] (excluding the one-time setup of secure channels between enclaves). We conducted ourexperiments on a public financial dataset [6] (the one that was used in the SecureBoost paper [5])with 150K samples and 10 attributes. All experiments were executed on a machine with a four-coreIntel Xeon E-2174G 3.8GHz processor with SGX capabilities, Ubuntu18.04 LTS, 64GB RAM, 1TBSSD, and AES-NI enabled.
The first set of experiments demonstrates the effectiveness of our attack for two features: Age(between 21 and 109) and Salary (between 0$ and 3M$). For each feature, we measure the accuracyof the attacker’s guess using four methods: (i) Random Min-Max: guess randomly knowing only4 A cc u r a cy P e r c en t age Random Approx DistRandom Min−MaxOur Approx DistOur Min−Max (a) A cc u r a cy P e r c en t age Random Approx DistRandom Min−MaxOur Approx DistOur Min−Max (b) R un t i m e ( s e c ond s ) (c) -2 -1 S i z e ( M B ) (d) Figure 1: Our proposed attack accuracy for guessing (a) age, (b) salary. Our proposed secure solutions’evaluation for (c) computation time, and (d) communication size.the minimum and maximum values. (ii) Our Attack with Min-Max: first sort samples based onthe inferred partial order and then assign the values (knowing the minimum and maximum). (iii)Random with Approximate Distribution: guess randomly assuming knowledge of a 10% incrementapproximate distribution [10, 8]. (iv) Our Attack with Approximate Distribution: first sort samplesbased on the inferred partial order and then assign the values (knowing a 10% increment approximatedistribution).The accuracy of guessing the age and salary values for 150K samples using the above four methodsis presented in Figures 1(a),(b). The obvious conclusions from the figures are as follows: (i) ourapproximate-distribution-based attack has the best accuracy among all other methods; it can guess78% of ages within ± ± , and almost all ages/salaries within ± ± , respectively, (ii) the accuracy of all methods increases as the acceptable age/salaryrange increases, and (iii) although our min-max based attack gives higher accuracy in comparison torandom min-max (up to 35% in age feature), no method can guess the target value very accuratelyjust by using the minimum and maximum of the data. In order to measure the performance of N-TEE and 1-TEE, we split the data among one active partyand one passive party. We set the maximum depth of individual regression trees to 3, the samples’fraction for fitting the regression trees to 0.8, and the learning rate to . . We used 128-bit AESencryption and 2048-bit Paillier encryption in N-TEE/1-TEE and SecureBoost, respectively. Wefocus on measuring computation time and communication size (not including transmission time).Figures 1(c),(d) show the computation time and communication size of N-TEE, 1-TEE, and Se-cureBoost. As is clear from the figures, both our methods are 1.2-5.4 × faster than SecureBoost,while eliminating its leakage from split-finding. For instance, training a model with samplesSecureBoost takes 1697s while N-TEE and 1-TEE take 314s and 323s, respectively. The main reasonfor the improvement in computation time is the use of AES and one-time pad instead of Paillier, whichmakes the cryptographic part of the computation much more efficient. Regarding communicationsize, we observe that N-TEE is the most efficient construction because it only transfers the passiveparties’ best local splits while 1-TEE and SecureBoost need to transfer all possible splits to the activeparty. Note that, since 1-TEE uses symmetric encryption instead of Paillier, its communication size isless than SecureBoost (we note that SecureBoost cannot benefit from the classic ciphertext-packingtechnique to reduce communication, as each encrypted value needs to be accessed separately inorder to compute all possible splits). According to our experiments, N-TEE and 1-TEE need 32-49 × and 5-10 × less communication than SecureBoost. E.g., for training a model with samplesSecureBoost needs to transfer a total of 307MB of data while N-TEE and 1-TEE need only 9MB and33MB, respectively. References [1] Yoshinori Aono, Takuya Hayashi, Le Trieu Phong, and Lihua Wang. Scalable and secure logistic regressionvia homomorphic encryption. In
Proceedings of the Sixth ACM Conference on Data and ApplicationSecurity and Privacy , pages 142–144, 2016.[2] Andrew Baumann, Marcus Peinado, and Galen Hunt. Shielding applications from an untrusted cloud withhaven.
ACM Transactions on Computer Systems (TOCS) , 33(3):1–26, 2015.[3] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel,Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine earning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security ,pages 1175–1191, 2017.[4] Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In
Proceedings of the 22ndacm sigkdd international conference on knowledge discovery and data mining , pages 785–794, 2016.[5] Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, and Qiang Yang. Secureboost: A losslessfederated learning framework. In
International Workshop on Federated Learning for User Privacy andData Confidentiality in Conjunction with IJCAI 2019, Macau, Macau . Online available: arXiv preprintarxiv:1901.08755, 2019.[6] Give Me Some Credit Dataset. , 2011.[7] Ioannis Demertzis, Dimitrios Papadopoulos, Charalampos Papamanthou, and Saurabh Shintre. SEAL:attack mitigation for encrypted databases via adjustable leakage. In Srdjan Capkun and Franziska Roesner,editors, , pages 2433–2450.USENIX Association, 2020.[8] Paul Grubbs, Kevin Sekniqi, Vincent Bindschaedler, Muhammad Naveed, and Thomas Ristenpart. Leakage-abuse attacks against order-revealing encryption. In ,pages 655–672. IEEE, 2017.[9] Nick Hynes, Raymond Cheng, and Dawn Song. Efficient deep learning on multi-source private data. arXivpreprint arXiv:1807.06689 , 2018.[10] Mohammad Saiful Islam, Mehmet Kuzu, and Murat Kantarcioglu. Access pattern disclosure on searchableencryption: ramification, attack and mitigation. In
Ndss , volume 20, page 12. Citeseer, 2012.[11] David Kaplan, Jeremy Powell, and Tom Woller. Amd memory encryption.
White paper , 2016.[12] Georgios Kellaris, George Kollios, Kobbi Nissim, and Adam O’neill. Generic attacks on secure outsourceddatabases. In
Proceedings of the 2016 ACM SIGSAC Conference on Computer and CommunicationsSecurity , pages 1329–1340, 2016.[13] Jakub Koneˇcn`y, H Brendan McMahan, Daniel Ramage, and Peter Richtárik. Federated optimization:Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 , 2016.[14] Evgenios M Kornaropoulos, Charalampos Papamanthou, and Roberto Tamassia. The state of the uniform:attacks on encrypted databases beyond the uniform query distribution. In , pages 1223–1240. IEEE, 2020.[15] Marie-Sarah Lacharité, Brice Minaud, and Kenneth G Paterson. Improved reconstruction attacks onencrypted data using range query leakage. In , pages297–314. IEEE, 2018.[16] Gang Liang and Sudarshan S Chawathe. Privacy-preserving inter-database operations. In
InternationalConference on Intelligence and Security Informatics , pages 66–82. Springer, 2004.[17] Moritz Lipp, Michael Schwarz, Daniel Gruss, Thomas Prescher, Werner Haas, Stefan Mangard, PaulKocher, Daniel Genkin, Yuval Yarom, and Mike Hamburg. Meltdown. arXiv preprint arXiv:1801.01207 ,2018.[18] Evangelia Anna Markatou and Roberto Tamassia. Full database reconstruction with access and searchpattern leakage. In
International Conference on Information Security , pages 25–43. Springer, 2019.[19] Frank McKeen, Ilya Alexandrovich, Alex Berenzon, Carlos V Rozas, Hisham Shafi, Vedvyas Shanbhogue,and Uday R Savagaonkar. Innovative instructions and software model for isolated execution.
Hasp@ isca ,10(1), 2013.[20] Xianrui Meng, Dimitrios Papadopoulos, Alina Oprea, and Nikos Triandopoulos. Privacy-preservinghierarchical clustering: Formal security and efficient approximation.
CoRR , abs/1904.04475, 2019.[21] Fan Mo and Hamed Haddadi. Efficient and private federated learning using tee. In
EuroSys , 2019.[22] Payman Mohassel and Peter Rindal. Aby3: A mixed protocol framework for machine learning. In
Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , pages35–52, 2018.[23] Valeria Nikolaenko, Udi Weinsberg, Stratis Ioannidis, Marc Joye, Dan Boneh, and Nina Taft. Privacy-preserving ridge regression on hundreds of millions of records. In , pages 334–348. IEEE, 2013.[24] Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, andManuel Costa. Oblivious multi-party machine learning on trusted processors. In { USENIX } SecuritySymposium ( { USENIX } Security 16) , pages 619–636, 2016.[25] Pascal Paillier. Public-key cryptosystems based on composite degree residuosity classes. In
Internationalconference on the theory and applications of cryptographic techniques , pages 223–238. Springer, 1999.
26] L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai. Privacy-preserving deep learning via additivelyhomomorphic encryption.
IEEE Transactions on Information Forensics and Security , 13(5):1333–1345,2018.[27] Felix Schuster, Manuel Costa, Cédric Fournet, Christos Gkantsidis, Marcus Peinado, Gloria Mainar-Ruiz,and Mark Russinovich. Vc3: Trustworthy data analytics in the cloud using sgx. In , pages 38–54. IEEE, 2015.[28] Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In
Proceedings of the 22nd ACMSIGSAC conference on computer and communications security , pages 1310–1321, 2015.[29] Florian Tramer and Dan Boneh. Slalom: Fast, verifiable and private execution of neural networks in trustedhardware. arXiv preprint arXiv:1806.03287 , 2018.[30] Jo Van Bulck, Marina Minkin, Ofir Weisse, Daniel Genkin, Baris Kasikci, Frank Piessens, Mark Silberstein,Thomas F Wenisch, Yuval Yarom, and Raoul Strackx. Foreshadow: Extracting the keys to the intel { SGX } kingdom with transient out-of-order execution. In { USENIX } Security Symposium ( { USENIX } Security 18) , pages 991–1008, 2018.[31] The federated AI ecosystem project WeBank AI Department. , 2020.[32] Tongyang Xu, Fan Liu, Ang Li, Christos Masouros, and Izzat Darwazeh. Constructive interferenceprecoding for reliable non-orthogonal iot signaling. In
IEEE INFOCOM 2019-IEEE Conference onComputer Communications Workshops (INFOCOM WKSHPS) , pages 590–595. IEEE, 2019.[33] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept andapplications.
ACM Transactions on Intelligent Systems and Technology (TIST) , 10(2):1–19, 2019.[34] Xiaoli Zhang, Fengting Li, Zeyu Zhang, Qi Li, Cong Wang, and Jianping Wu. Enabling execution assuranceof federated learning at untrusted participants. In
IEEE INFOCOM 2020-IEEE Conference on ComputerCommunications , pages 1877–1886. IEEE, 2020.
A SecureBoost Routines
For completeness, we include here the pseudocode of the SecureBoost algorithms from [5].
Algorithm 3
SecureBoost: Aggregate Encrypted Gradient Statistics Input:
I, instance space of current node Input: d, feature dimension Input: {(cid:104) g i (cid:105) , (cid:104) h i (cid:105)} i ∈ I (cid:46) g i and h i are gradients Output: G ∈ R d × l , H ∈ R d × l Passive Party for k = 0 to d do S k = { s k , s k , . . . , s kl } by percentiles on feature k for k = 0 to d do G kv = (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } (cid:104) g i (cid:105) H kv = (cid:80) i ∈{ i | s k,v ≥ x i,k >s k,v − } (cid:104) h i (cid:105) lgorithm 4 SecureBoost: Split Finding Input: