Making Paper Reviewing Robust to Bid Manipulation Attacks
Ruihan Wu, Chuan Guo, Felix Wu, Rahul Kidambi, Laurens van der Maaten, Kilian Q. Weinberger
MMaking Paper Reviewing Robust to Bid Manipulation Attacks
Ruihan Wu * 1
Chuan Guo * 2
Felix Wu
Rahul Kidambi
Laurens van der Maaten Kilian Q. Weinberger Abstract
Most computer science conferences rely on paperbidding to assign reviewers to papers. Althoughpaper bidding enables high-quality assignmentsin days of unprecedented submission numbers, italso opens the door for dishonest reviewers toadversarially influence paper reviewing assign-ments. Anecdotal evidence suggests that somereviewers bid on papers by “friends” or collud-ing authors, even though these papers are outsidetheir area of expertise, and recommend them foracceptance without considering the merit of thework. In this paper, we study the efficacy of such bid manipulation attacks and find that, indeed,they can jeopardize the integrity of the reviewprocess. We develop a novel approach for paperbidding and assignment that is much more robustagainst such attacks. We show empirically thatour approach provides robustness even when dis-honest reviewers collude, have full knowledge ofthe assignment system’s internal workings, andhave access to the system’s inputs. In additionto being more robust, the quality of our paper re-view assignments is comparable to that of cur-rent, non-robust assignment approaches.
1. Introduction
Peer review is a cornerstone of scientific publishing. Italso functions as a gatekeeper for publication in top-tiercomputer-science conferences. To facilitate high-qualitypeer reviews, it is imperative that paper submissions arereviewed by qualified reviewers. In addition to assess-ing a reviewer’s qualifications based on their prior publi-cations (Charlin & Zemel, 2013), many conferences im-plement a paper bidding phase in which reviewers expresstheir interest in reviewing particular papers. Facilitating * Equal contribution † Work done while at Cornell University. Department of Computer Science, Cornell University FacebookAI Research ASAPP Amazon Search & AI. Correspondenceto: Ruihan Wu < [email protected] > , Chuan Guo < [email protected] > . bids is important because the review quality is higher whenreviewers are interested in a paper (Stent & Ji, 2018).Unfortunately, paper bidding also creates the potential fordifficult-to-detect adversarial behavior by reviewers. Inparticular, a reviewer may place high bids on papers by“friends” or colluding authors, even when those papers areoutside of the reviewer’s area of expertise, with the purposeof accepting the papers without merit. Anecdotal evidencesuggests that such bid manipulation attacks may have, in-deed, influenced paper acceptance decisions in recent top-tier computer science conferences (Vijaykumar, 2020).This paper investigates the efficacy of bid manipulation at-tacks in a realistic paper-assignment system. We find thatsuch systems are, indeed, very vulnerable to adversarialbid, which is corroborated by prior work (Jecmen et al.,2020). Furthermore, we design a paper-assignment systemthat is robust against bid manipulation attacks. Specifically,our system treats paper bids as supervision for a model ofreviewer preferences, rather than directly using bids to as-sign papers. We then detect atypical patterns in the paperbids by measuring their influence on the model, and removesuch high-influence bids as they are potentially malicious.We evaluate the efficacy of our system on a novel, syntheticdataset of paper bids and assignments that we developed tofacilitate the study of robustness of paper-assignment sys-tems. We carefully designed this dataset to match the statis-tics of real bidding data from recent computer-science con-ferences. We find that our system produces high-qualitypaper assignments on the synthetic dataset, while also pro-viding robustness against groups of colluding, adversarialreviewers in a white-box setting in which the adversarieshave full knowledge of the system’s inner workings and itsinputs. We hope our findings will help computer-scienceconferences in performing high-quality paper assignmentsat scale, while also minimizing the surface for adversarialbehavior by a few bad actors in their community.
2. Bid Manipulation Attacks
We start by investigating the effectiveness of bid manipula-tion attacks on a typical paper assignment system.
Paper assignment system.
Most paper assignment sys-tems utilize a computed score s r,p for each reviewer-paper a r X i v : . [ c s . G T ] F e b aking Paper Reviewing Robust to Bid Manipulation Attacks Figure 1.
Probability of assigning an adversarial reviewer to the target paper before and after the reviewer executes their black-box bidmanipulation attack. See text for details. pair ( r, p ) that reflects the degree of relevance between thereviewer and the paper (Hartvigsen et al., 1999; Goldsmith& Sloan, 2007; Tang et al., 2012; Charlin & Zemel, 2013).The conference organizer can then maximize utility met-rics such as the total relevance score whilst maintaining ap-propriate balance constraints: i.e. , there are an adequatenumber of, say, R reviewers per paper and every reviewerreceives a manageable load of at most P papers. This ap-proach gives rise to the following optimization problem: max a ∈{ , } m × n m (cid:88) r =1 n (cid:88) p =1 a r,p s r,p (1)subject to m (cid:88) r =1 a r,p = R ∀ p, n (cid:88) p =1 a r,p ≤ P ∀ r, where m and n refer to the total number of reviewers andpapers, respectively. Eq. (1) is an assignment problem thatcan be solved using standard techniques such as the Hun-garian algorithm (Kuhn, 1955).The reviewer-paper relevance score, s r,p , is critical in ob-taining high-quality assignments. Arguably, an ideal rele-vance score incorporates both the reviewer’s expertise and interest towards the paper (Stent & Ji, 2018). Approachesfor measuring expertise include computing the similarityof textural features between reviewers and papers (Du-mais & Nielsen, 1992; Mimno & McCallum, 2007; Charlin& Zemel, 2013) as well as using authorship graphs (Ro-driguez & Bollen, 2008; Liu et al., 2014). In addition tothese features, paper assignment systems generally con-sider reviewer interest obtained via self-reported paperbids. For example, the NeurIPS-2014 assignment system(Lawrence, 2014) uses a formula for s r,p that incorporatesthe reviewer’s and paper’s subject area, TPMS score (Char-lin & Zemel, 2013), and the reviewer’s bid. Each reviewermay bid on a paper as none , in a pinch , willing , or eager to express their preference. The none option is the default For simplicity, we exclude the option not willing that ex-presses negative interest. bid when a reviewer did not enter a bid.
Bid manipulation attacks.
Although incorporating re-viewer interest via self-reported bids is beneficial to theoverall assignment quality, it also allows a malicious re-viewer to bid eager on a paper that is outside their area ofexpertise, with the sole purpose of influencing the accep-tance decision of a paper that was authored by a “friend”or a “rival”. If a single bid has too much influence on theoverall assignment, such bid manipulation attacks may beeffective and jeopardize the integrity of the review process.We demonstrate the feasibility of a simple black-box bid manipulation attack against the assignment systemin Eq. (1). For a target paper p , the malicious reviewerattacks the assignment system by bidding eager for p and none for all other papers. We evaluate the effectivenessof the attack by randomly picking 400 papers from oursynthetic conference dataset (see Section 5), and deter-mine paper assignments using Eq. (1) (with R = 3 and P = 6 ) using relevance scores from the NeurIPS-2014 sys-tem (Lawrence, 2014). Fig. 1 ( left ) shows the fraction ofadversarial reviewers ( m = 2 , ) that can secure theirtarget paper in the final assignment via the bid manipula-tion attack. As an attack is easier if a reviewer is alreadyranked high for a particular paper ( e.g. , because nobodyelse bids on this paper, or the subject areas match), we vi-sualize the success rate as a function of rank of the “true”paper-reviewer relevance score. More precisely, we rankall reviewers by their original (pre-manipulation) relevancescore s r,p and group them into bins of increasing size.The light gray bar in each bin reports the assignment suc-cess rate if all reviewers bid honestly. In the absence ofmalicious reviewers, the majority of assignments go to re-viewers ranked 1 to 7. However, with malicious bids, any reviewer stands a good chance of being assigned the targetpaper. For instance, the chance of getting a target paper fora reviewer ranked between 16 and 31 increases from 0% toover 70% when bidding maliciously. Even reviewers withthe lowest ranks (2048 and lower) have a 40% chance of aking Paper Reviewing Robust to Bid Manipulation Attacks being assigned the target paper by just changing their bids.This possibility is especially concerning because it may bemuch easier for an author to corrupt a non-expert reviewer( i.e. , a reviewer with a relatively low rank), simply becausethere are many more such reviewer candidates.
3. Predicting Relevance Scores
The success of the bid manipulation attack exposes an in-herent tension in the assignment process. Assigning pa-pers to a reviewer who has expressed explicit interest helpsin eliciting high-quality feedback. However, relying tooheavily on individual bids paves the way for misuse by ma-licious reviewers. To achieve a better trade-off, we proposeto use the bids from all reviewers (of which the vast ma-jority are honest) as labels to train a supervised model that predicts bids as the similarity score s r,p , and all other indi-cators ( e.g. , subject area matches, TPMS score (Charlin &Zemel, 2013), and paper title) as features. This indirect useof bids allows the scoring function to capture reviewer pref-erences but reduces the potential for abuse. Later, we willshow that this approach also allows for the development ofactive defenses against bid manipulation attacks. Scoring model.
Let X ∈ R ( mn ) × d be a feature matrixconsisting of d -dimensional feature vectors for every pairof m reviewers and n papers. Let Y denote the set of pos-sible bids in numerical form, e.g. Y = { , , , } . Wedefine y ∈ Y mn as the label vector containing the numeri-cal bids for all reviewer-paper pairs. We define a ridge re-gressor that maps reviewer-paper features to correspondingbids, similar to the linear regression model from Charlin &Zemel (2013): w ∗ = argmin w (cid:107) X w − y (cid:107) + λ (cid:107) w (cid:107) . (2)To ensure that no single reviewer has disproportionate in-fluence on the model, we restrict the maximum number ofpositive bids from a reviewer to be at most U = 60 andsubsample bids of a reviewer whenever the number of bidsexceeds U . In a typical CS conference, most reviewers bidon no more than 60 papers (out of thousands of submis-sions) (Shah et al., 2018).The trained model w ∗ can predict reviewer interest by com-puting a score s r,p for a reviewer-paper pair ( r, p ) as fol-lows: s r,p = X r,p w ∗ = X r,p H − X (cid:62) y , (3)where H = X (cid:62) X + λI is the ridge Hessian (size d × d )and X r,p is the feature vector for the pair ( r, p ) . These pre-dicted scores can then be used in the assignment algorithmin place of bids. In Appendix B, we validate the predic-tion accuracy of our model using the average precision-at-k(AP@k) metric. There is an important advantage to our method: bidding isa laborious and monotonous task, and as mentioned abovemost reviewers only bid on very limited papers. It is likelythat only a partial set of bids is observed among all papersthat the reviewer is interested in. The scoring model couldfill in missing scores by learning the latent interest from thefeatures of papers and reviewers. Completing the full bid-ding matrix improves the assignment quality, particularlyfor papers that received few bids originally.The choice of regression loss serves an important purpose.Since the bid value (between 0 and 3) reflects the degree ofinterest from a reviewer, the loss should reflect the severityof error when making a wrong prediction. For example, ifa reviewer expresses eager interest (bid score 3), predicting no bid (bid score 0) would incur a much greater loss thanpredicting willing (bid score 2). Effect against simple black-box attack.
Fig. 1 ( right )shows the effect of the proposed scoring model against thebid manipulation attack from Section 2. The assignmentprobability for honest bidders (light orange) is similar tothat of the NeurIPS-2014 system across different bins ofreviewer rank. However, deviations from benign biddingbehavior are clearly corrected by the model: in fact, theassignment probability decreases after the attack (dark or-ange). This can be explained by the fact that our approachdoes not use bids to assign reviewers to papers directly, butinstead to learn for what type of papers a reviewer may besuitable. The reviewer is actually well-suited for high rank-ing submissions, but by only bidding on the target paper(instead of honest bids on similar submissions) the modelreceives less signal that suggests the reviewer is a match forthe target paper.
4. Defending Against Colluding BidManipulation Attackers
Although the learning-based approach appears robustagainst manipulation of bids by one reviewer, attack-ers may have stronger capabilities. Specifically, an ad-versary can modify their bids based on knowledge ofa friend/rival’s submissions or another reviewer’s bids.Moreover, adversarial reviewers may collude to secure theassignment of a specific paper. We capture such capabili-ties in a threat model that describes our assumptions aboutthe adversary. We design an optimal white-box attack inthis threat model that drastically improves the adversary’ssuccess rate. Both the threat model and the white-box at-tack are intentionally designed to provide very broad capa-bilities to the adversary. Next, we design a defense that de-tects and removes white-box adversaries from the reviewerpool to provide security even under the new threat model. aking Paper Reviewing Robust to Bid Manipulation Attacks
Threat Model.
We make the following assumptionsabout adversarial reviewers: The adversary may col-lude with one or more reviewers to secure a target paper’sassignment. If any of the colluding reviewers are assignedthe paper in question, the attack is considered successful.Collusion with any reviewer is allowed except the top-ranked candidates (based on honest bidding), as this wouldnot be an abuse of the bidding process . The adversarycannot manipulate any training features. We are interestedin preventing against the additional security risk enabledby the bidding mechanism. An attack that succeeds by ma-nipulating features can also be used against an automatedassignment system that does not allow bidding. The ad-versary may have full knowledge of the assignment system. The adversary may have direct access to the features andbids of all other reviewers. The adversary may be ableto arbitrarily manipulate his/her bids and those of anyonein the colluding group.
To successfully attack the assignment system under theseassumptions, the adversary needs to maximize the pre-dicted relevance score of the target paper for him/herselfand/or the other colluding reviewers. This amounts to ex-ecuting a data poisoning attack (Biggio et al., 2012; Xiaoet al., 2015; Mei & Zhu, 2015; Jagielski et al., 2018; Kohet al., 2018) against the regression model that is used topredict scores, aiming to alter the score prediction for aspecific paper-reviewer pair.
Non-colluding attack.
We first devise an attack that max-imizes the malicious reviewer’s score s r,p for target paper p in the non-colluding setting. We represent reviewers as [ m ] = { , . . . , m } and let Y feas = { y (cid:48) ∈ Y n : |{ q : y (cid:48) q > }| ≤ U } denote the feasible set of bidding vectors for a particularreviewer for which the number of positive bids is at most U . Adversary r can change y r to the y (cid:48) r ∈ Y feas that maxi-mizes the relevance score: s ∗ r,p := max y (cid:48) r ∈Y feas X r,p H − ( X (cid:62) r y (cid:48) r + X (cid:62) [ m ] \{ r } y [ m ] \{ r } )= max y (cid:48) r ∈Y feas X r,p H − ( X (cid:62) r y (cid:48) r − X (cid:62) r y r + X (cid:62) y ) . It is straightforward to see that s ∗ r,p maximally increases thescore prediction for reviewer r : ∆ s ∗ r,p := s ∗ r,p − s r,p = max y (cid:48) r ∈Y feas X r,p H − X (cid:62) r ( y (cid:48) r − y r ) . (4) e.g. by posting the paper ID in a private chat channel of col-lege alumni or like minded members of the community. For this reason, our framework is not suitable for preventingthe attack in (Vijaykumar, 2020) since collusion likely occurredin the author stage.
Note that Eq. (4) maximizes the inner product between z := X r,p H − X (cid:62) r and y (cid:48) r − y r . To achieve the max-imum, papers q corresponding to the top- U positive val-ues in z should be assigned y r,q = max Y , and the re-maining bids are set to 0. This requires the adversary tosolve a top- U selection problems, which can be done in O ( d + n ( d + log U )) (Cormen et al., 2009). Colluding attack.
Adversarial reviewers can collude tomore effectively maximize the predicted score for reviewer r . An attack in this setting maximizes over the colludinggroup, M , and over the bids of every reviewer in M . Wenote that Eq. (4) is not specific to reviewer r , but that theinfluence of any reviewer t ’s bids on score prediction s r,p has the form: ∆ t s r,p := max y (cid:48) t ∈Y feas X r,p H − X (cid:62) t ( y (cid:48) t − y t ) . Hence, the influence from the members of M on s r,p are independent , which implies the adversaries can adopt agreedy approach. Specifically, M a colluding adversariescan alter the ( M a n ) -dimensional training label vector y M to y (cid:48)M ∈ Y M a feas to maximize the score prediction for re-viewer r via: ∆ s ∗ r,p = max ( M , y (cid:48)M ) ∈P ( r,M a ) X r,p H − X (cid:62)M ( y (cid:48)M − y M ) , = max M⊆ [ m ]: r ∈M , |M| = M a (cid:88) t ∈M max y (cid:48) t ∈Y feas X r,p H − X (cid:62) t ( y (cid:48) t − y t )= max M⊆ [ m ]: r ∈M , |M| = M a (cid:88) t ∈M ∆ t s r,p , (5)where P ( r, M a ) denotes the set of possible colluding par-ties of size M a and their bids: P ( r, M a ) := { ( M , y (cid:48)M ) : M ⊆ [ m ] ,r ∈ M , |M| = M a and y (cid:48)M ∈ Y M a feas } . The last line in Eq. (5) can be computed by first evaluating ∆ t s r,p for every t ∈ [ m ] \ { r } , and then greedily selectingthe top- ( M a − reviewers to form the colluding party with r . The computational complexity of the resulting attack is O ( d + mn ( d + log U ) + m log M a )) . Both the black-box attack from Section 2 and the white-boxattack described above adversarially manipulate paper bids.In contrast to honest reviewers whose bids are strongly cor-related with their expertise and subject of interest, attackersprovide “surprising” bids that have a large influence on thepredictions of the scoring model. This allows us to detectpotentially malicious bids using an outlier detection algo-rithm. Specifically, we make our paper assignment system aking Paper Reviewing Robust to Bid Manipulation Attacks
Algorithm 1
Paper assignment system that is robust againstcolluding bid manipulation attacks. Predict relevance scores s r,p for all reviewer-paperpairs; Initialize candidate set C = { ( r, p ) : rank ( s r,p ) is at least K for paper p } ; for reviewer-paper pair ( r, p ) ∈ C do Compute relevance score s † r,p using Eq. (7) Remove ( r, p ) from C if rank ( s † r,p ) is below K forpaper p ; end for Solve the assignment problem in Eq. (1) using s r,p forpairs in C .robust against the colluding bid manipulation attacks bydetecting and removing training examples that have a dis-proportional influence on model predictions. We make thesame assumptions about the attacker as in Section 4.1, and,in addition, that they are unaware of our active defense.To implement this system, we note that given a set ofmalicious reviewers M , we can re-compute the relevancescores for a reviewer-paper pair ( r, p ) by removing thesereviewers from the training set: ˜ s r,p = X r,p H − M c X (cid:62)M c y M c , where H M c = X (cid:62)M c X M c + λI is the Hessian matrix fordata points in the complement of the malicious reviewer set M . We assume that at most M d reviewers collude to formset M . Intuitively, ˜ s r,p reflects the relevance score for thepair ( r, p ) as predicted by other reviewers . Relying on theassumption that the vast majority of reviewers are benign, ˜ s r,p is likely close to the unobserved true preferences had r been benign.Following work on robust regression (Jagielski et al., 2018;Chen et al., 2013; Bhatia et al., 2015), this allows us tocompute relevance scores that ignore the most likely mali-cious reviewers in M by evaluating: s † r,p = min M⊆ [ m ]: r ∈M , |M| = M d X r,p H − M c X (cid:62)M c y M c ≤ ˜ s r,p . (6)That is, s † r,p overestimates the decrease in the predicted rel-evance score for ( r, p ) had r been benign. The optimizationproblem in Eq. (6) is intractable because it searches over (cid:0) m − M d − (cid:1) = Θ( m M d ) subsets of reviewers, M , and becauseit inverts a d × d Hessian for every M . To make optimiza-tion tractable, we approximate the Hessian H − M c by H − ,which is accurate for small M d . This approximation facili-tates a greedy search for M because it allows Eq. (6) to be decomposed: s † r,p ≈ min M⊆ [ m ]: r ∈M , |M| = M d X r,p H − X (cid:62)M c y M c = X r,p H − X (cid:62) y − max M⊆ [ m ]: t ∈M , |M| = M d (cid:88) t ∈M X r,p H − X t y t . (7)Eq. (7) can be computed efficiently by sorting the val-ues of S = { X r,p H − X t y t : t (cid:54) = r } and selecting r as well as the top M d − corresponding reviewers in S .The computational complexity of the resulting algorithm is O ( d + mnd + m log M d )) for each pair ( r, p ) . Assignment algorithm.
Efficient approximation for therobust relevance score s † r,p enables our robust assignmentalgorithm, which proceeds as follows. We first form the candidate set C of reviewer-paper pairs by selecting thetop- K reviewers for each paper according to the predictedrelevance score s r,p . For each pair ( r, p ) ∈ C , the algo-rithm marks r as potentially malicious and removes the pair ( r, p ) from C if r would not have belonged to the candidateset using the robust relevance score s † r,p . Since s † r,p ≤ ˜ s r,p ,an M a -colluding attack is always marked as malicious if M a ≤ M d . After removing every potentially maliciouspair from C , the assignment problem in Eq. (1) is solvedover the remaining reviewer-paper pairs in the candidate setto produce the final assignment . The resulting assignmentalgorithm is summarized in Algorithm 1. The algorithmtrades off two main goals:1. Every paper needs to be assigned to a sufficient num-ber of reviewers that have the expertise and willingnessto review. Therefore, the approach that removes poten-tially malicious reviewer candidates needs to have a lowfalse positive rate (FPR).2. The final assignment should be robust against collusionattacks. Therefore, the approach that filters out poten-tially malicious reviewers needs to have a high true pos-itive rate (TPR).This trade-off between FPR and TPR is governed by thehyperparameter M d . Using a higher value of M d can pro-vide robustness against larger collusions, but it may alsoremove many benign reviewers from the candidate set evenwhen insufficient alternative reviewers are available. Weperform a detailed study of this trade-off in Section 5.
5. Experiments
We empirically study the efficacy of our robust paper bid-ding and assignment algorithm. Our experiments show thatour assignment algorithm removes a large fraction of mali- This can be achieved by setting s r,p = −∞ for all ( r, p ) / ∈ C . aking Paper Reviewing Robust to Bid Manipulation Attacks [ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ] reviewer's original rank s u cc e ss r a t e attack success rate VS original reviewer rank M a =1 M a =2 M a =3 M a =4 M a =5 M a =10 Figure 2.
Success rate after the white-box bid manipulation attackagainst an undefended linear regression scoring model. cious reviewers, while still preserving the utility of bids forhonest reviewers.
Dataset.
Because real bidding data is not publicly avail-able, we construct a synthetic conference dataset from theSemantic Scholar Open Research Corpus (Ammar et al.,2018). This corpus contains publicly available academicpapers annotated with attributes such as citation, venue,and field of study. To simulate a NeurIPS-like conferenceenvironment, we collect n = 2446 papers published in AIconferences between 2014 and 2015 to serve as submittedpapers. We also select m = 2483 authors to serve as re-viewers, and generate bids based on paper citations. Gen-erated bids are selected from the set Y = { , , , } , cor-responding to the bids none , in a pinch , willing , and eager .We generated bids in such a way as to mimic bidding statis-tics from a recent, major AI conference. Our paper and re-viewer features include paper/reviewer subject area, papertitle, and a TPMS-like similarity score. We refer to the ap-pendix for more details on our synthetic dataset. For fullreproducibility we release our code and synthetic data publicly and invite program chairs across disciplines to useour approach on their real bidding data. We first show that the white-box attack from Section 4.1can succeed against our relevance scoring model if detec-tion of malicious reviewers is not used. We perform thewhite-box attacks as follows: The relevance scoring model is trained to predict scores s r,p for every reviewer-paper pair. We randomly select 400 papers and rank all m = 2483 reviewers for these papers based on s r,p . We discard the K = 50 highest-ranked reviewers as at- https://github.com/facebookresearch/secure-paper-bidding https://drive.google.com/drive/folders/1khI9kaPy_8F0GtAzwR-48Jc3rsQmBhfe M a T P R M d = 1 M d = 2 M d = 3 M d = 4 M d = 5 Figure 3.
TPR for detecting successful white-box attacks usingAlgorithm 1. For colluding parties of size M a ≤ M d , the detec-tion algorithm has a near-perfect TPR. Detection remains viableeven when M a > M d for moderately high values of M d . tacker candidates for paper p because high-ranked review-ers need not act maliciously to be assigned. We group the remaining reviewers into bins of exponen-tially growing size (powers of two), and sample 10 mali-cious reviewers from each bin without replacement. Each selected reviewer chooses its most suitable M a colluders and modifies their bids using the attack from Sec-tion 4.1, targeting paper p . Result.
We run our assignment algorithm on the mali-ciously modified bids and evaluate the chance of assign-ment for reviewer r before and after the attack. Fig. 2shows the fraction of malicious reviewers that successfullyalter the paper assignments and is assigned their target pa-per. Each line shows the attack success rate with a certaincolluding party size of M a . When bidding honestly, all re-viewers are below rank K = 50 and have no chance of be-ing assigned. With a colluding party size of M a = 10 , a re-viewer has a 22% chance of being assigned the target paperat an original rank of 51. At the same rank, the success rateis up to 5% even when no collusion occurs. Increasing thecollusion size M a strictly increases the assignment prob-ability, while attackers starting from a lower original rankhave a lower success rate. The latter trend shows that themodel provides a limited degree of robustness even withoutthe detection mechanism. We evaluate the robust assignment algorithm against suc-cessful attacks from Section 4.1.
What percentage of attacks is accurately detected?
Fig. 3 shows the true positive rate (TPR) of detecting ma-licious reviewers as a function of collusion size, M a (onthe x -axis), for different values of the hyperparameter M d .First, we measure the algorithm against all attacks that suc-ceeded against the undefended scoring model ( cf. Fig. 2).The results show that when M a ≤ M d , the detection TPRis very close to 100%, which implies almost all malicious aking Paper Reviewing Robust to Bid Manipulation Attacks Setting FPR Assignment Quality
NeurIPS-2014 – – 0.990 2.732 0.732 0.737 –TPMS only – – 0.323 0.872 0.949 0.997 – M d = 0 – – 0.442 1.200 0.848 0.943 – M d = 1 M d = 2 M d = 3 M d = 4 M d = 5 Table 1.
FPR and assignment quality after detection using different settings of M d . A higher value of M d offers a better protectionagainst large colluding parties (see Fig. 3), but also increases the detection FPR. Nevertheless, assignment quality is minimally impactedeven with a high FPR since the majority of false positives have low rank and are unlikely to be assigned to begin with. reviewers are removed in this case. The TPR decreases asthe size of the collusion, M a increases but still providessome protection even when M a > M d . For instance, when M a = 5 and M d = 4 (darkest blue line), approximately40% of the successful attacks are detected. Increasing M d will protect against larger colluding parties at the cost ofincreasing the false positive rate (FPR), that is, the numberof times in which an honest reviewer is mistaken for an ad-versary. A high FPR can negatively impact the quality ofthe assignments.The degree of knowledge that we assume the attacker maypossess far exceed that of typical reviewers. As a result,Fig. 3 may drastically underestimate the efficacy of our de-tection framework for practical applications. We furtherformulate a stronger colluding black-box attack and evalu-ate against it in the appendix. Our results are very encour-aging as it suggests that conference organizers can obtainrobustness against more than 80% of successful colludingblack-box attacks with M a = 10 when applying our detec-tion framework. What is the quality of the final assignments?
To studythe effect of false positives from detection on the final pa-per assignments, we also evaluate assignment quality interms of fraction of positive bids , average bid score , av-erage TPMS , and average maximum TPMS ( i.e. , maximumTPMS score among assigned reviewers for each paper av-eraged over all papers). Higher values of these metrics in-dicate a higher assignment quality. The first row in Table 1shows the assignment quality when using the NeurIPS-2014 (Lawrence, 2014) relevance scores. As expected, itover-emphasizes positive bids, which constitutes its inher-ent vulnerability. The second line shows the assignmentquality when using only the TPMS score, which serves asa baseline for evaluating how much utility from bids is ourrobust assignment framework preserving. In contrast, usingTPMS scores over-emphasizes average TPMS and averagemaximum TPMS. The third line shows our assignment algorithm using thelinear regression model without malicious reviewer detec-tion ( M d = 0 ). As it fills in the initially sparse biddingmatrix, it has significantly more papers to choose fromand yields assignments with fewer positive bids — how-ever the assignment quality is substantially higher in termsof TPMS metrics compared to when using NeurIPS-2014scores. The regression model offers a practical trade-off be-tween relying on bids that reflect reviewer preference andrelying on factors related to expertise (such as TPMS).The remaining rows report results for the robust assignmentalgorithm with increasing values of M d . As expected, de-tection FPR increases as M d increases, but only has a lim-ited effect on the assignment quality metrics. The mainreason for this is that most false positives are low-rankedreviewers, who are unlikely to be assigned the paper even ifthey were not excluded from the candidate set. Indeed, de-tection FPR is significantly lower for top-5 reviewers (sec-ond column) compared to that of top-50 reviewers (thirdcolumn). Overall, our results show that the assignmentquality is hardly impacted by the detection mechanism.We observed that a small number of papers were not as-signed sufficient reviewers because the detection removedtoo many reviewers from the set of candidate reviewers forthose papers. We report this number in the last column ( Comparison with robust regression.
One effective de-fense against label-poisoning attacks for linear regressionis the TRIM algorithm (Jagielski et al., 2018), which fitsthe model on a subset of the points that incur the least loss.The algorithm assumes that L out of the mn training points aking Paper Reviewing Robust to Bid Manipulation Attacks Defense Assignment Quality Detection TPRFrac. of pos. Avg. bid score Avg. TPMS Avg. max. TPMS M a = a = a = a = a = TRIM ( L = 10000) ( L = 30000) ( M d = 1) ( M d = 5) Table 2.
Comparison of assignment quality and detection TPR against white-box attack between the TRIM robust regression algorithmand our robust assignment algorithm. See text for details. are poisoned and optimize: min w , I (cid:107) X I w − y I (cid:107) + λ (cid:107) w (cid:107) s.t. I ⊆ { , . . . , mn } , |I| = mn − L, where X I , y I denote the subset of mn − L training datapoints selected by the index set I . We apply TRIM to iden-tify the L poisoned pairs ( r, p ) and remove them from theassignment candidate set. We then proceed to assign theremaining mn − L pairs using Eq. (3).Table 2 shows the comparison between TRIM and our ro-bust assignment algorithm in terms of assignment qualityand detection TPR. The first and third rows correspond tothe TRIM algorithm and Algorithm 1 that achieve a com-parable assignment quality. Both methods fail to detect col-luding attacks with M a > , but Algorithm 1 is drasticallymore effective when M a = 1 . The second and fourth rowscompare settings of TRIM and Algorithm 1 that achieve asimilar detection TPR. Indeed, both have close to de-tection rate for M a = 1 , . . . , . However, the assignmentquality for TRIM is much worse, with all quality metricsbeing lower than using TPMS score alone ( cf. row 2 in Ta-ble 1). Note that TRIM requires a drastic overestimate ofthe number of poisoned data ( L = 30 , in order to de-tect most attack instances, which means that many benigntraining samples are being misidentified as malicious. Running time.
As described in Section 4.2, our detec-tion algorithm has a computational complexity of O ( d + mnd + m log M d ) for each reviewer-paper pair. In prac-tice, pairs belonging to the same paper can be processed ina batch to re-use intermediate computation, which amountsto an average of 26 seconds per paper. This process can beeasily parallelized across papers for efficiency.
6. Related Work
Our work fits in a larger body of work on automatic paperassignment systems, which includes studies on the designof relevance scoring functions (Dumais & Nielsen, 1992;Mimno & McCallum, 2007; Rodriguez & Bollen, 2008;Liu et al., 2014) and appropriate quality metrics (Gold-smith & Sloan, 2007; Tang et al., 2012). These studies havecontributed to the development of conference managementplatforms such as EasyChair, HotCRP, and CMT that sup-port most major computer science conferences. Despite advances in automatic paper assignment, (Rennie,2016) highlights shortcomings of peer-review systems ow-ing to issues such as prejudices, misunderstandings, andcorruption, all of which serve to make the system ineffi-cient. For instance, the standard objective for assignment(say, Eq. (1)) seeks to maximize the total relevance of as-signed reviewers for the entire conference, which may beunfair to papers from under-represented areas. This has ledto efforts that design objective functions and constraints topromote fairness in the assignment process for all submit-ted papers (Garg et al., 2010; Long et al., 2013; Stelmakhet al., 2018; Kobren et al., 2019).Furthermore, the assignment problem faces the additionalchallenge of coping with the implicit bias of review-ers (Stelmakh et al., 2019). This issue is particularly preva-lent when authors of competing submissions participate inthe review process, as they have an incentive to providenegative reviews in order to increase the chance of theirown paper being accepted (Anderson et al., 2007; Thurner& Hanel, 2011). In order to alleviate this problem, recentstudies have devised assignment algorithms that promoteimpartiality in reviewers (Aziz et al., 2016; Xu et al., 2018).We contribute to this line of work by identifying and re-moving reviewers who adversarially alter their bids to beassigned papers for which they have adverse incentives.More recently, Jecmen et al. (2020) studied the bid manip-ulation problem and considered an orthogonal approach todefending against it. Their method focuses on probabilis-tic assignment and upper limits the assignment probabilityfor any paper-reviewer pair. As a result, the success rateof a bid manipulation attack is reduced. In contrast, ourwork seeks to limit the disproportional influence of mali-cious bids rather than uniformly across all paper-reviewerpairs, and further considers the influence of colluding at-tackers on the assignment system.
7. Conclusion
This study demonstrates some of the risks of paper bid-ding mechanisms that are commonly utilized in computer-science conferences to assign reviewers to paper submis-sions. Specifically, we show that bid manipulation attacksmay allow adversarial reviewers to review papers writtenby friends or rivals, even when these papers are outside oftheir area of expertise. We developed a novel paper assign- aking Paper Reviewing Robust to Bid Manipulation Attacks ment system that is robust against such bid manipulationattacks, even in settings when multiple adversaries colludeand have in-depth knowledge about the assignment system.Our experiments on a synthetic but realistic dataset of con-ference papers demonstrate that our assignment system is,indeed, robust against such powerful attacks. At the sametime, our system still produces high-quality paper assign-ments for honest reviewers. Our assignment algorithm iscomputationally efficient, easy to implement, and shouldbe straightforward to incorporate into modern conferencemanagement systems. We hope that our study contributesto a growing body of work aimed at developing techniquesthat can help improve the fairness, objectivity, and qualityof the scientific peer-review process at scale.
Acknowledgements
This research is supported by grants from the NationalScience Foundation NSF (III-1618134, III- 1526012, IIS-1149882, IIS-1724282, and TRIPODS-1740822, OAC-1934714), the Bill and Melinda Gates Foundation, andthe Cornell Center for Materials Research with fundingfrom the NSF MRSEC program (DMR-1719875), and SAPAmerica.
References
Ammar, W., Groeneveld, D., Bhagavatula, C., Beltagy,I., Crawford, M., Downey, D., Dunkelberger, J., Elgo-hary, A., Feldman, S., Ha, V., et al. Construction ofthe literature graph in semantic scholar. arXiv preprintarXiv:1805.02262 , 2018.Anderson, M. S., Ronning, E. A., De Vries, R., and Mar-tinson, B. C. The perverse effects of competition on sci-entists’ work and relationships.
Science and engineeringethics , 13(4):437–461, 2007.Aziz, H., Lev, O., Mattei, N., Rosenschein, J. S., andWalsh, T. Strategyproof peer selection: Mechanisms,analyses, and experiments. In
Thirtieth AAAI Confer-ence on Artificial Intelligence , 2016.Bhatia, K., Jain, P., and Kar, P. Robust regression via hardthresholding. In
Advances in Neural Information Pro-cessing Systems , pp. 721–729, 2015.Biggio, B., Nelson, B., and Laskov, P. Poisoning at-tacks against support vector machines. arXiv preprintarXiv:1206.6389 , 2012.Charlin, L. and Zemel, R. The toronto paper matching sys-tem: an automated paper-reviewer assignment system.In
ICML , 2013.Chen, Y., Caramanis, C., and Mannor, S. Robust sparseregression under adversarial corruption. In
InternationalConference on Machine Learning , pp. 774–782, 2013.Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein,C.
Introduction to Algorithms, Third Edition . The MITPress, 3rd edition, 2009. ISBN 0262033844.Dean, J. and Henzinger, M. R. Finding related pages in theworld wide web.
Computer networks , 31(11-16):1467–1479, 1999.Dumais, S. T. and Nielsen, J. Automating the assignmentof submitted manuscripts to reviewers. In
Proceedingsof the 15th annual international ACM SIGIR conferenceon Research and development in information retrieval ,pp. 233–244, 1992.Garg, N., Kavitha, T., Kumar, A., Mehlhorn, K., andMestre, J. Assigning papers to referees.
Algorithmica ,58(1):119–136, 2010.Goldsmith, J. and Sloan, R. H. The ai conference paperassignment problem. In
Proc. AAAI Workshop on Pref-erence Handling for Artificial Intelligence, Vancouver ,pp. 53–57, 2007.Hartvigsen, D., Wei, J. C., and Czuchlewski, R. The con-ference paper-reviewer assignment problem.
DecisionSciences , 30(3):865–876, 1999. aking Paper Reviewing Robust to Bid Manipulation Attacks
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru,C., and Li, B. Manipulating machine learning: Poisoningattacks and countermeasures for regression learning. In , pp.19–35. IEEE, 2018.Jecmen, S., Zhang, H., Liu, R., Shah, N. B., Conitzer, V.,and Fang, F. Mitigating manipulation in peer reviewvia randomized reviewer assignments. arXiv preprintarXiv:2006.16437 , 2020.Kobren, A., Saha, B., and McCallum, A. Paper match-ing with local fairness constraints. In
Proceedings of the25th ACM SIGKDD International Conference on Knowl-edge Discovery & Data Mining , pp. 1247–1257, 2019.Koh, P. W., Steinhardt, J., and Liang, P. Stronger data poi-soning attacks break data sanitization defenses. arXivpreprint arXiv:1811.00741 , 2018.Kuhn, H. W. The hungarian method for the assignmentproblem.
Naval research logistics quarterly , 2(1-2):83–97, 1955.Lawrence, N. Paper allocation for nips, 2014. https://inverseprobability.com/2014/06/28/paper-allocation-for-nips . [On-line; accessed on 2020-10-02].Liu, X., Suel, T., and Memon, N. A robust model for paperreviewer assignment. In
Proceedings of the 8th ACMConference on Recommender systems , pp. 25–32, 2014.Long, C., Wong, R. C.-W., Peng, Y., and Ye, L. On goodand fair paper-reviewer assignment. In , pp. 1145–1150. IEEE, 2013.Mei, S. and Zhu, X. Using machine teaching to identifyoptimal training-set attacks on machine learners. In
Pro-ceedings of the AAAI Conference on Artificial Intelli-gence , volume 29, 2015.Mimno, D. and McCallum, A. Expertise modeling formatching papers with reviewers. In
Proceedings of the13th ACM SIGKDD international conference on Knowl-edge discovery and data mining , pp. 500–509, 2007.Rennie, D. Let’s make peer review scientific.
Nature , 2016.Rodriguez, M. A. and Bollen, J. An algorithm to determinepeer-reviewers. In
Proceedings of the 17th ACM con-ference on Information and knowledge management , pp.319–328, 2008.Shah, N. B., Tabibian, B., Muandet, K., Guyon, I., andVon Luxburg, U. Design and analysis of the nips 2016review process.
The Journal of Machine Learning Re-search , 19(1):1913–1946, 2018. Stelmakh, I., Shah, N. B., and Singh, A. Peerreview4all:Fair and accurate reviewer assignment in peer review. arXiv preprint arXiv:1806.06237 , 2018.Stelmakh, I., Shah, N., and Singh, A. On testing for bi-ases in peer review. In
Advances in Neural InformationProcessing Systems , pp. 5287–5297, 2019.Stent, A. and Ji, H. A review of reviewerassignment methods, 2018. https://naacl2018.wordpress.com/2018/01/28/a-review-of-reviewer-assignment-methods .[Online; accessed on 2020-10-02].Tang, W., Tang, J., Lei, T., Tan, C., Gao, B., and Li, T.On optimization of expertise matching with various con-straints.
Neurocomputing , 76(1):71–83, 2012.Thurner, S. and Hanel, R. Peer-review in a world with ra-tional scientists: Toward selection of the average.
TheEuropean Physical Journal B , 84(4):707–711, 2011.Vijaykumar, T. N. Potential organized fraud inacm/ieee computer architecture conferences,2020. https://medium.com/@tnvijayk/potential-organized-fraud-in-acm-ieee-computer-architecture-conferences-ccd61169370d .[Online; accessed on 2020-10-13].Weinberger, K., Dasgupta, A., Langford, J., Smola, A., andAttenberg, J. Feature hashing for large scale multitasklearning. In
Proceedings of the 26th annual internationalconference on machine learning , pp. 1113–1120, 2009.Xiao, H., Biggio, B., Nelson, B., Xiao, H., Eckert, C., andRoli, F. Support vector machines under adversarial labelcontamination.
Neurocomputing , 160:53–62, 2015.Xu, Y., Zhao, H., Shi, X., and Shah, N. B. Onstrategyproof conference peer review. arXiv preprintarXiv:1806.06266 , 2018. aking Paper Reviewing Robust to Bid Manipulation Attacks
Supplementary Material: Making Paper Reviewing Robust to BidManipulation Attacks
A. Dataset Construction
In this section, we describe how we subsampled data from the Semantic Scholar Open Research Corpus (S2ORC) (Ammaret al., 2018), extracted reviewer/paper features such as subject area and TPMS, and simulated bids using citation. Our datais publicly released for reproducibility and to facilitate future research. A.1. Conference Simulation
The goal of our dataset is to simulate a NeurIPS-like conference environment, where the organizers assign reviewers topapers based on expertise and interest. We first retrieve the collection of 6956 papers from S2ORC that are published inML/AI/CV/NLP venues between the years 2014-2015, which includes the following conferences: AAAI, AISTATS, ACL,COLT, CVPR, ECCV, EMNLP, ICCV, ICLR, ICML, IJCAI, NeurIPS, and UAI. We believe the diversity of subject areasrepresented by the above conferences is an accurate reflection of typical ML/AL conferences in recent years. We will referto this collection of papers as the corpus . Subject areas.
Most conferences require authors to indicate primary and secondary subject areas for their submittedpapers. However, the S2ORC only contains a field of study attribute for most of the retrieved papers in the corpus, whichis often the broad category of computer science . To identify the suitable fine-grained subjects for each paper, we adopt anunsupervised learning approach of clustering the papers by relatedness and treating each discovered cluster as a subjectarea.Similarity is defined in terms of co-citations – a common signal used in information retrieval for discovering relateddocuments (Dean & Henzinger, 1999). For a paper p , let N ( p ) denote the union of in-citations and out-citations for p . Thesimilarity between two papers p, q is defined as σ ( p, q ) = | N ( p ) ∩ N ( q ) | (cid:112) | N ( p ) | · (cid:112) | N ( q ) | , (S1)which is the cosine similarity in document retrieval. We perform agglomerative clustering using average linkage to reducethe set of papers to 1000 clusters. After removing small cluster (less than 5 papers), we obtain 368 clusters to serve assubject areas. Table S1 shows a few sample clusters along with papers contained in the cluster. Most of the discoveredclusters are highly coherent with members sharing keywords in their titles despite the definition of similarity depending entirely on co-citations.To populate the list of subject areas for a given paper p , we first compute its subject relatedness to a cluster C by: σ ( p, C ) = 1 | C | (cid:88) q ∈ C σ ( p, q ) . (S2)Given the set of clusters representing subject areas, we identify the top-5 clusters according to σ ( p, C ) to be the list ofsubject areas for the paper p , denoted subj ( p ) . Reviewers.
The S2ORC dataset contains entries of authors along with their list of published papers. We utilize thisinformation to simulate reviewers by collecting the set of authors who has cited at least one paper from the corpus. Thetotal number of retrieved authors is 234,598. Because the vast majority of retrieved authors are very loosely related to thefield of ML/AI, they would not be suitable reviewer candidates for a real ML/AI conference. Therefore, we retain onlyauthors who have cited at least 15 papers from the corpus to serve as reviewers. We also remove authors who cited more https://drive.google.com/drive/folders/1khI9kaPy_8F0GtAzwR-48Jc3rsQmBhfe?usp=sharing https://scikit-learn.org/stable/modules/clustering.html aking Paper Reviewing Robust to Bid Manipulation Attacks Subject Area Papers
Multi-task learning Encoding Tree Sparsity in Multi-Task Learning: A Probabilistic FrameworkMulti-Task Learning and Algorithmic StabilityExploiting Task-Feature Co-Clusters in Multi-Task LearningEfficient Output Kernel Learning for Multiple TasksLearning Multiple Tasks with Multilinear Relationship Networks
Etc.
Video segmentation Efficient Video Segmentation Using Parametric Graph PartitioningVideo Segmentation with Just a Few StrokesCo-localization in Real-World ImagesSemantic Single Video Segmentation with Robust Graph RepresentationPatchCut: Data-driven object segmentation via local shape transfer
Etc.
Topic modeling On Conceptual Labeling of a Bag of WordsTopic Modeling with Document Relative SimilaritiesDivide-and-Conquer Learning by Anchoring a Conical HullSpectral Methods for Supervised Topic ModelsModel Selection for Topic Models via Spectral Decomposition
Etc.
Feature selection Embedded Unsupervised Feature SelectionFeature Selection at the Discrete LimitBayes Optimal Feature Selection for Supervised Learning with General Performance MeasuresReconsidering Mutual Information Based Feature Selection: A Statistical Significance ViewUnsupervised Simultaneous Orthogonal basis Clustering Feature Selection
Etc.
Table S1.
Sample subject areas and paper titles of cluster members. than 50 papers from the corpus, since these reviewers represent senior researchers that would typically serve as area chairs.The number of remaining reviewers is , .Most conferences also solicit self-reported subject areas from reviewers. We simulate this attribute by leveraging theclusters discovered through co-citation. For each subject area C , we count the number of times C appeared in subj ( p ) foreach of the papers p that the reviewer r has cited. The 5 most frequently appearing clusters (ties are broken randomly)serve as the reviewer’s subject areas, denoted subj ( r ) . TPMS score.
The TPMS score (Charlin & Zemel, 2013) is computed by measuring the similarity between a reviewer’sprofile – represented by a set of papers that the reviewer uploads – and a target paper. We simulate this score using thelanguage model-based approach from the original TPMS paper, which we detail below for completeness. For a reviewer r , let A r denote the bag-of-words representation for the set of papers that the reviewer has authored. More specifically, wecollect the abstracts of the papers that r has authored, remove all stop words, and pool the remaining words together into A r as a multi-set. Similarly, let A p denote the bag-of-words representation for the abstract of a paper p . The simulatedTPMS is computed as: TPMS r,p = (cid:88) w ∈ A p log f rw , (S3)where f rw is the Dirichlet-smoothed normalized frequency of the word w in A r . Let D denote the bag-of-words represen-tation for the entire corpus of (abstracts of) papers, and let D ( w ) (resp. A r ( w ) ) denote the occurrences of w in the corpus(resp. A r ). Then f rw := (cid:18) | A r || A r | + β (cid:19) | A r ( w ) || A r | + (cid:18) β | A r | + β (cid:19) | D ( w ) || D | , aking Paper Reviewing Robust to Bid Manipulation Attacks where β is a smoothing factor. We set β = 1000 in our experiment. The obtained scores are normalized per paper between0 and 1. A.2. Simulating Bids
The most challenging aspect of our simulation is the bids. At first, it may seem natural to simulate bids using citations,since it is a proxy of interest and can be easily obtained from the S2ORC dataset. However, we have observed that bidsare heavily skewed towards a few very influential papers, while the distribution of bids is much more uniform across allpapers. To overcome this issue, we instead model a reviewer’s bidding behavior based on the following assumptions:1. A reviewer will only bid on papers from subject areas that he/she is familiar with.2. Given two papers from the same subject area, a reviewer favors bidding on a paper whose title/abstract is a better matchwith the reviewer’s profile.We define several scores that reflect the above aspects and combine them to obtain the final bids. In practice, reviewers willoften also rely on TPMS to sort the papers to bid on. However, since our simulated TPMS depends entirely on the abstract,we omit TPMS in our bidding model. Nevertheless, we have observed empirically that TPMS is highly correlated with thebids that we obtain.
Subject score.
We leverage citation to reflect the degree of interest in the subject of a paper. Let icf ( q ) denote the inversecitation frequency (ICF) of a paper q in the corpus:icf ( q ) = log q . The purpose of the ICF is to down-weight commonly cited papers to avoid overcrowding of bids. Denote by C ∗ ( q ) the topcluster that q belongs to according to Eq. (S2). The subject score for a paper p is defined as:subject-score r,p = (cid:88) q : r cites q icf ( q ) | C ∗ ( q ) | { p ∈ C ∗ ( q ) } . (S4)In other words, for each paper q that r cites, we merge all papers from the same subject area of q , represented by C ∗ ( q ) ,into the reviewer’s pool. Each paper in C ∗ ( q ) is weighted by the reciprocal of the cluster size and the ICF of q , and thesubject score is the resulting sum after accumulating over all papers q that the reviewer cites. Note that every paper withinthe same subject cluster has the exact same subject score, which is non-zero only if the reviewer has bid on a paper withinthis subject area. This property reflects the assumption that a reviewer is only interested in papers from familiar subjectareas, and is indifferent to different papers in the same subject absent of title/abstract information. To avoid overcrowdingby frequently cited papers, we set subject-score r,p = 0 for any paper p that received over 1000 citations. Title/abstract score.
To measure the degree of title/abstract similarity between a reviewer and a paper, we computethe inner product between the TF-IDF vectors of the reviewer’s and paper’s title/abstract. Let idf ( w ) denote the in-verse document frequency of a word w . For each reviewer r , let tf-idf ( r ) denote the vector, indexed by words, suchthat tf-idf ( r ) w = ( | A r ( w ) | / | A r | ) · idf ( w ) for each word w . Similarly, we can define the TF-IDF vector for a paper p , andthe abstract score between a pair ( r, p ) is given by the inner product:abstract-score r,p = tf-idf ( r ) · tf-idf ( p ) . (S5)We can define the title score in an analogous manner based on the bag-of-words representation of titles instead of abstracts. Bidding.
We simulate bids by combining the subject/title/abstract scores as follows. First, we define a total score :total-score r,p = ( title-score r,p + abstract-score r,p ) · subject-score r,p , (S6)which reflects the assumptions we made about a reviewer’s bidding behavior, i.e. , a higher total score reflects a higherreviewer interest in the paper. The total score gives us a ranking of papers in the corpus, denoted by rank r ( p ) , for eachpaper p . To obtain the positive bids, we randomly retain high-ranked papers with a decaying probability: Pr( r bids on p ) = 1 / (1 + exp( α · ( rank r ( p ) − µ )) , aking Paper Reviewing Robust to Bid Manipulation Attacks (a) distribution of Figure S1.
Distribution of the number of positive bids before and after subsampling. where α and µ are hyperparameters that control the steepness of the drop in sampling probability for low-ranked papers,and the average number of papers that each reviewer bids on. We set α = 0 . and µ = 80 in our experiment.The quality of bids obtained from this sampling procedure is very reasonable. However, the majority of papers had veryfew bids (see Fig. S1(a)) – contrary to statistics observed in a real conference such as NeurIPS-2016 (see Figure 1 in (Shahet al., 2018)). To match the distribution of the number of bids per reviewer/paper to that of a real conference, we furthersubsample papers (resp. reviewers) to encourage selecting ones with more bids. The distribution of the number positivebids per reviewer/paper after subsampling is shown in Fig. S1(b). Our finalized conference dataset contains m = 2483 reviewers and n = 2446 submitted papers – a realistic balance of papers and reviewers for recent ML/AI conferences.Finally, some conferences allow more fine-grained bids, such as in a pinch , willing and eager for conferences managedusing CMT. To simulate bid scores that reflect the degree of interest, we quantize the total score of all positive bids into thediscrete range { , , } based on the distribution of bid scores in a real conference: at a ratio of for the bids 1, 2and 3. B. Features and Training
We provide details regarding feature extraction and model training in this section. To fully imitate a conference man-agement environment, we extract relevant features from papers and reviewers that are obtainable in a realistic scenario,including: paper/reviewer subject area (5 areas for each), bag-of-words vector for paper title, and (simulated) TPMS.These features are further processed and concatenated as input to the linear regression model in Section 3.Table S2 lists all the extracted features and their dimensions. Paper title (PT) is the vectorized count of words appearing in aking Paper Reviewing Robust to Bid Manipulation AttacksFeatures paper titles (PT) paper subject area (PS) reviewer subject area (RS)
930 368 368
Features intersected subject area (IS) TPMS vector (TV) RS ⊗ PS
368 12 135424
Features RS ⊗ PT IS ⊗ PT IS ⊗ TV Table S2.
Extracted features and their dimensionalities. See the text for details. k=1 k=2 k=3 k=4 k=5 k=6 k=7 k=8 k=9 k=10AP@k per reviewer train test
AP@k per paper train test
Table S3.
Average precision@k per reviewer/paper for the trained linear regressor. M a T P R Top-50 M d = 1 M d = 2 M d = 3 M d = 4 M d = 5 Figure S2.
TPR for detecting colluding white-box attacks that succeed in achieving top-50 rank. the paper’s title, while paper subject area (PS), reviewer subject area (RS) and intersected subject area (IS) are categoricalfeatures represented using binary vectors. The first dimension for the TPMS vector (TV) is the TPMS score for thereviewer-paper pair. We also quantize the raw TPMS into 11 bins and use the bin index as well as the quantized scores,which results in the remaining 11 dimensions for the TPMS vector.RS ⊗ PS, RS ⊗ PT, IS ⊗ PT and IS ⊗ TV are additional quadratic features that capture the interaction between feature pairs.The introduction of these quadratic features results in a very high-dimensional, albeit extremely sparse feature vector, andhence many dimensions could be collapsed without a significant impact to performance. We apply feature hashing (Wein-berger et al., 2009) to the quadratic features at a hash ratio of 0.01, which reduces the total feature dimensionality to d = 10 , . Model performance.
To validate our linear regression model and the selected features, we test the average precision at k(AP@k) for the trained model on a train-test split. Table S3 shows the AP@k per reviewer (P@k for finding papers relevantto a reviewer, averaged across all reviewers) and the AP@k per paper for the linear regressor. It is evident that both metricsare at an acceptable level for real world deployment, and the train-test gap is minimal, indicating that the model is able togeneralize well beyond observed bids.We also perform a qualitative evaluation of the end-to-end assignment process using the relevance scoring model. We aking Paper Reviewing Robust to Bid Manipulation Attacks
Reviewer Assigned Papers Bid Scores
Kavita Bala 1. Learning Lightness from Human Judgement on Relative Reflectance2. Simulating Makeup through Physics-Based Manipulation of Intrinsic Image Layers3. Learning Ordinal Relationships for Mid-Level Vision4. Automatically Discovering Local Visual Material Attributes5. Recognize Complex Events from Static Images by Fusing Deep Channels6. Learning a Discriminative Model for the Perception of Realism in Composite Images 333000Ryan P. Adams 1. Stochastic Variational Inference for Hidden Markov Models2. Parallel Markov Chain Monte Carlo for Pitman-Yor Mixture Models3. Celeste: Variational Inference for a Generative Model of Astronomical Images4. Measuring Sample Quality with Stein’S Method5. Parallelizing MCMC with Random Partition Trees6. Hamiltonian ABC 300000Peter Stone 1. Qualitative Planning with Quantitative Constraints for Online Learning of Robotic Behaviours2. An Automated Measure of MDP Similarity for Transfer in Reinforcement Learning3. On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems4. A Framework for Task Planning in Heterogeneous Multi Robot Systems Based on Robot Capabilities5. A Strategy-Aware Technique for Learning Behaviors from Discrete Human Feedback6. Stick-Breaking Policy Learning in Dec-Pomdps 333300Yejin Choi 1. Don’T Just Listen, Use Your Imagination: Leveraging Visual Common Sense for Non-Visual Tasks2. Segment-Phrase Table for Semantic Segmentation, Visual Entailment and Paraphrasing3. Refer-To-As Relations as Semantic Knowledge 300Emma Brunskill 1. Policy Evaluation Using the Ω -Return2. Towards More Practical Reinforcement Learning3. High Confidence Policy Improvement4. Sample Efficient Reinforcement Learning With Gaussian Processes5. Policy Tree: Adaptive Representation for Policy Gradient6. Abstraction Selection in Model-Based Reinforcement Learning 333330Elad Hazan 1. Online Linear Optimization via Smoothing2. Online Learning for Adversaries with Memory: Price of Past Mistakes3. Hierarchies of Relaxations for Online Prediction Problems with Evolving Constraints4. Hard-Margin Active Linear Regression5. Online Gradient Boosting6. Robust Multi-Objective Learning With Mentor Feedback 300000 Table S4.
Assigned papers for six representative reviewers. select six representative (honest) reviewers from our dataset – Kavita Bala , Ryan P. Adam , Peter Stone , Yejin Choi ,Emma Brunskill and Elad Hazan – representing distinct areas of interest in ML/AI. Table S4 shows the assigned papersfor the selected reviewers, which appear to perfectly match the area of expertise for the respective reviewers. Many of theassigned papers have a bid score of 0 despite being very relevant for the reviewer, which shows that the scoring model isable to discover missing bids and improve the overall assignment quality. C. Additional Experiment on White-box Attack
In Section 5 we evaluated our defense against white-box attacks that succeeded in securing the target paper assignment.However, in doing so, it is possible that malicious reviewers that did not succeed initially will inadvertent become high-ranked after other reviewers are removed from the candidate set. Therefore, it may be necessary to detect all attackinstances in the candidate set rather than ones that were successfully assigned. https://scholar.google.com/citations?user=Rh16nsIAAAAJ https://scholar.google.com/citations?user=grQ_GBgAAAAJ https://scholar.google.com/citations?user=qnwjcfAAAAAJ https://scholar.google.com/citations?user=vhP-tlcAAAAJ https://scholar.google.com/citations?user=HaN8b2YAAAAJ https://scholar.google.com/citations?user=LnhCGNMAAAAJ aking Paper Reviewing Robust to Bid Manipulation Attacks [ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ][ , ] reviewer's original rank s u cc e ss r a t e attack success rate VS original reviewer rank M a =1 M a =2 M a =3 M a =4 M a =5 M a =10 Figure S3.
Success rate after the colluding black-box attack against an undefended linear regression scoring model.
Fig. S2 shows the detection TPR for all attackers that were initially ranked below K = 50 but managed to move into thecandidate set after the attack. Since this attacker pool includes many that obtained a relatively low rank, detection TPRis much higher than that of Fig. 3. For instance, for M d = 5 , even when the colluding party is significantly larger at M a = 10 , detection remains viable with a TPR of more than 40%. This experiment shows that our detection mechanismis unlikely to inadvertently increase the success rate of failed attacks. D. Black-box Attack
The white-box attack from Section 4.1 assumed that the adversary has extensive knowledge about the assignment systemand all reviewers’ features/bids. In this section, we propose a more realistic colluding black-box attack , where the adversaryonly has access to the features/bids of reviewers in the colluding party. This attack represents a reasonable approximationof what a real world adversary could achieve, and we show that it is potent against the scoring model in Section 3 absentof any detection mechanism.
Colluding black-box attack.
The failure of the simple black-box attack from Section 2 is due to the malicious reviewer r bidding positively only on a single paper, instead of also on a group of papers that are similar to p . We alter the attackstrategy by giving the largest bid score to U = 60 papers p (cid:48) that are most similar to p (including p itself). In practice, thiscan be done by comparing the titles and abstracts of p (cid:48) to the target paper p . We simulate this attack in our experiment byselect papers p (cid:48) whose feature vector X r,p (cid:48) have a high inner product with X r,p .We can extend this strategy to allow for colluding attacks. The malicious reviewer first selects M a − reviewers withthe most similar background to form the colluding group. In simulation, we measure reviewer similarity by the innerproduct between their respective reviewer-related features. Mimicking r ’s paper selection strategy, every reviewer r (cid:48) in thecolluding group now gives the largest bid score to the U = 60 papers p (cid:48) with the highest inner product between X r (cid:48) ,p (cid:48) and X r,p . Attack performance.
Fig. S3 shows the success rate of the colluding black-box attack against the linear regressionmodel. Note that this attack is much more successful than the simple black-box attack from Section 2, which had a successrate of 0% for all reviewers below rank 16. Here, the success rate before attack is initially 0%, which increased to close to5% after attack even without collusion ( M a = 1 ). Increasing the colluding party size strictly improves attack performance,while attackers with lower initial rank are less successful. Compared to the white-box attack from Section 4.1 (see Fig. 2),the colluding black-box attack is substantially less potent as expected. Detection performance.
For completeness, we evaluate the detection algorithm from Section 4.2 against successfulcolluding black-box attacks. In Fig. S4, we plot detection TPR as a function of the size of the colluding party ( M a ) for aking Paper Reviewing Robust to Bid Manipulation Attacks M a T P R TPR for detecting successful black-box attacks M d = 1 M d = 2 M d = 3 M d = 4 M d = 5 M a T P R Top-50 M d = 1 M d = 2 M d = 3 M d = 4 M d = 5 Figure S4.
TPR for detecting colluding black-box attacks that succeeded in securing the assignment (left) and achieving a top-50 rank(right). various choices of the detection parameter M d . For both attacks that succeeded (left) and ones that achieved a top-50(right) rank, detection TPR is close to 1 when M a ≤ M d , and remains very high for M a > M d . For instance, at M a = 10 and M d = 5= 5