A Robust Opinion Spam Detection Method Against Malicious Attackers in Social Media
AA Robust Opinion Spam Detection Method AgainstMalicious Attackers in Social Media
Amir Jalaly Bidgoly a, ∗ , Zoleikha Rahmanian a a Department of Information Technology and Computer Engineering, University of Qom, Qom, Iran.
Abstract
Online reviews are potent sources for industry owners and buyers, however, oppor-tunistic people may try to destruct or promote their desired product by publishing fakecomments named spam opinion. So far, many models have been developed to detectspam opinions, but none have addressed the issue of spam attacks. It is a way a smartspammer can deceive the system in a manner in which he can continue generatingspams without the fear of being detected and blocked by the system. In this paper, thespam attacks are discussed. Moreover, a robust graph-based spam detection methodis proposed. The method respectively estimates honesty, trust and reliability values ofreviews, reviewers, and products considering possible deception scenarios. The paperalso presents the efficiency of the proposed method as compared to other graph-basedmethods through some case studies.
Keywords:
Spam Attack, Spam Detection, Spam Opinion, Deception, Robustness.
1. Introduction
Due to the importance of users’ reviews on social networks and commercial web-sites, many malicious people are struggling to meet their false goals by publishing falseopinions. Some of them are trying to promote their non quality goods; others are tryingto destroy the quality of their competitors. Thus, researchers have been working since2007 to provide an appropriate solution to identify spam reviews [10]. ∗ Corresponding author:
Email address: [email protected] (Amir Jalaly Bidgoly)
Preprint submitted to Elsevier August 21, 2020 a r X i v : . [ c s . S I] A ug n spam detection systems, deception means those so called spam attackers who areaware of the existence of the spam detection method and its mechanism, mislead thesystem and bypass the mechanism to spread opinion spams according to its maliciousgoals. For example, in a repeat-based method (e.g.[11]), if spam attackers are aware ofthe system’s method, they can easily deceive the system by collecting data from uniqueand non repetitive reviews. As a further instance, in text-based systems [18, 20], if thespam attackers are aware of what the words are as a benchmark for spam recognition,they can deceive the system by avoiding their choice.Robustness in these methods is the ability of the system against these deceptivebehaviors. In a robust spam detection system, the spam attacker should not be ableto deceive the system even with the complete awareness of the detection mechanismand performing deceiving behaviors. To our knowledge, no previous research has beenpointed out the robustness issue of spam detection systems against attacks, and it can beargued that this area of research is still in the early stages. As can be seen in Fig.1, thispaper proposed an unsupervised graph-based method in which the graph nodes includereviews, reviewers, and products. The proposed method may detect spam attackers indifferent scenarios.For this aim, it calculates the trust score for the reviewer, the honesty score forreviews, and the reliability score for products; their values are updated in an iterativealgorithm. The reliability score should represent the true reputation value of the prod-ucts ignoring spammers. The spams and spam attackers are expected to be identifiedwith having low honesty and trust values. The proposed method is compared with twoother graph-based methods [22, 4]; the results show that the proposed method has im-proved the system robustness. One challenge to have a complete assessment in spamdetection methods is the lack of an appropriate technique to generate spam reviews;thus, the paper also developed a simulation tool, which can produce a large number ofreviews tailored to the desired behavior pattern.The paper continues as follows. In the next section, related works are reviewed.In section III the motivation and the foundations of the research are presented. Thesimulator is described in section IV. Further, the proposed method and algorithm arepresented in section V and section VI respectively. Section VII describes the method2 igure 1: The interact of different graph nodes using some case studies, and finally, the paper ends in section VIII with the conclusionand future works.
2. Related work
Approaches for detecting spam comments are categorized in several ways. Somedivide them into two general types: supervised [7, 15, 16, 18, 20, 21] and unsupervisedmethods[1, 4, 12, 23]. However, in some other researches, they are categorized intothree categories: spam reviews, spam attacker, group spam attacker [8].In the set of spam reviews, the contents of reviews are studied and evaluated. Oneof the most important approaches in this category is repeated-based detection. Thesemethods try to identify spam reviews by going through repetitive patterns of reviewsfrom the same or different reviewers about similar or different products [11, 15]. Inaddition to this, concept repetition can also be introduced as a measurement criterionfor spam comments; the method provided by Alger et al. to identify spam comments[2]. Doing multiple counterfeit reviews is time consuming and costly. Spam attackersoften do not produce a large number of exclusive counterfeit reviews. They tend to copythe existing text. Therefore, identifying similar opinions is a central part of detectingspam comments. Some literature uses a linguistic character in the text of review [3, 19].In the field of identifying spammers, some methods [1, 4, 23] use inter relationships3 onest Behavior Attack BehaviorAfter a while (collecting trust)
Figure 2: User deceptive behaviour over time between the review, the reviewer, and product graph, as shown in Fig.1, to identify thespammer and also compute the trustworthiness of the reviewer, the honesty of review,and the reliability of the store.Some researchers believe that spammers use a specific period to generate spamcomments. The numbers of reviews rise dramatically in that interval; thus, they useburst patterns to identify spam attacker[5, 25] . Some of them, for example, use timeseries to identify spammers[9, 26]. Few pieces of research have been yet done in spam-mer groups identification [17, 24]. The proposed algorithm in the research includes twosteps: 1) the extraction of the repeated pattern, and 2) ranking the group based on thespam group indices.
3. Motivation and research foundations
Despite the many types of research that have been taken to identify spam comments,so far, no one has been addressing the issue of spam attacks. Today, smart spammers,with the knowledge of spam detection methods, can easily deceive the spam detectionsystem and continue their malicious activity. For example, in text-based systems, Spamattacker deceives the method by modifying the text of reviews.Deception can be performed in two manners: 1) deceptive behavior over time (inlength deception), and 2) deceptive behavior over the product (in width deception). Indeceptive behavior over time as can be observed in Fig.2 (the quality of all productsis 5), spam attacker exhibits honest behavior for a while and after gaining enough4 onflicting Behavior
Fake ReviewHonest Review
Figure 3: User deceptive behavior in width trust, discloses its deceitful behavior. It means that smart spammer has a conflictingbehavior over time: honest behavior in the first period to increase his/her credential,and dishonest behavior in the next period to achieve his/her malicious goals using thegained social trust.In contrast to in length deception, in deceptive behavior over the product (in widthdeception), the spam attacker exhibits conflicting behavior over different products. Ascan be seen an instance of this attack in Fig.3, he sends fake reviews for the productthat wants to slander (here’s product 2), while writing honest reviews on other products(here’s product 2, 3) to keep its social trust value.As mentioned earlier, detecting these types of attacks need analyzing the completeknowledge of nodes behaviors, which are used by graph-based spam detection meth-ods; however, the current graph-based techniques almost can be deceived via men-tioned attacks.
4. Simulator software
Since none of the current spam data sets include the spam attacks, and generationof enough human opinions and performing mentioned attacks are hard in practice, aSimulation tool has been developed on the basis of PDETool platform[13, 14] to sim-ulate the spam attack scenarios and evaluate the proposed method. The tool simulatesthe reviewing process in an e-Commerce website and can generate enough samples for5 igure 4: A view of the simulator environment any given scenario. Moreover, it is capable of simulating any other desired scenariothat may be required in evaluating spam detection methods.The tool defines the reviewing environment as a graph, which includes two typesof node: 1) product nodes, and 2) reviewer one. Each product has a defined quality.A reviewer should be connected to a product using connectors to produce a reviewscenario. The reviewer nodes have two sub types: honest and spammer reviewers,which are represented by blue and red users in the tool graphical user interface. Thehonest reviewers honestly score the products. To be more specific, their score hasa normal distribution with the mean of product quality and a given variance. Thevariance default value is 0.5; however, can be changed by the user. In contrast, thespammer behavior is defined using a provided script, which enables the modeler todefine any complicated scenario including spam attacks. Moreover, the software isalso capable of defining individual spam behavior for each product. An instance of thedefined mode in this tool is given in Fig.4 As shown in the figure, there are 3 reviewersand 3 products in this model. The model includes a spam reviewer (the 3rd reviewer)who falsely scores the last product (i.e., product no 3) across the red connection.6 able 1: Symbols used in proposed model
Definition NotationReview vReviewer RProduct PScore of review v v.scoreProduct of review v v.productReview of reviewer r r.reviewReviewer of product p p.reviewerReviews of product pby reviewer r p.r.reviewsThe maximum differencebetween thescore of reviewerand the majorityof the community WNumber of review review.number
5. Proposed model for detecting Spam attacker
In this section, a method for spam and spam attack detection is proposed, whichis robust against mentioned deception scenarios in previous sections. The method isa graph-based model that is defined by three types of nodes: review, reviewer, andproduct. It estimates the value of trustworthiness, honesty, and reliability for thesenodes respectively, which are demarcated in the following in detail. Symbols used inthe formulas are given in Table.1.
Reviewer Trustworthiness : The reviewer’s trust score (denoted by T ( r ) ) is thenormal honest behavior performed by the user. It is estimated using the honesty meanof his published reviews and its sequence. The sequence helps the model to give moreweight to recent reviews, which is essential for in length spam attack detection. The7rust of a reviewer is calculated through the following formula: T ( r ) = (cid:80) ∀ review ∈ r.review review.number ∗ H ( r.review ) (cid:80) ∀ review ∈ r.review review.number (1) Review Honesty : The review’s honesty (denoted by H ( v ) ) indicates the accuracyof the opinion. The honesty value is estimated based on its maximum distant fromthe estimation of the true quality of the product (i.e., product reliability). The honestyscore is defined as follows: H ( v ) = −| normalized ( v.score ) − R ( v.product ) | W R ( v.product ) > . → W = RR ( v.product ) < . → W = 1 − R (2)The honesty score is a value between zero and one. The higher value indicates amore honest review. The value of one means that the review is perfectly honest since itfully matches the product reliability. It is not able that the review scores (i.e., v.score should be normalized in the range [0, 1] before being used in the above equation (inthe case of the systems uses 1-5 scores). Product Reliability : The reliability score of the product (denoted as R ( p ) ) is theestimation of the true quality of the product. It depends on both the trust score of thereviewers and their reviews honesty. The product reliability score is defined as follows.The score is a value in the range of [0, 1]. R ( p ) = (cid:80) r ∈ p.reviewers (cid:80) v ∈ p.r.reviews T ( r ) ∗ H ( v ) ∗ v.score (cid:80) r ∈ p.reviewers (cid:80) v ∈ p.r.reviews T ( r ) ∗ H ( v ) (3) Iterative Computation Algorithm
As all trustworthiness, honesty, and reliability values are interdependent for esti-mating them, the mentioned formulas should be computed in a loop until the resultconverges to a value. The algorithm output is independent of the initial values ofthe nodes (trustworthiness, honesty, and reliability). The proposed algorithm for thismethod can be seen in Fig.5. 8 igure 5: The algorithm of the proposed model
6. Deception scenarios and algorithms implementation
In this section, the efficiency of the proposed method (ROSD ) is presented usingsome spam attack scenarios. Moreover, the results are compared with other well knowngraph-based approaches including Wang’s [23] and Fayazbakhsh’s models [4]. In allscenarios, 1000 reviews are generated using the simulation tool, and the results of allthree methods are presented and compared. Since the result values for ROSD is [0, 1],WNG is [-1, +1], and FYZ is [0, 1], the bench marking is done through the followingdefined measure: 1) the ability to detect spams and spammers, and 2) the number ofdeviations that the spam attacker can create in the actual value of products reliability.Whatever spam attacker cannot deflect the reliability score of a product from its actualscore indicates a better system performance. It is noteworthy to consider Fayazbakhsh’smodel that calculates only a suspicious score of reviewers and products while has nosolution for calculating the suspicious score of reviews.
In the first scenario, the spammer tries to slander a product without any deceptionbehavior. As can be seen in Fig.6, there are 10 reviewers and three products. The lastreviewer is a spammer who gives zero to product3. In this scenario, spam attacker9 igure 6: Scenario 1: Simple spamming against a product wants to slander the product and does not use deception scenario. It is important tonote that the true quality of all products is considered to be 3 out of 5.The results are shown in Table.2 As it is presented, ROSD and WNG can find aspammer, while FYZ is unable to detect the spammer as FYZ only tries to find spammerthat sends high score (4 or 5 scores). Also in finding the spam reviews, both of themodels have acceptable results. As can be seen in the last row of Table.2, in all threemodels spammer has not been able to change the reliability score of the target productsignificantly from the actual quality. Also as shown in Table.3., if spammer simplypublishes positive and fake opinion to promote a product in the same conditions andthe spammer constantly gives the score 5 out of 5 to the corresponding product then thesame results will be achieved. Note that the true quality of the corresponding product(product3) is considered to be 1 out of 5.
In the second scenario, an over product attack is simulated. The simulation modelis represented in Fig.7. 8 and is similar to previous ones; however, the spammer isconnected to all products. He gives a right score to all products but the last one. As itis presented, only the proposed model can find spam attackers, and this suggests thatother models are deceived in this way since they are unable to find the spam attacker.10 able 2: Results table for the scenario1- slandering a product
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.8667 1 0.9975The average trustworthiness rating of spam attacker 0 -1 0.9916The average honesty score of non spam reviews 0.8662 0.9238 -The average honesty score of spam reviews 0 -1 -The average reliability of target products before spammer 0.6 1 0.9942The average reliability of target products after spammer 0.6067 1 0.9953Deviation value in product reliability 0.006 0 0.001
Table 3: Results table for the scenario1- promoting a product
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.8789 1 0.9963The average trustworthiness rating of spam attacker 0 -1 0.9967The average honesty score of non-spam reviews 0.8759 0.8917 -The average honesty score of spam reviews 0 -1 -The average reliability of target products before spammer 0.2 -1 0.9941The average reliability of target products after spammer 0.1915 -1 0.9954Deviation value in product reliability 0.0085 0 0.00111 igure 7: Scenario 2: An over product attack
However, in finding the spam review, both models have acceptable results. As can beseen in the last row of Table.4, in all three models, spam attacker has not been ableto change the reliability score of the target product critically from the actual quality.Also, in this scenario, if spam attacker tries to promote a product and using false pos-itive scores instead of slandering, as indicated in Table.5, the same results have beenachieved (Here the true score of the selected product is assumed to be 1 out of 5).
In this scenario, a slandering attack over time is simulated. As can be seen in Fig.8,there are 3 reviewers and 3 products. The last reviewer is spam attacker who gives atrue score of 3 to product 3 in the intervals of time (20 first reviews), and then givesthe score 1 in the intervals of time (20-second reviews); this process continues until theend of the review generation. It should be noted that the true quality of all products isconsidered to be 3 out of 5.As illustrated in Table.6, in this scenario results are similar to the earlier scenario,and the proposed model only may detect spam attacker. The results for promotingattack over time are the same; it is shown in Table.7.12 able 4: Results table for the scenario 2- Selective product slandering
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.8719 1 0.9960The average trustworthiness rating of spam attacker 0.5730 1 0.9966The average honesty score of non spam reviews 0.8684 0.9171 -The average honesty score of spam reviews 0 -1 -The average reliability of target products before spammer 0.6 1 0.9965The average reliability of target products after spammer 0.6060 1 0.9952Deviation value in product reliability 0.006 0 0.0013
Table 5: Results table for the scenario 2- Selective product promoting
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.8704 1 0.9961The average trustworthiness rating of spam attacker 0.5865 1 0.9962The average honesty score of non spam reviews 0.8726 0.8985 -The average honesty score of spam reviews 0 -1 -The average reliability of target products before spammer 0.2 -1 0.9935The average reliability of target products after spammer 0.2016 -1 0.9952Deviation value in product reliability 0.0016 0 0.00213 igure 8: Scenario 3: An over time attackTable 6: Results table for the scenario3- slandering attack over time
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewers 0.8651 1 0.9814The average trustworthiness rating of spam attacker 0.5285 1 0.9890The average honesty score of non spam reviews 0.9081 0.8694 -The average honesty score of spam reviews 0.3486 -1 -The average reliability of target products before spammer 0.6 1 1The average reliability of target products after spammer 0.5736 1 0.9816Deviation value in product reliability 0.0264 0 0.018414 able 7: Results table for the scenario3- promoting attack over time
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewers 0.8678 1 0.9900The average trustworthiness rating of spam attacker 0.5570 1 0.9926The average honesty score of non spam reviews 0.9081 0.8694 -The average honesty score of spam reviews 0.3486 -1 -The average reliability of target products before spammer 0.6 1 1The average reliability of target products after spammer 0.6181 1 0.9868Deviation value in product reliability 0.0181 0 0.0132
In this scenario, an over product attack on real data is implemented. 20 spam datarecords have been artificially added to an existing spam data set to reach this goal [6].This data set is the opinions collected from the movie Lenz website and includes 16reviewers and 670 products. To emulate the deception scenario in this data, a spamattacker is added to the data that gives 0.5 (the lowest score in real data is 0.5) to somegoal products and sends the correct score (similar to honest reviews) to other products.The average score of the attacker’s target products is about 3.75, so his reviews shoulddefinitely be identified as spam. The results are shown in Table.8 As it is presented,only the proposed model can find spam attackers, and in the case of finding the spamreview, both models have acceptable results. However, as can be seen in the last row ofTable 8, in the proposed model, FYZ spam attacker has not been able to change greatlythe reliability score of the target product from the actual quality.
In this scenario, all the condition is the same as the previous scenario; however, toemulate the deception scenario in this data, a spam attacker is added to the data thatgives 5 (the highest score in real data is 5) to some goal products and sends the correct15 able 8: Results table for the scenario 4
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.9103 0.9307 1The average trustworthiness rating of spam attacker 0.5596 0.9682 1The average honesty score of non spam reviews 0.9140 0.3823 -The average honesty score of spam reviews 0.1167 -0.1245 -The average reliability of target products before spammer 0.9106 0.8230 1The average reliability of target products after spammer 0.8604 0.2506 1Deviation value in product reliability 0.0502 0.5724 0score (similar to honest reviews) to other products. The average score of the attacker’starget products is about 1.5, so his reviews should definitely be identified as spam. Asbefore only the proposed model performs well Table.9.
7. Conclusion
Given the growing importance of users’ comments in the virtual world, providinga robust method for detecting smart spam attackers is indispensable. In this paper, amodel that would be robust against various types of spam deception behaviors has beenproposed. The efficiency of the proposed method is studied in some attack scenariosand compared with two famous and well known models in this area. The results showthat not only the proposed model can find spam attacker in every deception scenario,but also has a considerable improvement over the other model. It is capable of detectingthe spammer and decreasing its trust. Moreover, it does not allow the attacker to deviatethe product reputation to his malicious goals. As future work, it could be useful toimplement more deceptive scenarios on review spam detection models and resist thecurrently proposed model against other deceptive scenarios.16 able 9: Results table for the scenario 5
Item ROSD WNG FYZThe average trustworthiness rating of honest reviewer 0.9119 0.9307 1The average trustworthiness rating of spam attacker 0.5015 0.9886 1The average honesty score of non spam reviews 0.9154 0.3844 -The average honesty score of spam reviews 0 -0.024 -The average reliability of target products before spammer 0.2053 -0.6484 1The average reliability of target products after spammer 0.2053 0.3760 1Deviation value in product reliability 0 1.022 0
References [1] Akoglu, L., Chandy, R., and Faloutsos, C. (2013). Opinion fraud detection inonline reviews by network effects.
ICWSM , 13:2–11.[2] Algur, S. P., Patil, A. P., Hiremath, P., and Shivashankar, S. (2015). Conceptuallevel similarity measure based review spam detection. In
Signal and Image Pro-cessing (ICSIP), 2010 International Conference on , pages 416–423. IEEE.[3] Banerjee, S. and Chua, A. Y. (2014). Applauses in hotel reviews: Genuine ordeceptive? In
Science and Information Conference (SAI), 2014 , pages 938–942.IEEE.[4] Fayazbakhsh, S. K. and Sinha, J. (2012). Review spam detection: a network-basedapproach.
Final Project Report: CSE , 590.[5] Fei, G., Mukherjee, A., Liu, B., Hsu, M., Castellanos, M., and Ghosh, R. (2013).Exploiting burstiness in reviews for review spammer detection.
ICWSM , 13:175–184.[6] Harper, F. M. and Konstan, J. A. (2016). The movielens datasets: History andcontext.
Acm transactions on interactive intelligent systems (tiis) , 5(4):19.177] Harris, C. (2012). Detecting deceptive opinion spam using human computation. In
Workshops at AAAI on Artificial Intelligence .[8] Heydari, A., ali Tavakoli, M., Salim, N., and Heydari, Z. (2015). Detection ofreview spam: A survey.
Expert Systems with Applications , 42(7):3634–3642.[9] Heydari, A., Tavakoli, M., and Salim, N. (2016). Detection of fake opinions usingtime series.
Expert Systems with Applications , 58:83–92.[10] Jindal, N. and Liu, B. (2007). Analyzing and detecting review spam. In icdm ,pages 547–552. IEEE.[11] Jindal, N. and Liu, B. (2008). Opinion spam and analysis. In
Proceedings of the2008 International Conference on Web Search and Data Mining , pages 219–230.ACM.[12] Jindal, N., Liu, B., and Lim, E.-P. (2010). Finding unusual review patterns usingunexpected rules. In
Proceedings of the 19th ACM international conference onInformation and knowledge management , pages 1549–1552. ACM.[13] Khalili, A., Abdollahi Azgomi, M., and Jalaly Bidgoly, A. (2013). Simgine: Asimulation engine for stochastic discrete-event systems based on sdes description.
Simulation , 89(4):539–555.[14] Khalili, A., Bidgoly, A. J., and Azgomi, M. A. (2009). Pdetool: A multi-formalism modeling tool for discrete-event systems based on sdes description. In
International Conference on Applications and Theory of Petri Nets , pages 343–352.Springer.[15] Li, F., Huang, M., Yang, Y., and Zhu, X. (2011). Learning to identify reviewspam. In
IJCAI Proceedings-International Joint Conference on Artificial Intelli-gence , volume 22, page 2488.[16] Lim, E.-P., Nguyen, V.-A., Jindal, N., Liu, B., and Lauw, H. W. (2010). Detectingproduct review spammers using rating behaviors. In
Proceedings of the 19th ACM nternational conference on Information and knowledge management , pages 939–948. ACM.[17] Mukherjee, A., Liu, B., Wang, J., Glance, N., and Jindal, N. (2011). Detectinggroup review spam. In Proceedings of the 20th international conference companionon World wide web , pages 93–94. ACM.[18] Narayan, R., Rout, J. K., and Jena, S. K. (2018). Review spam detection usingopinion mining. In Sa, P. K., Sahoo, M. N., Murugappan, M., Wu, Y., and Majhi,B., editors,
Progress in Intelligent Computing Techniques: Theory, Practice, andApplications , pages 273–279. Springer Singapore.[19] Ott, M., Cardie, C., and Hancock, J. (2012). Estimating the prevalence of de-ception in online review communities. In
Proceedings of the 21st internationalconference on World Wide Web , pages 201–210. ACM.[20] Pak, A. and Paroubek, P. (2010). Twitter as a corpus for sentiment analysis andopinion mining. In
LREc , volume 10.[21] Radovanovi, D. and Krstaji, B. (2018). Review spam detection using machinelearning.[22] Wang, G., Xie, S., Liu, B., and Philip, S. Y. (2011). Review graph based onlinestore review spammer detection. In
Data mining (icdm), 2011 ieee 11th interna-tional conference on , pages 1242–1247. IEEE.[23] Wang, G., Xie, S., Liu, B., and Yu, P. S. (2012). Identify online store reviewspammers via social review graph.
ACM Transactions on Intelligent Systems andTechnology (TIST) , 3(4):61.[24] Wang, Z., Hou, T., Song, D., Li, Z., and Kong, T. (2016). Detecting reviewspammer groups via bipartite graph projection.
The Computer Journal , 59(6):861–874.[25] Xie, S., Wang, G., Lin, S., and Yu, P. S. (2012a). Review spam detection viatemporal pattern discovery. In
Proceedings of the 18th ACM SIGKDD internationalconference on Knowledge discovery and data mining , pages 823–831. ACM.1926] Xie, S., Wang, G., Lin, S., and Yu, P. S. (2012b). Review spam detection viatime series pattern discovery. In