Comparing Human Computation, Machine, and Hybrid Methods for Detecting Hotel Review Spam

Abstract

Most adults in industrialized countries now routinely check online reviews before selecting a product or service such as lodging. This reliance on online reviews can entice some hotel managers to pay for fraudulent reviews – either to boost their own property or to disparage their competitors. The detection of fraudulent reviews has been addressed by humans and by machine learning approaches yet remains a challenge. We conduct an empirical study in which we create fake reviews, merge them with verified reviews and then employ four methods (Naive Bayes, SVMs, human computation and hybrid human-machine approaches) to discriminate the genuine reviews from the false ones. We find that overall a hybrid human-machine method works better than either human or machine-based methods for detecting fraud – provided the most salient features are chosen. Our process has implications for fraud detection across numerous domains, such as financial statements, insurance claims, and reporting clinical trials.