Characterizing Key Stakeholders in an Online Black-Hat Marketplace
Shehroze Farooqi, Muhammad Ikram, Emiliano De Cristofaro, Arik Friedman, Guillaume Jourjon, Mohamed Ali Kaafar, M. Zubair Shafiq, Fareed Zaffar
CCharacterizing Key Stakeholders in anOnline Black-Hat Marketplace
Shehroze Farooqi
University of IowaIowa City, IA, [email protected]
Guillaume Jourjon
Data61 CSIROSydney, NSW, [email protected]
Muhammad Ikram
Data61 CSIRO, UNSWSydney, NSW, [email protected]
Mohamed Ali Kaafar
Data61 CSIROSydney, NSW, [email protected]
Emiliano De Cristofaro
University College LondonLondon, [email protected]
Zubair Shafiq
University of IowaIowa City, IA, USAzubair-shafi[email protected]
Arik Friedman
AtlassianSydney, NSW, [email protected]
Fareed Zaffar
LUMSLahore, [email protected]
Abstract —Over the past few years, many black-hat mar-ketplaces have emerged that facilitate access to reputationmanipulation services such as fake Facebook likes, fraudulentsearch engine optimization (SEO), or bogus Amazon reviews. Inorder to deploy effective technical and legal countermeasures,it is important to understand how these black-hat marketplacesoperate, shedding light on the services they offer, who is selling,who is buying, what are they buying, who is more successful, whyare they successful, etc. Toward this goal, in this paper, we presenta detailed micro-economic analysis of a popular online black-hatmarketplace, namely, SEOClerks.com. As the site provides non-anonymized transaction information, we set to analyze sellingand buying behavior of individual users, propose a strategy toidentify key users , and study their tactics as compared to other( non-key ) users. We find that key users : (1) are mostly locatedin Asian countries, (2) are focused more on selling black-hatSEO services, (3) tend to list more lower priced services, and(4) sometimes buy services from other sellers and then sell athigher prices. Finally, we discuss the implications of our analysiswith respect to devising effective economic and legal interventionstrategies against marketplace operators and key users . I. I
NTRODUCTION
Reputation plays a very important role in online servicesincluding e-commerce sites, search engines, or online socialnetworks. For instance, Amazon uses customer reviews to helpusers assess the credibility of sellers, Google relies on PageR-ank to determine search ranking of websites, while Facebooklikes often offer a measure of the popularity of brands. As aresult, it is not surprising that an increasing number of black-hat marketplaces facilitate access to reputation manipulationservices. A multitude of online and underground (i.e., hostedas Tor hidden services) black-hat marketplaces sell servicesto generate bogus reviews, obtain fake likes, artificially boostPageRank, etc. Several companies such as Amazon and Face-book have filed lawsuits against users who provide reputationmanipulation services [8], [18]. For instance, Amazon recentlyconducted a sting operation on Fiverr and sued more thana thousand “John Doe” fraudsters for selling bogus reviews[5]. Law enforcement agencies have also cracked down ondifferent underground black-hat marketplaces [1], [2], [4]. However, the cleanup or closure of a black-hat marketplacetypically leads to increased popularity of other services [19].In a way, the overall black-hat marketplace ecosystem isgenerally robust to such measures, highlighting the multi-faceted and complex nature of the problem. Therefore, thedesign and implementation of effective technical and legalcountermeasures requires a thorough examination and deepunderstanding of how these black-hat marketplaces operate.Prior work has studied their evolution and the types of fraud-ulent and illicit services they offer [6], [10]–[16], [19]–[21],[23]–[25]. However, very little work has focused on individualsellers, buyers, and services: arguably, such an analysis is quitechallenging, as most online and underground marketplaces donot reveal detailed buyer-seller transaction information. Forinstance, many black-hat marketplaces only provide aggregatepositive and negative ratings which makes it impossible totrack specific transactions among users on the marketplace.Aiming to address this gap, this paper presents a first-of-its-kind, detailed micro-economic analysis of a popular onlineblack-hat marketplace: SEOClerks.com. We select SEOClerksas it provides detailed ratings, allowing us to analyze indi-vidual transaction-level information. Moreover, SEOClerks ismore popular than most of the other online black-hat market-places studied in prior work (e.g., [24], [25]). At the time ofwriting, SEOClerks is ranked in the top 12K websites globallyby Alexa; whereas, for example, Sandaha.com is ranked 213K,Zhubajei.com 353K, and Shuakewang.com 1,128K.Our goal is to identify key stakeholders on online black-hat marketplaces and understand their role in order to developeffective countermeasures. First, we identify key users who areamong the early joiners, are very active, and make the mostmoney on the marketplace. Next, we characterize how keyusers differ as compared to other ( non-key ) users. We compareand contrast key and non-key users in terms of the servicesthey offer, and their selling and buying behavior.We start our analysis with a general characterization ofSEOClerks, finding that it has over 262K users and 39Klisted services. Using individual buyer ratings as a proxy for a r X i v : . [ c s . C Y ] A p r ales, our lower-bound estimate of the marketplace revenueis $1.3 million. Moreover, we estimate that SEOClerks op-erators have earned hundreds of thousands of dollars fromfees/commissions and advertising.Next, we look for key users on the marketplace, identifying99 of them. These are among the early joiners (accounts wereregistered around the launch of the marketplace), are veryactive (they have logged on to the site within a week of ourcrawl), and/or make the most money on the marketplace. Theseusers, although accounting for less than 0.04% of all users andoffering only 9% of all services, actually generate 56% of themarketplace revenue. We also find that a majority of key usersare located in Asian countries (India, Indonesia), while buyersare relatively concentrated in European and North Americancountries (USA, UK, Italy).The vast majority of services on SEOClerks are fraudulent,e.g., selling inbound links from other web pages (“backlinks”)to improve Google PageRank, inflating website traffic for clickfraud, fake Instagram followers, Twitter retweets, or Facebooklikes. Black-hat SEO services offered by key users actuallyaccount for a majority of their revenue. Key users are typicallyallowed to offer lower priced services (starting at $1) and theirservices tend to receive more views than the services offeredby other users.Also, some key users purchase services from other sellerson SEOClerks and sell it at higher prices. For example, akey user offers a service for bogus SoundCloud plays and hasalso repeatedly purchased a similar service from another seller.Finally, we show that SEOClerks operators use an escrowmechanism to get transaction/commission fees and to resolvedisputes between sellers and buyers; thus, their marketplaceaccounts on PayPal, Payza, and BitPay can be targeted foreconomic and legal intervention.Overall, black-hat marketplaces constitute a key link in theInternet fraud chain [12]. Through their characterization, ourwork aims to help in devising effective economic and legalintervention strategies. Since key users constitute a majorityof the marketplace revenue, targeting them can considerablylimit fraudulent activities out of black-hat marketplaces.II. D ATA
Data Collection.
We conducted a complete crawl ofSEOClerks.com in February 2015, using the Scrapy webcrawler . SEOClerks has a directory of user profiles thatcontains username, account creation date, last login date,location, user reputation level, average response time, rat-ings, description of skills, and the list of services offered.SEOClerks also has a directory of services that containsservice price, service creation date, a description of the service,seller’s username, expected delivery time, number of ordersin progress, number of views, and positive/negative buyerratings. We collected all publicly available information fromboth user and service directories. We also crawled individualbuyer ratings on service pages to identify their buyers. TABLE I:
Statistics of SEOClerks marketplace.
General Statistics.
Table I summarizes overall statistics ofthe SEOClerks marketplace. SEOClerks is ranked by Alexain the top 12K websites globally and top 3K in India. Ourcrawled data includes 262,909 users and 39,520 services. 22%of the services on SEOClerks are sold at least once. Theaverage revenue per sold service is $152. The estimated totalrevenue of SEOClerks is $1,349,316, which is obtained bymultiplying the price of each service with the correspondingrating count. Since buyers are not required but are highly-recommended to rate the purchased services, our estimaterepresents a lower-bound on the actual total revenue. We alsonote that several services include some add-ons (or “serviceextras”) for additional payment. From our crawls, we cannotidentify the purchase of these add-ons. Thus, our lower boundon the estimated revenue does not include service extras.
Ethical Considerations.
As we collected and analyzed datapertaining to possibly fraudulent activities, we requested ap-proval from our Institutional Review Board, which classifiedour research as exempt . We note that: (1) we did not engagein any fraudulent transactions at the marketplace, and (2) weonly collected publicly available information.III. I
DENTIFYING K EY S TAKEHOLDERS
Our work aims to identify and analyze key stakeholderswho are crucial for the success of a black-hat marketplace. Wehypothesize that key users of an online black-hat marketplace(1) join the marketplace soon after it was launched; (2) areamong the most successful sellers on the marketplace; and (3)are very active on the marketplace. Below, we further discussand use these three criteria to identify key users on SEOClerks.
Early Joiners.
We first analyze the registration of users overtime on SEOClerks using the account creation date reportedfor each user. Figure 1(a) plots the daily registration rate ofnew users and the cumulative number of users on SEOClerks.We note that the first user account was registered in mid-2011. Our assessment is confirmed by the Internet ArchiveWayback Machine , which has the first snapshot of SEOClerksdating back to October 7, 2011. Note that the number ofusers initially grew fairly slowly (daily new users < . The vertical black line in Figure 1(a) marks thechange point in early 2013 after which we observe a sharp http://web.archive.org ime1/12 7/12 1/13 7/13 1/14 7/14 1/15 N u m be r o f U s e r s Daily New UsersTotal Users (a) Registration of Users over time
Revenue ($)10 C u m u l a t i v e S e ll e r C oun t (b) Distribution of Sellers’ Revenue Days10 C u m u l a t i v e S e ll e r C oun t (c) Distribution of Last Login Date of Sellers. Seller Join Date2012 2013 2014 2015 S e ll e r La s t Log i n D a t e (d) Relationship between seller join date, last login date, and revenue. Circlesize represents seller revenue. Red circles represent key users while bluecircles represent non-key users. Fig. 1:
Identification of key users on SEOClerks increase in new user registrations. The users who joined themarketplace before this cutoff date are labeled as early joiners .Using this criterion, we identify a total of 391 early joiners.
Top Sellers.
We define a user as a seller if the user hasposted at least one service on SEOClerks. In total, we identify8,861 sellers on SEOClerks. Figure 1(b) plots the revenuedistribution for sellers on SEOClerks. Out of 8,861 sellers,only 2,228 sellers sold at least one service. The long-taildistribution indicates that a small number of sellers account formost of the marketplace revenue. We label the top 10% sellers(marked by the vertical black line) among the 2,228 sellersas top sellers . These 222 top sellers account for $1,181,339(88%) revenue on SEOClerks.
Active Sellers.
We identify active sellers on SEOClerks byanalyzing their last login date. Whenever a user logs in toSEOClerks, the last login date is updated on the user’s profile.Figure 1(c) plots the distribution of sellers’ last login date (atthe time of our crawl) on SEOClerks. We observe that more than half of the sellers on SEOClerks are not active. We notethat 2,826 (32%) sellers logged in to the marketplace within aweek of our crawl (marked by the vertical black line). Sinceactive sellers need to log in frequently in order to respond tocustomers and receive new orders, we label these 2,826 sellerswho logged in within a week of our crawl as active sellers . Identifying Key Users.
Figure 1(d) visualizes marketplacesellers using a scatter plot for join date and last login date,where the radius of each circle is proportional to the sellerrevenue. We mark the users who satisfy the aforementionedthree criteria with red circles. The remaining users are markedwith blue circles. It is surprising to note that a vast majority ofusers who joined the marketplace before 2013 logged in veryrecently. We also observe that a majority of these users arealso top sellers on SEOClerks. We label a total of 99 sellerswho satisfy the aforementioned three criteria as key users .We next analyze the characteristics of these key users withthe aim of facilitating the design of technical countermeasures3nd strategies for economic or legal intervention.IV. M
ARKETPLACE A NALYSIS
This section presents an in-depth analysis of SEOClerkswith an emphasis on comparing and contrasting key users andnon-key users. We analyze a wide range of characteristics forservices, sellers, and buyers on the marketplace.
A. Services
A vast majority of services on SEOClerks are gearedtowards fraudulent services such as selling backlinks for black-hat SEO, website traffic, Instagram followers, Twitter retweets,Facebook likes, URL spam, etc. We identified a total of 39,520services offered on SEOClerks. A total of 3,645 (9%) serviceswere posted by key users, while the remaining 35,875 (91%)services were posted by non-key users. Below we characterizedifferent aspects of the services offered by key users and non-key users.
Pricing.
The services on SEOClerks are priced anywhere inthe range of $1-$999. Figure 2(a) plots the distributions ofservice prices for key users and non-key users. We observethat a vast majority of services are priced in the lower range.For instance, 3,197 (88%) services offered by key users and31,719 (80%) services offered by non-key users are priced upto $20. Note that $999 is the maximum service price allowedby SEOClerks, while $5 is the minimum allowed service pricefor the newly registered sellers. The mode of service pricedistribution for key users is $1 and that for non-key users is$5, which accounts for 416 (11%) services for key users and11,227 (31%) services for non-key users. As we discuss later,only experienced sellers on SEOClerks are allowed to postservices that are priced below the $5 limit. Since key usersare much more experienced than non-key users, more than aquarter of the services offered by key users are under $5, whileonly 11% of the services offered by non-key users are under$5.
Sales.
We recorded a total of 233,638 sales resulting in theestimated revenue of $1,349,316 on SEOClerks. Key usersaccount for more than half of the total sales and revenue. Morespecifically, key users made 121,923 sales accounting for anestimated revenue of $758,959 (56%), while non-key usersmade 111,715 sales accounting for an estimated revenue of$590,357 (44%). Figures 2(b) and 2(c) show the distributionsof service volume and revenue for key and non-key users. It isnoteworthy that a vast majority of services by key users (1,874= 51%) and non-key users (26,547 = 74%) have no sales andthus zero revenue.We observe a skewed distribution of sales volume andrevenue. For key users, 9% of the services had just one sale,5% of the services had two sales, and 3% of services hadthree sales. For non-key users, 8% of the services had just onesale, 3% of the services had two sales, and 2% of serviceshad three sales. On the other hand, a few popular servicesaccount for a large fraction of sales. For key users, the mostpopular service in terms of sales volume is “add 2000 to 2500 Youtube views or 600+ INSTAGRAM Followers or 1000Likes” (priced at $2) and has 3,853 sales resulting in $7,706revenue. For non-key users, the most popular service in termsof sales volume is “400 Facebook Fanpage likes OR 1300Twitter Marketing OR 1500 INSTAGRAM Marketing” (pricedat $3) and has 2,968 sales resulting in $8,904 revenue. Whilewe note that low priced services tend to have high sale volume,higher priced services still tend to generate more revenue. Forkey users, the top service in terms of revenue is “Backlinksto improve Google search ranking” (priced at $29) attracting1,364 sales yielding $39,556 in revenue. For non-key users,the top service in terms of revenue is “Google X Factor LinkCircle For Higher Ranking And Quality Links” (priced at $57)attracting 550 sales yielding $31,350 in revenue.
View Count.
To further examine why key users account formore sales and revenue, we analyze the correlation betweenservice view count and sales volume. Figure 2(d) plots thedistribution of view count for services offered by key usersand non-key users. We note that the services offered by keyusers are generally viewed more than those by non-key users.For example, the average number of views for key users is9,218 while the average number of views for non-key usersis 1,962. Moreover, 1.3% of the services offered by key usersare viewed more than 100 thousand times while only 0.1%of the services offered by non-key users are viewed over100 thousand times. Our eyeball analysis revealed that mostservices featured on the homepage are posted by key users.We surmise that the services offered by key users tend tohave higher view counts because they are more frequentlyfeatured on the marketplace. To test whether higher viewcounts translate into more sales, we analyze the correlationbetween service view count and sale volume. Figures 3(a) and3(b) visualize the correlation between service view count andsale volume for key users and non-key users, respectively. Wenote that services with more views tend to have higher salesvolume for both key and non-key users. Thus, due to theirhigher view count, it is expected that the services offered bykey users tend to garner more sales than those by non-keyusers.
Service Categorization.
To systematically analyze differenttypes of fraudulent services on SEOClerks, we use keywordanalysis and manual curation to group top selling servicesinto various categories based on their target, e.g., Twitterfollowers, Instagram followers, search engine manipulation,etc. Tables II and III list the top categories of servicesand the top selling service for each category for key usersand non-key users, respectively. We note that a majority ofservices target black-hat search engine optimization and socialnetwork reputation manipulation for both key users and non-key users. Specifically, more than 40% of services offeredby key users target black-hat SEO; whereas, 23% of servicesoffered by non-key users target black-hat SEO. Black-hat SEOservices account for more than half of the revenue of servicessold by key users and 31% of the total marketplace revenue.In contrast, more than 50% of the services offered by non-4 rice ($)10 CD F o f S e r v i c e s Key UsersNon-Key Users (a) Price
Volume10 CD F o f S e r v i c e s Key UsersNon-Key Users (b) Volume
Revenue ($)10 CD F o f S e r v i c e s Key UsersNon-Key Users (c) Revenue
Number of Views10 CD F o f S e r v i c e s Key UsersNon-key Users (d) Views
Fig. 2:
Distribution of service price, volume, revenue, and views on SEOClerks.
Views10 V o l u m e (a) Key users Views10 V o l u m e (b) Non-key users Fig. 3:
Scatter plot of service view count and sales volume. key users target popular social media platforms while about28% percent of services of key users are targeted towardssocial media platforms. The largest service category among social media platforms for non-key users is Twitter. The mostpopular service in Twitter category provides “1 million Twitterfollowers” for $849 and has garnered $11,037 in total revenue.5 ategory % of Services Revenue Top ServiceDescription Revenue Price
Black-hat SEO 40.6% $417,865 (55%) Backlinks to improve search ranking $39,556 $29Instagram 13.9% $78,274 (10%) 1,000 Instagram followers $7,706 $10YouTube 8.0% $73,409 (10%) 100,000 safe YouTube views $3,5160 $120Twitter 16.1% $52,583 (7%) 50,0000 followers or 2,000 re-tweets $4,320 $20Website traffic 9.5% $49,599 (6%) Promote on a large Facebook group $8,640 $10
TABLE II:
Service categories and the most popular service in each category for key users.
Category % of Services Revenue Top ServiceDescription Revenue Price
Black-hat SEO 23.0% $173,081 (29%) Rank your website on first page $31,350 $57Twitter 22.2% $82,147 (14%) 1 million Twitter followers $11,037 $849Instagram 12.6% $47,591 (8%) 1,000 Instagram followers $4,418 $2YouTube 10.9% $24,417 (4%) 8,000 safe YouTube views $2,052 $12Website traffic 7.0% $19,226 (3%) Views UP - Web Traffic Bot $2,360 $40
TABLE III:
Service categories and the most popular service in each category for non-key users.
In contrast, the largest service category among social mediaplatforms for key users is Instagram. The most popular servicein Instagram category provides “1,000 Instagram followers”for $10 and has garnered $7,706 in total revenue.
B. Users
General Stats.
We find a list of 262,909 users on SEOClerks.We label a user as a seller if the user has listed at leastone service. Similarly, we label a user as a buyer if the userhas purchased at least one service. Note that a user may becategorized both as seller and buyer.We identified 8,861 sellersand 33,092 buyers on SEOClerks.
Reputation.
SEOClerks uses a tiered reputation system tocategorize users. The system assigns users one of the available8 reputation levels. New users start from level 1. A user’slevel is upgraded automatically based on fulfillment of certainrequirements for the first five levels (1,2,3,4,5), while level Xusers (X3,X4,X5) are considered elite and they are selectedmanually by staff members of SEOClerks. The details ofrequirements and benefits for level promotion are describedin [3]. A higher reputation level provides more benefits andless restrictions. For example, users at higher reputation levelscan price services below the $5 limit and get faster paymentclearance.Table IV lists user reputation level statistics for key usersand non-key users on SEOClerks. We note that key usersare generally more experienced than non-key users. Most keyusers are at reputation level 3 (62%) while most non-key usersare at reputation level 1. We surmise that key users receivepreferential treatment from the marketplace staff. For example,we note that 28 out of 99 key users are at reputation level X.In contrast, only 14 non-key users are at reputation level Xeven though they contain more than a hundred sellers in thetop 10 percentile.Recall that users can be sellers and/or buyers: in thefollowing, we analyze them separately.
C. Seller Analysis
We identify 8,861 sellers on SEOClerks, out of which 99are labeled key users and the remaining 8,762 are labeled asnon-key users. Note that some non-key users have not soldany service yet—these “zero-sale” sellers are included in ourstatistics.
Geographic Characteristics.
SEOClerks provides the geo-graphic location of users based on IP geolocation and/or man-ual input from users. Table V lists the geographic distributionof sellers across top-five countries. We note that a substantialfraction of sellers are from a few Asian countries including In-dia, Bangladesh, Pakistan, Indonesia, and Philippines. This issomewhat expected because of their relatively lower per-capitaincome [22]. We also note that key users are concentratedmore in Asian countries as compared to non-key users. Somesellers may be using USA-based VPNs/proxies to manipulatetheir geolocation for credibility [17].
Number of Services.
Figure 4(a) plots the distribution ofthe number of services listed by key and non-key users onSEOClerks. Key users list 3,645 services while the remaining35,875 services are offered by non-key users. Note that morethan 50% of non-key users listed only one service and morethan 90% percent posted less than 10 services. Key users tendto post more services (per seller) as compared to non-keyusers. Only 5% key users posted one service and 64% postedmore than 10 services. The seller with most listed servicesamong key users had 1,092 services. In contrast, the seller withmost listed services among non-key users had 458 services.
Revenue.
Figure 4(b) plots the distribution of seller revenuefor key and non-key users. Overall, key users account for$758,959 (56%) revenue, while non-key users account for$590,357 (44%) revenue. It is noteworthy that more than 75%of non-key users are zero-sale sellers. The long-tail distributionindicates that a few sellers account for most revenue for non-key users. Recall that we labeled top 10% (228) sellers interms of revenue as top sellers. Out of the 228 top sellers,99 sellers were identified as key users. The minimum and6 eputation Key Users Non-Key UsersLevel
Number of Users % of Users Revenue Number of Users % of Users RevenueX5 1 1% $15,560 1 ≈
0% $599X4 2 2% $36,680 1 ≈
0% $259X3 25 25% $446,643 12 ≈
0% $54,7335 0 0% $0 1 ≈
0% $994 1 1% $2,351 2 ≈
0% $20,6943 61 62% $217,255 960 11% $383,6262 0 0% $0 334 4% $12,8171 9 9% $40,470 7,451 85% $117,530
TABLE IV:
User reputation statistics on SEOClerks.
Number of Services10 CD F o f S e ll e r s Key UsersNon-key Users (a) Services
Revenue ($)10 CD F o f S e ll e r s Key UsersNon-key Users (b) Revenue
Fig. 4:
Distributions of seller services and revenue on SEOClerks.
Country Non-Key Users Key Users
Total 8,762 99
India 18% 29%USA 15% 15%Bangladesh 10% 18%Pakistan 7% 12%Indonesia 5% 2%
TABLE V:
Geographic location of sellers.
Country Non-Key Users Key Users
Total 33,013 79
USA 31% 13%Italy 6% -UK 6% 1%India 6% 28%Indonesia 4% 2%
TABLE VI:
Geographic location of buyers. maximum revenue earned by a key user is $721 and $94,190,respectively. The remaining 129 out of 228 top sellers accountfor 72% revenue of non-key users.
D. Buyer Analysis
We identify 33,092 buyers on SEOClerks, out of which 79buyers are labeled key users and the remaining 33,013 arelabeled non-key users.
Geographic Characteristics.
Table VI lists the geographicdistribution of buyers across top-five countries. Overall, buyers are relatively concentrated in the North American and Euro-pean countries such as USA, Italy, UK, and Canada. However,we note that a large number of buyers labeled as key usersare located in India. Recall that all buyers who are key usersare also top sellers on the marketplace. These key users alsopurchase services of other sellers. Regardless of the role ofthe marketplace users, our findings somewhat mirror the site’saudience statistics as estimated by Alexa. Alexa estimates that13.8% of the site’s visitors are from USA, followed by 13.5%from India, and 4.7% from Italy.
Purchase Statistics.
Figure 5(a) plots the distributions of thepurchase volume by key and non-key users. We note that amajority of key users (88%) are buyers and they purchasedservices more than non-key users. For key users, the medianpurchase volume is 5 and the average is 24. For non-keyusers, the median purchase volume is 2 and the average is5. Figure 5(b) plots the distributions of buyer expense (thetotal amount of money spent by a buyer) by key and non-key users. We note that key users also spend more money topurchase services as compared to non-key users. For key users,the median buyer expense is $50 and average is $141. Fornon-key users, the median buyer expense is $10 and averageis $41.
Reselling Behavior.
We next analyze users with dual rolesof a buyer and seller (i.e., they sold at least one service andalso purchased at lease one service). Figure 6 visualizes the7 urchase Volume10 CD F o f B u y e r s Key UsersNon-key Users (a) Volume
Buyer Expense ($)10 CD F o f B u y e r s Key UsersNon-key Users (b) Expense
Fig. 5:
Distributions of buyer purchase volume and expense on SEOClerks.
Number of services sold0 1000 2000 3000 4000 5000 6000 7000 8000 N u m be r o f s e r v i c e s pu r c ha s ed (a) Key Users Number of services sold0 2000 4000 6000 8000 10000 N u m be r o f s e r v i c e s pu r c ha s ed (b) Non-key Users Fig. 6:
Each point in the scatterplots represent the number of services sold and purchased by a user on SEOClerks. There are many sellerswho are also frequently buying a large number of services scatter plot of the services sold and purchased by all dual rolekey and non-key users on the marketplace. 79 key users and1,101 non-key users have a dual role of buyers and sellers. Forexample, a key user purchased 432 services and also sold 450services while another non-key user purchased 240 servicesand sold 530 services. To understand the behavior of theseusers, we manually analyze the services purchased and soldby them. We find that a majority of the dual users are buyingand then selling the same kind of services. This behavior issometimes due to users purchasing services from other sellersfor less price and reselling them at higher prices. For example,a key user offers a service providing 1,000 Instagram followersfor $4, and the same user has repeatedly purchased similarservices from multiple users for $2. As another example, akey user offers a service providing 1,000 SoundCloud plays for$1, and the same user has repeatedly purchased a service from another user providing 15,000 SoundCloud plays for $1. Wesurmise that a user may also sometimes purchase services fromother sellers to fulfill existing orders (e.g., due to receiving anunusually large number of orders or temporary infrastructureoutages).
Buyer-Service Correlation.
We next analyze the relationshipbetween buyers and services. Figure 7 visualizes the scatterplot between buyers and services. Note that each data point inthe scatter plot represents a buyer-service pair, with servicesand buyers sorted in the descending order with respect to theirpurchase frequency. Darker circles represent fewer purchasesand lighter circles represent many repeated purchases. Forclarity, we also set the size of circles proportional to purchasefrequency.Figures 7(a) and 7(b) visualize the purchases made by keyusers from key and non-key users, respectively. It is interesting8 ervice Index (descending sort)0 50 100 150 B u y e r I nde x ( de sc end i ng s o r t ) (a) Services of key users purchased by other key users Service Index (descending sort)0 50 100 150 B u y e r I nde x ( de sc end i ng s o r t ) (b) Services of non-key users purchased by key users Service Index (descending sort)0 500 1000 1500 B u y e r I nde x ( de sc end i ng s o r t ) (c) Services of key users purchased by non-key users Service Index (descending sort)0 1000 2000 3000 4000 5000 6000 B u y e r I nde x ( de sc end i ng s o r t ) × (d) Services of non-key users purchased by non-key users Fig. 7:
Scatter plot illustrating the relationship between services and buyers. to note that many key users have purchased services fromother key and non-key users. Furthermore, a few servicestend to have many repeat purchases from several key users(lighter circles are concentrated on the bottom-left of thescatter plot). To further investigate this finding, we identifythe key users who are purchasing a large number of services.We find that 7 key users are among the top 10% buyerson the marketplace. Our manual inspection of these 7 keyusers revealed that these key users are purchasing servicesthat are similar to the services offered by them. For example,a key user offers a service providing Instagram followers,and the same user has repeatedly purchased services offeringInstagram followers from other sellers. These are dual roleusers on the marketplace.Figures 7(c) and 7(d) visualize the purchases made by non-key users from key users and non-key users, respectively. Weobserve that a vast majority of non-key users buy a serviceonly once. However, a few popular services by key users tendto have many repeat buyers (lighter circles are concentratedon the bottom-left of the scatter plot). V. D
ISCUSSION
We start the discussion by presenting further interestingobservations.
A. Additional Findings
Marketplace Commission.
SEOClerks charges 20% commis-sion for each order. It also charges a nominal transaction pro-cessing fee (varying depending on the mode of payment). The20% commission is charged from sellers and the transactionprocessing fee is charged from buyers, thus, according to ourestimates, the operators of SEOClerks have earned at least$269,863 in commissions. Note that SEOClerks operators alsooffer a variety of services to temporarily “feature” services onthe marketplace homepage. Based on the number of transac-tions of these services, we estimate that SEOClerks operatorshave earned thousands of dollars.
Revenue Underestimation.
Recall that buyers are not man-dated to provide feedback ratings on SEOClerks. Moreover,SEOClerks allows sellers to list “service extras” which cost inaddition to the base service price. From our crawled data, we9annot tell whether a buyer bought service extras, thereforeour revenue estimate represents the lower bound on the actualmarketplace revenue.
Data Trust.
Given the black-hat nature of SEOClerks, it ispossible that some information on the websites (e.g., userlevels) may be manipulated by the marketplace operators.While we cannot completely rule this out, we created freshaccounts on both marketplaces and positively verified theirinformation (e.g., geographic location, join date, user level,etc.) to lend some confidence to our collected data.
B. Countermeasures
We now discuss potential countermeasures to curb theactivities of black-hat marketplaces, including those targetingkey users on SEOClerks as well as the operators.
Targeting Key Users.
While key users constitute less than0.04% of SEOClerks users, they account for more than halfof the revenue. Specifically targeting these key users canconsiderably limit fraudulent activities out of black-hat mar-ketplaces. Furthermore, active experiments could be conductedto understand the working of their infrastructure, e.g., creatinghoneypot accounts to identify the fake accounts used forproviding likes/followers [7], [21].
Targeting Marketplace Operators.
Another approach is togo after the monetary systems used by black-hat marketplaces.More specifically, SEOClerks uses an escrow mechanism toget transaction/commission fees and to resolve disputes be-tween sellers and buyers. Buyers on SEOClerks can purchaseservices using standard credit/debit card, PayPal, Payza, orusing cryptocurrencies. For PayPal and Payza, the marketplaceaccount of SEOClerks is registered to Ionicware Inc. Forall cryptocurrency transactions, SEOClerks uses an accounton BitPay which is also registered to Ionicware Inc. Thesemarketplace accounts on PayPal, Payza, and BitPay can betargeted for economic and legal interventions. Another possi-ble countermeasure would be to seek court injunctions andshutdown these websites [8], [18] by targeting either thedomain registrar or the hosting company. However, this actionmight not be as effective due to possibly lengthy procedures,possibly allowing websites to change name and/or relocate toother hosting providers.VI. R
ELATED W ORK
Prior work has looked at black-hat marketplaces to analyzethem in terms of demographics, nature and quality of offeredservices, revenue models, and financial intervention. Whileour analysis is also based on measurements (e.g., via periodiccrawls) as in some of the related work, there are two keydifferences between this paper’s methodology and prior work:(i) the object of our measurement campaign, and (ii) ourinvestigation aimed to identify key stakeholders who dominatethe black-hat marketplace. To the best of our knowledge, wepresent first-of-its-kind study to identify and understand therole of key stakeholders on black-hat marketplaces.
Crowdturfing markets.
Wang et al. [24] studied “crowdturf-ing” (astroturfing + crowdsourcing) on two large Chinese mali-cious crowdsourcing markets (Zhubajie and Sandaha), and sur-veyed several USA-based and Indian malicious crowdsourcingsites such as ShortTask, MinuteWorkers, etc. Unlike our work,they focused on buyer-driven malicious crowdsourcing mar-kets. Overall, in addition to the market size estimation, theywere able to measure real-world ramifications of these servicesby becoming active customers in one of these markets. Xu etal. [25] analyzed several black-hat marketplaces. They foundthat, compared to normal sellers, fraudulent sellers escalatetheir reputations at least 10 times. Thus, fraudulent sellersprofit by harnessing crowd-sourced human laborers to conductfake transactions for their offered services. Note that SEO-Clerks is ranked higher than these marketplaces and—unlikemost other black-hat marketplaces—provides non-anonymizedtransaction information which allows us to analyze selling andbuying behavior of users in detail. In [16] and [10], [11],the authors studied services and crowdturfing, respectively,on Freelancer and Fiverr. They developed machine learningmodels to detect crowdturfing within mostly legitimate con-tent. Our work confirms many findings from [9]–[11], [16] interms of services popularity and target. However, our analysisdiffers in that Fiverr and Freelancer offer mostly legitimateservices (more than 80%, according to the authors), whereasSEOClerks is a dedicated black-hat marketplace.
Standalone merchants.
Thomas et al. [23] analyzed traffick-ing of fake accounts in Twitter. They bought accounts from27 merchants and developed a classifier to detect them. Basedon this classifier, they successfully identified several millionfraudulent accounts, of which 95% were disabled with thehelp of Twitter. In a similar study, Stringhini et al. [20], [21]measured the market of Twitter followers, providing Twitterfollowers for sale. Based on this measurement campaign,the authors evaluated several machine learning techniques todetect sybil accounts. In our prior work [7], we presented ameasurement study of Facebook like farms, which providepaid services to boost the number of page likes. We notethat this line of research focuses on individual merchants andtheir operational aspects, whereas our work studies operationof black-hat marketplaces involving thousands of merchants.
Underground forums and markets.
Motoyama et al. [15]analyzed social dynamics in six underground forums and cat-egorized illegal merchandize traded on these forums. Christin[6] studied Silk Road, an anonymous underground marketplacefor contrabands, drugs, and pornography, providing a detailedanalysis of the items being sold and the seller population.Buyer feedback was used to estimate total revenue and volumeof the transactions. Silk Road data suggests a core clique oftop sellers, and our analysis shows a similar trend, where asmall group of sellers joined the marketplace early and alsohappen to be the most successful sellers. Soska et al. [19]conducted a longitudinal analysis of 16 underground onlinemarketplaces over a time period of two and a half years tounderstand the evolution of online anonymous marketplaces.10hese anonymous marketplaces do not expose individual buyerinformation, thus the authors were unable to perform analysisof buyers. VII. C
ONCLUSION
This paper presented a comprehensive analysis of key stake-holders in a popular online black-hat marketplace, SEOClerks.com. These key users are among the early joiners, are mostactive, and make the most money on the marketplace. Specif-ically, 99 key users (out of a total of 262K users) account formore than 56% of the total revenue. We compare and contrastkey and non-key users by analyzing the services they offer,and their selling and buying behavior. We find that a majorityof key users on SEOClerks are located in Asian countries,and that some of them purchase services from other sellersand then sell them at higher prices.Black-hat marketplaces constitute a key link in the Internetfraud chain [12]. Overall, our findings highlight opportunitiesfor economic and legal intervention to counter black-hatmarketplaces, as we demonstrate that a significant part of theactivity is concentrated in the hands of relatively few actors.More specifically, since key users constitute a majority of themarketplace revenue, targeting them specifically can consider-ably limit fraudulent activities out of black-hat marketplaces.In future, we are interested in studying the role of key users onmultiple black-hat marketplaces over time. In the long term,our goal is to develop statistical models for early detection ofkey users to minimize activities out of black-hat marketplaces.R
WWW , 2013.[7] E. De Cristofaro, A. Friedman, G. Jourjon, M. A. Kaafar, and M. Z.Shafiq. Paying for Likes? Understanding Facebook Like Fraud UsingHoneypots. In
IMC , 2014. [8] Facebook, Inc. Updates in Facebooks Fight Against Spam and Spam-mers. http://on.fb.me/1cazvTm, January 2011.[9] H. Ge, J. Caverlee, and K. Lee. Crowds, Gigs, and Super Sellers: AMeasurement Study of a Supply-Driven Crowdsourcing Marketplace. In
ICWSM , 2015.[10] K. Lee, S. Webb, and H. Ge. The Dark Side of Micro-Task Marketplaces:Characterizing Fiverr and Automatically Detecting Crowdturfing. In
ICWSM , 2014.[11] K. Lee, S. Webb, and H. Ge. Characterizing and Automatically DetectingCrowdturfing in Fiverr and Twitter.
Social Network Analysis and Mining ,5(2), 2015.[12] K. Levchenko, A. Pitsillidis, N. Chachra, B. Enright, M. Felegyhazi,C. Grier, T. Halvorson, C. Kanich, C. Kreibich, H. Liu, D. McCoy,N. Weaver, V. Paxson, G. M. Voelker, and S. Savage. Click Trajectories:End-to-End Analysis of the Spam Value Chain. In
IEEE Symposium onSecurity and Privacy , 2011.[13] D. McCoy, H. Dharmdasani, C. Kreibich, G. M. Voelker, and S. Savage.Priceless: The Role of Payments in Abuse-advertised Goods. In
CCS ,2012.[14] D. McCoy, A. Pitsillidis, G. Jordan, N. Weaver, C. Kreibich, B. Krebs,G. M. Voelker, S. Savage, and K. Levchenko. PharmaLeaks: Under-standing the Business of Online Pharmaceutical Affiliate Programs. In
USENIX Security Symposium , 2012.[15] M. Motoyama, D. McCoy, K. Levchenko, S. Savage, and G. M. Voelker.An Analysis of Underground Forums. In
IMC , 2011.[16] M. Motoyama, D. McCoy, K. Levchenko, S. Savage, and G. M. Voelker.Dirty Jobs: The Role of Freelance Labor in Web Service Abuse. In
USENIX Security Symposium , 2011.[17] J. A. Muir and P. C. V. Oorschot. Internet geolocation: Evasion andcounterevasion. volume 42, New York, NY, USA, Dec. 2009.[18] B. F. Rubin. Amazon sues alleged reviews-for-pay sites. http://cnet.co/1EUtHtF, 2015.[19] K. Soska and N. Christin. Measuring the Longitudinal Evolution ofthe Online Anonymous Marketplace Ecosystem. In
USENIX SecuritySymposium , 2015.[20] G. Stringhini, M. Egele, C. Kruegel, and G. Vigna. Poultry Markets:On the Underground Economy of Twitter Followers. In
WOSN , 2012.[21] G. Stringhini, G. Wang, M. Egeley, C. Kruegel, G. Vigna, H. Zheng, andB. Zhao. Follow the Green: Growth and Dynamics in Twitter FollowerMarkets. In
IMC , 2013.[22] The Data World Bank. GDP per capita based on Purchasing PowerParity (PPP). http://goo.gl/zgoRlI, 2014.[23] K. Thomas, D. McCoy, C. Grier, A. Kolcz, and V. Paxson. TraffickingFraudulent Accounts: The Role of the Underground Market in TwitterSpam and Abuse. In
USENIX Security Symposium , 2013.[24] G. Wang, C. Wilson, X. Zhao, Y. Zhu, M. Mohanlal, H. Zheng, andB. Y. Zhao. Serf and Turf: Crowdturfing for Fun and Profit. In
WWW ,2012.[25] H. Xu, D. Liu, H. Wang, and A. Stavrou. E-commerce ReputationManipulation: The Emergence of Reputation-Escalation-as-a-Service. In
WWW , 2015., 2015.