Luca Allodi

Comparing Vulnerability Severity and Exploits Using Case-Control Studies

(U.S) Rule-based policies to mitigate software risk suggest to use the CVSS score to measure the individual vulnerability risk and act accordingly: an HIGH CVSS score according to the NVD (National (U.S.) Vulnerability Database) is therefore translated into a “Yes”. A key issue is whether such rule is economically sensible, in particular if reported vulnerabilities have been actually exploited in the wild, and whether the risk score do actually match the risk of actual exploitation. We compare the NVD dataset with two additional datasets, the EDB for the white market of vulnerabilities (such as those present in Metasploit), and the EKITS for the exploits traded in the black market. We benchmark them against Symantec’s threat explorer dataset (SYM) of actual exploit in the wild. We analyze the whole spectrum of CVSS submetrics and use these characteristics to perform a case-controlled analysis of CVSS scores (similar to those used to link lung cancer and smoking) to test its reliability as a risk factor for actual exploitation. We conclude that (a) fixing just because a high CVSS score in NVD only yields negligible risk reduction, (b) the additional existence of proof of concepts exploits (e.g. in EDB) may yield some additional but not large risk reduction, (c) fixing in response to presence in black markets yields the equivalent risk reduction of wearing safety belt in cars (you might also die but still. . . ). On the negative side, our study shows that as industry we miss a metric with high specificity (ruling out vulns for which we shouldn’t worry). [In order to address the feedback from BlackHat 2013’s audience, the final revision (V3) provides additional data in Appendix A detailing how the control variables in the study affect the results.](U.S.) Rule-based policies for mitigating software risk suggest using the CVSS score to measure the risk of an individual vulnerability and act accordingly. A key issue is whether the ‘danger’ score does actually match the risk of exploitation in the wild, and if and how such a score could be improved. To address this question, we propose using a case-control study methodology similar to the procedure used to link lung cancer and smoking in the 1950s. A case-control study allows the researcher to draw conclusions on the relation between some risk factor (e.g., smoking) and an effect (e.g., cancer) by looking backward at the cases (e.g., patients) and comparing them with controls (e.g., randomly selected patients with similar characteristics). The methodology allows us to quantify the risk reduction achievable by acting on the risk factor. We illustrate the methodology by using publicly available data on vulnerabilities, exploits, and exploits in the wild to (1) evaluate the performances of the current risk factor in the industry, the CVSS base score; (2) determine whether it can be improved by considering additional factors such the existence of a proof-of-concept exploit, or of an exploit in the black markets. Our analysis reveals that (a) fixing a vulnerability just because it was assigned a high CVSS score is equivalent to randomly picking vulnerabilities to fix; (b) the existence of proof-of-concept exploits is a significantly better risk factor; (c) fixing in response to exploit presence in black markets yields the largest risk reduction.

Quantitative Assessment of Risk Reduction with Cybercrime Black Market Monitoring

Cybercrime is notoriously maintained and empowered by the underground economy, manifested in black markets. In such markets, attack tools and vulnerability exploits are constantly traded. In this paper, we focus on making a quantitative assessment of the risk of attacks coming from such markets, and investigating the expected reduction in overall attacks against final users if, for example, vulnerabilities traded in the black markets were all to be promptly patched. In order to conduct the analysis, we mainly use the data on (a) vulnerabilities bundled in 90+ attack tools traded in the black markets collected by us; (b) actual records of 9 × 107 attacks collected from Symantecs Data Sharing Programme WINE. Our results illustrate that black market vulnerabilities are an important source of risk for the population of users; we further show that vulnerability mitigation strategies based on black markets monitoring may outperform traditional strategies based on vulnerability CVSS scores by providing up to 20% more expected reduction in attacks.

A preliminary analysis of vulnerability scores for attacks in wild: the ekits and sym datasets

NVD and Exploit-DB are the de facto standard databases used for research on vulnerabilities, and the CVSS score is the standard measure for risk. On open question is whether such databases and scores are actually representative of attacks found in the wild. To address this question we have constructed a database (EKITS) based on the vulnerabilities currently used in exploit kits from the black market and extracted another database of vulnerabilities from Symantecs Threat Database (SYM). Our final conclusion is that the NVD and EDB databases are not a reliable source of information for exploits in the wild, even after controlling for the CVSS and exploitability subscore. An high or medium CVSS score shows only a significant sensitivity (i.e. prediction of attacks in the wild) for vulnerabilities present in exploit kits (EKITS) in the black market. All datasets exhibit a low specificity.

The Heavy Tails of Vulnerability Exploitation

In this paper we analyse the frequency at which vulnerabilities are exploited in the wild by relying on data collected worldwide by Symantec’s sensors. Our analysis comprises 374 exploited vulnerabilities for a total of 75.7 Million recorded attacks spanning three years (2009-2012). We find that for some software as little as 5% of exploited vulnerabilities is responsible for about 95% of the attacks against that platform. This strongly skewed distribution is consistent for all considered software categories, for which a general take-away is that less than 10% of vulnerabilities account for more than 90% of the attacks (with the exception of pre-2009 Java vulnerabilities). Following these findings, we hypothesise vulnerability exploitation may follow a Power Law distribution. Rigorous hypothesis testing results in neither accepting nor rejecting the Power Law Hypothesis, for which further data collection from the security community may be needed. Finally, we present and discuss the Law of the Work-Averse Attacker as a possible explanation for the heavy-tailed distributions we find in the data, and present examples of its effects for Apple Quicktime and Microsoft Internet Explorer vulnerabilities.

Then and Now: On the Maturity of the Cybercrime Markets The Lesson That Black-Hat Marketeers Learned

Cybercrime activities are supported by infrastructures and services originating from an underground economy. The current understanding of this phenomenon is that the cybercrime economy ought to be fraught with information asymmetry and adverse selection problems. They should make the effects that we observe every day impossible to sustain. In this paper, we show that the market structure and design used by cyber criminals have evolved toward a market design that is similar to legitimate, thriving, online forum markets such as eBay. We illustrate this evolution by comparing the market regulatory mechanisms of two underground forum markets: 1) a failed market for credit cards and other illegal goods and 2) another, extremely active marketplace for vulnerabilities, exploits, and cyber attacks in general. The comparison shows that cybercrime markets evolved from unruly, scam for scammers market mechanisms to mature, regulated mechanisms that greatly favors trade efficiency.

Economic Factors of Vulnerability Trade and Exploitation

Cybercrime markets support the development and diffusion of new attack technologies, vulnerability exploits, and malware. Whereas the revenue streams of cyber attackers have been studied multiple times in the literature, no quantitative account currently exists on the economics of attack acquisition and deployment. Yet, this understanding is critical to characterize the production of (traded) exploits, the economy that drives it, and its effects on the overall attack scenario. In this paper we provide an empirical investigation of the economics of vulnerability exploitation, and the effects of market factors on likelihood of exploit. Our data is collected first-handedly from a prominent Russian cybercrime market where the trading of the most active attack tools reported by the security industry happens. Our findings reveal that exploits in the underground are priced similarly or above vulnerabilities in legitimate bug-hunting programs, and that the refresh cycle of exploits is slower than currently often assumed. On the other hand, cybercriminals are becoming faster at introducing selected vulnerabilities, and the market is in clear expansion both in terms of players, traded exploits, and exploit pricing. We then evaluate the effects of these market variables on likelihood of attack realization, and find strong evidence of the correlation between market activity and exploit deployment. We discuss implications on vulnerability metrics, economics, and exploit measurement.

Self-organizing Techniques for Knowledge Diffusion in Dynamic Social Networks

In this paper,we model a knowledge diffusion process in a dynamic social network and study two different techniques for self-organization aimed at improving the average knowledge owned by agents and the overall knowledge diffusion within the network.One is a weak self-organization technique requiring a system-level central control, while the other is a strong self-organization technique that each agent exploits based on local information only. The two techniques are aimed at increasing the knowledge diffusion by mitigating the hype effect and the network congestion that the system dynamics shows systematically. Results of simulations are analyzed for different configurations, discussing how the improvements in knowledge diffusion are influenced by the emergent network topology and the dynamics produced by interacting agents. Our theoretical results, while preliminary, may have practical implications in contexts where the polarization of interests in a community is critical.

The Work-Averse Attacker Model

In this paper we present and validate a novel attacker model based on the economic notion that the attacker has limited resources to forge a new attack. We focus on the vulnerability exploitation case, whereby the attacker has to choose whether to exploit a new vulnerability or keep an old one. We postulate that most vulnerabilities remain unattacked, and that the exploit development cycle relates to software updates rather than to the disclosure of new vulnerabilities. We develop a simple mathematical model to show the mechanisms underlying our observations and name it “The Work-Averse Attacker Model”. We then leverage Symantec’s data sharing platform WINE to validate our model by analysing records of attacks against more than 1M real systems. We find the ‘Model of the Work-Averse Attacker’ to be strongly supported by the data and, in particular, that: (a) the great majority of attacks per software version is driven by one vulnerability only; (b) an exploit lives two years before being substituted by a new one; (c) the exploit arrival rate depends on the software’s update rate rather than on time or knowledge of the vulnerability.

Modifying trust dynamics through cooperation and defection in evolving social networks

We present a model of social network that shows a dynamic emergent behavior simulating actors that exchange knowledge based on their preferences, expertise and friendship relations. The network presents a stochastic interaction behavior that tends to create communities, driven by the assortative mixing and triadic closures. Our first research goal is to investigate the features driving the formation of communities and their characteristics under different configurations of the network. In particular we focus on trust which we analyze qualitatively as dependent on the frequency and pattern of interactions. To this aim, we ran simulations of different network configurations and analyzed the resulting statistics. The second research goal is to study the effects of node deception and cooperation on the social network behavior; our primary metric is trust and we evaluated how, under specific conditions, it is possible to manipulate trust in some non trivial ways.

The asymmetric diffusion of trust between communities: simulations in dynamic social networks

In this work, we present a model of social network showing non-trivial effects on the dynamics of trust and communication. Our models results meet the characteristics of a typical social network, such as the limited node degree, assortativeness, clustering and communities formation. Simulations have been run first to present some of the most fundamental relations among the main models attributes. Next, we focused on the emerging asymmetry with which trust develops within different communities in a network. In particular, we considered categories of nodes differing for their communication profiles and a specific example of bridge between two communities. The results are discussed to provide insights about the dynamic formation of communities based on trust relations. These results are the basis for future works with the aim of better understanding the dynamics of the diffusion of trust and its influence on a growing social network.