Eric Nunes

Darknet and deepnet mining for proactive cybersecurity threat intelligence

In this paper, we present an operational system for cyber threat intelligence gathering from various social platforms on the Internet particularly sites on the darknet and deepnet. We focus our attention to collecting information from hacker forum discussions and marketplaces offering products and services focusing on malicious hacking. We have developed an operational system for obtaining information from these sites for the purposes of identifying emerging cyber threats. Currently, this system collects on average 305 high-quality cyber threat warnings each week. These threat warnings include information on newly developed malware and exploits that have not yet been deployed in a cyber-attack. This provides a significant service to cyber-defenders. The system is significantly augmented through the use of various data mining and machine learning techniques. With the use of machine learning models, we are able to recall 92% of products in marketplaces and 80% of discussions on forums relating to malicious hacking with high precision. We perform preliminary analysis on the data collected, demonstrating its application to aid a security expert for better threat analysis.

Cyber-Deception and Attribution in Capture-the-Flag Exercises

Attributing the culprit of a cyber-attack is widely considered one of the major technical and policy challenges of cyber-security. The lack of ground truth for an individual responsible for a given attack has limited previous studies. Here, we overcome this limitation by leveraging DEFCON capture-the-flag (CTF) exercise data where the actual ground-truth is known. In this work, we use various classification techniques to identify the culprit in a cyberattack and find that deceptive activities account for the majority of misclassified samples. We also explore several heuristics to alleviate some of the misclassification caused by deception.

Proactive identification of exploits in the wild through vulnerability mentions online

The number of software vulnerabilities discovered and publicly disclosed is increasing every year; however, only a small fraction of them is exploited in real-world attacks. With limitations on time and skilled resources, organizations often look at ways to identify threatened vulnerabilities for patch prioritization. In this paper, we present an exploit prediction model that predicts whether a vulnerability will be exploited. Our proposed model leverages data from a variety of online data sources (white-hat community, vulnerability researchers community, and darkweb/deepweb sites) with vulnerability mentions. Compared to the standard scoring system (CVSS base score), our model outperforms the baseline models with an F1 measure of 0.40 on the minority class (266% improvement over CVSS base score) and also achieves high True Positive Rate at low False Positive Rate (90%, 13%, respectively). The results demonstrate that the model is highly effective as an early predictor of exploits that could appear in the wild. We also present a qualitative and quantitative study regarding the increase in the likelihood of exploitation incurred when a vulnerability is mentioned in each of the data sources we examine.

Argumentation models for cyber attribution

A major challenge in cyber-threat analysis is combining information from different sources to find the person or the group responsible for the cyber-attack. It is one of the most important technical and policy challenges in cybersecurity. The lack of ground truth for an individual responsible for an attack has limited previous studies. In this paper, we take a first step towards overcoming this limitation by building a dataset from the capture-the-flag event held at DEFCON, and propose an argumentation model based on a formal reasoning framework called DeLP (Defeasible Logic Programming) designed to aid an analyst in attributing a cyber-attack. We build models from latent variables to reduce the search space of culprits (attackers), and show that this reduction significantly improves the performance of classification-based approaches from 37% to 62% in identifying the attacker.

Patch before exploited: An approach to identify targeted software vulnerabilities

The number of software vulnerabilities discovered and publicly disclosed is increasing every year; however, only a small fraction of these vulnerabilities are exploited in real-world attacks. With limitations on time and skilled resources, organizations often look at ways to identify threatened vulnerabilities for patch prioritization. In this chapter, an exploit prediction model is presented, which predicts whether a vulnerability will likely be exploited. Our proposed model leverages data from a variety of online data sources (white hat community, vulnerability research community, and dark web/deep web (DW) websites) with vulnerability mentions. Compared to the standard scoring system (CVSS base score) and a benchmark model that leverages Twitter data in exploit prediction, our model outperforms the baseline models with an F1 measure of 0.40 on the minority class (266% improvement over CVSS base score) and also achieves high true positive rate and low false positive rate (90%, 13%, respectively), making it highly effective as an early predictor of exploits that could appear in the wild. A qualitative and a quantitative study are also conducted to investigate whether the likelihood of exploitation increases if a vulnerability is mentioned in each of the examined data sources. The proposed model is proven to be much more robust than adversarial examples—postings authored by adversaries in the attempt to induce the model to produce incorrect predictions. A discussion on the viability of the model is provided, showing cases where the classifier achieves high performance, and other cases where the classifier performs less efficiently.

Argumentation-Based Cyber Attribution: The DeLP3E Model

In cyber attribution, knowledge bases consisting of all the available information for a specific domain, along with the current state of affairs, will typically contain contradictory data coming from different sources, as well as data with varying degrees of uncertainty attached. In this chapter, we propose a probabilistic structured argumentation framework that arises from the extension of Presumptive Defeasible Logic Programming (PreDeLP) with probabilistic models, and argue that this formalism is especially suitable for handling such contradictory and uncertain data–hence the framework would be well-suited for cyber attribution. We conclude with the demonstration—via a case study—of how our framework can be used to address the attribution problem in cybersecurity.

Enhanced Data Collection for Cyber Attribution

Cyber attribution is a difficult problem, and conducting attribution research is made even more difficult by a lack of data with ground truth. In this chapter, we describe a game-based framework (Capture-the-Flag) to produce cyber attribution data with deception. We discuss the motivation and the design of the contest and the framework to record data. The framework is available as open source software.

Baseline Cyber Attribution Models

Attributing the culprit of a cyberattack is widely considered one of the major technical and policy challenges of cybersecurity. While the lack of ground truth for an individual responsible for a given attack has limited previous studies, here we overcome this limitation by leveraging DEFCON capture-the-flag (CTF) exercise data where the actual ground truth is known. In this chapter, we use various classification techniques to identify the culprit in a cyberattack and find that deceptive activities account for the majority of misclassified attacks. We also explore several heuristics to alleviate some of the misclassification caused by deception.

Belief revision in DeLP3E

Any artificial intelligence tool designed for cyber-attribution must deal with information coming from different sources that invariably leads to incompleteness, overspecification, or inherently uncertain content. The presence of these varying levels of uncertainty doesn’t mean that the information is worthless—rather, these are hurdles that the knowledge engineer must learn to work with. In this chapter, we continue developing the DeLP3E model introduced in the previous chapter, focusing now on the problem of belief revision in DeLP3E. We first propose a non-prioritized class of revision operators called AFO (Annotation Function-based Operators); then, we go on to argue that in some cases it may be desirable to define revision operators that take quantitative aspects into account (such as how the probabilities of certain literals or formulas of interest change after the revision takes place). As a result, we propose the QAFO (Quantitative Annotation Function-based Operators) class of operators, a subclass of AFO, and study the complexity of several problems related to their specification and application in revising knowledge bases. Finally, we present an algorithm for computing the probability that a literal is warranted in a DeLP3E knowledge base, and discuss how it could be applied towards implementing QAFO-style operators that compute approximations rather than exact operations.

Applying argumentation models for cyber attribution

A major challenge in cyberthreat analysis is combining information from different sources to find the person or the group responsible for the cyber-attack. In this chapter, we leverage the dataset from the capture-the-flag event held at DEFCON discussed in Chap. 2, and propose DeLP3E model comprised solely of the AM (that is, without probabilistic information) designed to aid an analyst in attributing a cyberattack. We build models from latent variables to reduce the search space of culprits (attackers), and show that this reduction significantly improves the accuracy of the classification-based approaches discussed in Chap. 2 from 37% to 62% in identifying the attacker.