Junji Nakazato

nicter: a large-scale network incident analysis system: case studies for understanding threat landscape

We have been developing the Network Incident analysis Center for Tactical Emergency Response (nicter), whose objective is to detect and identify propagating malwares. The nicter mainly monitors darknet, a set of unused IP addresses, to observe global trends of network threats, while it captures and analyzes malware executables. By correlating the network threats with analysis results of malware, the nicter identifies the root causes (malwares) of the detected network threats. Through a long-term operation of the nicter for more than five years, we have achieved some key findings that would help us to understand the intentions of attackers and the comprehensive threat landscape of the Internet. With a focus on a well-knwon malware, i. e., W32.Downadup, this paper provides some practical case studies with considerations and consequently we could obtain a threat landscape that more than 60% of attacking hosts observed in our dark-net could be infected by W32.Downadup. As an evaluation, we confirmed that the result of the correlation analysis was correct in a rate of 86.18%.

A neural network model for detecting DDoS attacks using darknet traffic features.

This paper presents a fast and large-scale monitoring system for detecting one of the major cyber-attacks, Distributed Denial of Service (DDoS). The proposed system monitors the packet traffic on a subnet of unused IPs called darknet. Almost all darknet packets are originated from malicious activities. However, it is not obvious what traffic patterns DDoS attacks have. Therefore, we adopt a classifier and train it with traffic features of known DDoS attacks using 80/TCP and 53/UDP packets which can be labeled based on the header information and payloads. The proposed system consists of the two parts: pre-processing and classifier. In the pre-processing part, darknet packets for 30 seconds are transformed into a feature vector which consists of 17 traffic features on darknet traffic. As for the classifier part, we adopt Resource Allocating Network with Locality Sensitive Hashing (RAN-LSH) in which data to be trained are selected by using LSH and fast online learning is actualized by training only selected data. The learning of RAN-LSH is carried out not only with the training data for 80/TCP and 53/UDP packets but also with new training data labeled by a supervisor. The performance of the proposed detection system is evaluated for 9,968 training data obtained from 80/TCP and 53/UDP packets and 5,933 test data obtained from darknet packets with other protocols and source/destination ports. The results indicate that the proposed system detects backscatter packets caused by DDoS attacks accurately and adapts to new attacks quickly.

An autonomous online malicious spam email detection system using extended RBF network

In this paper, we propose a new online system to detect malicious spam emails and to adapt to the changes of malicious URLs in the body of spam emails by updating the system daily. For this purpose, we develop an autonomous system that learns from double-bounce emails collected at a mail server. To adapt to new malicious campaigns, only new types of spam emails are learned by introducing an active learning scheme into a classifier model. Here, we adopt Resource Allocating Network with Locality Sensitive Hashing (RAN-LSH) as a classifier model with data selection. In this data selection, the same or similar spam emails that have already been learned are quickly searched for a hash table using Locally Sensitive Hashing, and such spam emails are discarded without learning. On the other hand, malicious spam emails are sometimes drastically changed along with a new arrival of malicious campaign. In this case, it is not appropriate to classify such spam emails into malicious or benign by a classifier. It should be analyzed by using a more reliable method such as a malware analyzer. In order to find new types of spam emails, an outlier detection mechanism is implemented in RAN-LSH. To analyze email contents, we adopt the Bag-of-Words (BoW) approach and generate feature vectors whose attributes are transformed based on the normalized term frequency-inverse document frequency. To evaluate the developed system, we use a dataset of double-bounce spam emails which are collected from March 1st, 2013 to August 29th, 2013. In the experiment, we study the effect of introducing the outlier detection in RAN-LSH. As a result, by introducing the outlier detection, we confirm that the detection accuracy is enhanced on average over the testing period.

Detection of DDoS Backscatter Based on Traffic Features of Darknet TCP Packets

In this work, we propose a method to discriminate backscatter caused by DDoS attacks from normal traffic. Since DDoS attacks are imminent threats which could give serious economic damages to private companies and public organizations, it is quite important to detect DDoS backscatter as early as possible. To do this, 11 features of port/IP information are defined for network packets which are sent within a short time, and these features of packet traffic are classified by Suppurt Vector Machine (SVM). In the experiments, we use TCP packets for the evaluation because they include control flags (e.g. SYN-ACK, RST-ACK, RST, ACK) which can give label information (i.e. Backscatter or non-backscatter). We confirm that the proposed method can discriminate DDoS backscatter correctly from unknown dark net TCP packets with more than 90% accuracy.

Detecting Malicious Spam Mails: An Online Machine Learning Approach

Malicious spam is one of the major problems of the Internet nowadays. It brings financial damage to companies and security threat to governments and organizations. Most recent spam emails contain URLs that redirect spam receivers to malicious Web servers. In this paper, we propose an online machine learning based malicious spam email detection system. The term-weighting scheme represents each spam email. These feature vectors are then used as the input of the classifier. The learning is periodically performed to update the classifier so that the system provides increased adaptability to take account of spam emails whose contents change from time to time. A real data set is labeled by the SPIKE system which is developed by NICT. Evaluation experiments show that the detection system is efficient and accurate to identify malicious spam emails.

Adaptive DDoS-Event Detection from Big Darknet Traffic Data

This paper presents an adaptive large-scale monitoring system to detect Distributed Denial of Service (DDoS) attacks whose backscatter packets are observed on the darknet (i.e., unused IP space). To classify DDoS backscatter, 17 features of darknet traffic are defined from IPs/ports information for source and destination hosts. To adapt to the change of DDoS attacks, we newly implement an online learning function in the proposed monitoring system, where an SVM classifier is continuously trained with darknet features transformed from packets during a certain period. In the performance evaluation, we use the MWS Dataset 2014 that consists of darknet packets collected from 1st January 2014 to 28th February 2014 (8 weeks). We demonstrate that the proposed system keeps good test performance in the detection of DDoS backscatter (0.98 in F-measure).

Large-Scale Monitoring for Cyber Attacks by Using Cluster Information on Darknet Traffic Features☆

Abstract This paper presents a machine learning approach to large-scale monitoring for malicious activities on Internet. In the proposed system, network packets sent from a subnet to a darknet (i.e., a set of unused IPs) are collected, and they are transformed into 27-dimensional TAP (Traffic Analysis Profile) feature vectors. Then, a hierarchical clustering is performed to obtain clusters for typical malicious behaviors. In the monitoring phase, the malicious activities in a subnet are estimated from the closest TAP feature cluster. Then, such TAP feature clusters for all subnets are visualized on the proposed monitoring system in real time. In the experiment, we use a big data set of 303,733,994 darknet packs collected from February 1st to February 28th, 2014 (28 days) for monitoring. As a result, we can successfully detect an indication of the pandemic of a new malware, which attacked to the vulnerability of Synology NAS (port 5,000/TCP).

Threshold anonymous group identification and zero-knowledge proof

We show that the communication efficient t-out-of-m scheme proposed by De Santis, Di Crescenzo, and Persiano [Communicationefficient anonymous group identification, ACM Conference on Computer and Communications Security, (1998) 73-82] is incorrect; an authorized group may fail to prove the identity even though the verifier is honest. We rigorously discuss the condition where the scheme works correctly. In addition, we propose a new scheme attaining Θ(mn) communication complexity, where n is the security parameter. It improves the current best communication complexity Θ(mn) of the t-out-of-m scheme, and it can be also considered as a zero-knowledge proof for t out of m secrets.

Privacy enhancing credentials

Using pairing techniques, we propose an anonymous authenticated key exchange scheme based on credentials issued by a trusted third party. The protocol satisfies several security properties related to user privacy such as unforgeability, limitability, non-transferability, and unlinkability.