Shigeki Goto

Identifying Heavy-Hitter Flows from Sampled Flow Statistics ∗

SUMMARY With the rapid increase of link speed in recent years, packet sampling has become a very attractive and scalable means in collecting flow statistics; however, it also makes inferring original flow characteristics much more difficult. In this paper, we develop techniques and schemes to identify flows with a very large number of packets (also known as heavy-hitter flows) from sampled flow statistics. Our approach follows a two-stage strategy: We first parametrically estimate the original flow length distribution from sampled flows. We then identify heavy-hitter flows with Bayes’ theorem, where the flow length distribution estimated at the first stage is used as an a priori distribution. Our approach is validated and evaluated with publicly available packet traces. We show that our approach provides a very flexible framework in striking an appropriate balance between false positives and false negatives when sampling frequency is given.

A new intrusion detection method based on process profiling

There are two well-known models for intrusion detection-anomaly intrusion detection (AID) model and misuse intrusion detection (MID) model. The former analyzes user behavior and the statistics of a process in a normal situation, and checks whether the system is being used in a different manner. The latter maintains a database of known intrusion techniques and detects intrusion by comparing behavior against the database. An intrusion detection method based on an AID model can detect a new intrusion method, but needs to update the data describing user behavior and statistics in normal usage. We call these information profiles. There are several problems in AID to be addressed. The profiles tend to be large. Detecting intrusion needs a large amount of system resources, like CPU time and memory and disk space. An MID model requires fewer system resources to detect intrusion. However, it cannot detect new, unknown intrusion methods. Our method solves these problems by recording system calls from daemon processes and setuid programs. We improved detection accuracy by adopting a DP matching scheme.

Detecting Android Malware by Analyzing Manifest Files

The threat of Android malware has increased owing to the increasing popularity of smartphones. Once an Android smartphone is infected with malware, the user suffers from various damages, such as the theft of personal information stored in the smartphones, the unintentional sending of short messages to premium-rate numbers without the users knowledge, and the ability for the infected smartphones to be remotely operated and used for other malicious attacks. However, there are currently insufficient defense mechanisms against Android malware. This study proposes a new method to detect Android malware. The new method analyzes only manifest files that are required in Android applications. It realizes a lightweight approach for detection, and its effectiveness is experimentally confirmed by employing real samples of Android malware. The result shows that the new method can effectively detect Android malware, even when the sample is unknown.

Detecting Malicious Websites by Learning IP Address Features

Web-based malware attacks have become one of the most serious threats that need to be addressed urgently. Several approaches that have attracted attention as promising ways of detecting such malware include employing various blacklists. However, these conventional approaches often fail to detect new attacks owing to the versatility of malicious websites. Thus, it is difficult to maintain up-to-date blacklists with information regarding new malicious websites. To tackle this problem, we propose a new method for detecting malicious websites using the characteristics of IP addresses. Our approach leverages the empirical observation that IP addresses are more stable than other metrics such as URL and DNS. While the strings that form URLs or domain names are highly variable, IP addresses are less variable, i.e., IPv4 address space is mapped onto 4-bytes strings. We develop a lightweight and scalable detection scheme based on the machine learning technique. The aim of this study is not to provide a single solution that effectively detects web-based malware but to develop a technique that compensates the drawbacks of existing approaches. We validate the effectiveness of our approach by using real IP address data from existing blacklists and real traffic data on a campus network. The results demonstrate that our method can expand the coverage/accuracy of existing blacklists and also detect unknown malicious websites that are not covered by conventional approaches.

Remote attack detection method in IDA: MLSI-based intrusion detection using discriminant analysis

In order to detect intrusions, IDA (Intrusion Detection Agent system) initially monitors system logs in order to discover an MLSI-which is an certain event which in many cases occurs during an intrusion. If an MLSI is found, then IDA judges whether the MLSI is accompanied by an intrusion. We adopt discriminant analysis to analyze information after IDA detects an MLSI in a remote attack. Discriminant analysis provides a classification function that allows IDA to separate intrusive activities from non-intrusive activities. Using discriminant analysis, we can detect intrusions by analyzing only a part of system calls occurring on a host machine, and we can determine whether an unknown sample is an intrusion. In this paper, we explain in detail how we perform discriminant analysis to detect intrusions, and evaluate the classification function. We also describe how to extract a sample from system logs, which is necessary to implement the discriminant analysis function in IDA.

Active measurement and analysis of delay time in the Internet

This paper analyzes the delay time distribution in the Internet. We send sample packets with a time stamp to measure the delay time. It is well known that the delay time in communication links follows the exponential distribution. However, the earlier models cannot explain the distribution when a communication link is heavily overloaded. This paper proposes to use a new model for the Internet. We have applied our model to the measurement results successfully.

SFMap: Inferring Services over Encrypted Web Flows Using Dynamical Domain Name Graphs

Most modern Internet services are carried over the web. A significant amount of web transactions is now encrypted and the transition to encryption has made it difficult for network operators to understand traffic mix. The goal of this study is to enable network operators to infer hostnames within HTTPS traffic because hostname information is useful to understand the breakdown of encrypted web traffic. The proposed approach correlates HTTPS flows and DNS queries/responses. Although this approach may appear trivial, recent deployment and implementation of DNS ecosystems have made it a challenging research problem; i.e., canonical name tricks used by CDNs, the dynamic and diverse nature of DNS TTL settings, and incomplete measurements due to the existence of various caching mechanisms. To tackle these challenges, we introduce domain name graph (DNG), which is a formal expression that characterizes the highly dynamic and diverse nature of DNS mechanisms. Furthermore, we have developed a framework called Service-Flow map (SFMap) that works on top of the DNG. SFMap statistically estimates the hostname of an HTTPS server, given a pair of client and server IP addresses. We evaluate the performance of SFMap through extensive analysis using real packet traces collected from two locations with different scales. We demonstrate that SFMap establishes good estimation accuracies and outperforms a state-of-the-art approach.

A multi-agent monitoring and diagnostic system for TCP/IP-based network and its coordination

Abstract This paper describes an application of an AI-based multiagent system to the management and diagnosis of TCP/IP-based intranet/intra-AS (autonomous system) computer networks. A copy of this system is attached to each network segment and is made responsible for that segment. It captures packets in the promiscuous mode and analyzes their data in real time. Based on this analysis, the data needed to manage the local network are obtained, any changes in the local network or network components are recognized, and problems are detected. When a problem is reported by a user or detected by the system, the problem is diagnosed cooperatively or autonomously depending on its type. The activities of the agents are coordinated based on the concepts of coordination levels and functional organizations. An example of cooperative diagnosis clarifies why this multiagent approach is essential for network management.

Network surveillance for detecting intrusions

The paper proposes a network surveillance method for detecting malicious activities. Based on the hypothesis that unusual conducts like system exploitation will trigger an abnormal network traffic, we try to detect this anomalous traffic pattern as a sign of malicious, or at least suspicious activities. Capturing and analyzing of a network traffic pattern is implemented with an idea of port profiling, where measures representing various characteristics of connections are monitored and recorded for each port. Though the generation of the port profiles requires a small amount of calculation, they exhibit high stability and robustness. By comparing the pattern exhibited by live traffic with the expected behavior recorded in the profile, intrusive activities like compromising backdoors or invoking trojan programs are successfully detected.

DomainProfiler: Discovering Domain Names Abused in Future

Cyber attackers abuse the domain name system (DNS) to mystify their attack ecosystems, they systematically generate a huge volume of distinct domain names to make it infeasible for blacklisting approaches to keep up with newly generated malicious domain names. As a solution to this problem, we propose a system for discovering malicious domain names that will likely be abused in future. The key idea with our system is to exploit temporal variation patterns (TVPs) of domain names. The TVPs of domain names include information about how and when a domain name has been listed in legitimate/popular and/or malicious domain name lists. On the basis of this idea, our system actively collects DNS logs, analyzes their TVPs, and predicts whether a given domain name will be used for malicious purposes. Our evaluation revealed that our system can predict malicious domain names 220 days beforehand with a true positive rate of 0.985.