Alok Tongaonkar

NetworkProfiler: Towards automatic fingerprinting of Android apps

Network operators need to have a clear visibility into the applications running in their network. This is critical for both security and network management. Recent years have seen an exponential growth in the number of smart phone apps which has complicated this task. Traditional methods of traffic classification are no longer sufficient as the majority of this smart phone app traffic is carried over HTTP/HTTPS. Keeping up with the new applications that come up everyday is very challenging and time-consuming. We present a novel technique for automatically generating network profiles for identifying Android apps in the HTTP traffic. A network profile consists of fingerprints, i.e., unique characteristics of network behavior, that can be used to identify an app. To profile an Android app, we run the app automatically in an emulator and collect the network traces. We have developed a novel UI fuzzing technique for running the app such that different execution paths are exercised, which is necessary to build a comprehensive network profile. We have also developed a light-weight technique, for extracting fingerprints, that is based on identifying invariants in the generated traces. We used our technique to generate network profiles for thousands of apps. Using our network profiles we were able to detect the presence of these apps in real-world network traffic logs from a cellular provider.

Understanding mobile app usage patterns using in-app advertisements

Recent years have seen an explosive growth in the number of mobile devices such as smart phones and tablets. This has resulted in a growing need of the operators to understand the usage patterns of the mobile apps used on these devices. Previous studies in this area have relied on volunteers using instrumented devices or using fields in the HTTP traffic such as User-Agent to identify the apps in network traces. However, the results of the former approach are difficult to be extrapolated to real-world scenario while the latter approach is not applicable to platforms like Android where developers generally use generic strings, that can not be used to identify the apps, in the User-Agent field. In this paper, we present a novel way of identifying Android apps in network traces using mobile in-app advertisements. Our preliminary experiments with real world traces show that this technique is promising for large scale mobile app usage pattern studies. We also present an analysis of the official Android market place from an advertising perspective.

SAMPLES: Self Adaptive Mining of Persistent LExical Snippets for Classifying Mobile Application Traffic

We present SAMPLES: Self Adaptive Mining of Persistent LExical Snippets; a systematic framework for classifying network traffic generated by mobile applications. SAMPLES constructs conjunctive rules, in an automated fashion, through a supervised methodology over a set of labeled flows (the training set). Each conjunctive rule corresponds to the lexical context, associated with an application identifier found in a snippet of the HTTP header, and is defined by: (a) the identifier type, (b) the HTTP header-field it occurs in, and (c) the prefix/suffix surrounding its occurrence. Subsequently, these conjunctive rules undergo an aggregate-and-validate step for improving accuracy and determining a priority order. The refined rule-set is then loaded into an application-identification engine where it operates at a per flow granularity, in an extract-and-lookup paradigm, to identify the application responsible for a given flow. Thus, SAMPLES can facilitate important network measurement and management tasks --- e.g. behavioral profiling [29], application-level firewalls [21,22] etc. --- which require a more detailed view of the underlying traffic than that afforded by traditional protocol/port based methods. We evaluate SAMPLES on a test set comprising 15 million flows (approx.) generated by over 700 K applications from the Android, iOS and Nokia market-places. SAMPLES successfully identifies over 90% of these applications with 99% accuracy on an average. This, in spite of the fact that fewer than 2% of the applications are required during the training phase, for each of the three market places. This is a testament to the universality and the scalability of our approach. We, therefore, expect SAMPLES to work with reasonable coverage and accuracy for other mobile platforms --- e.g. BlackBerry and Windows Mobile --- as well.

Towards self adaptive network traffic classification

Abstract A critical aspect of network management from an operator’s perspective is the ability to understand or classify all traffic that traverses the network. The failure of port based traffic classification technique triggered an interest in discovering signatures based on packet content. However, this approach involves manually reverse engineering all the applications/protocols that need to be identified. This suffers from the problem of scalability; keeping up with the new applications that come up everyday is very challenging and time-consuming. Moreover, the traditional approach of developing signatures once and using them in different networks suffers from low coverage. In this work, we present a novel fully automated packet payload content (PPC) based network traffic classification system that addresses the above shortcomings. Our system learns new application signatures in the network where classification is desired. Furthermore, our system adapts the signatures as the traffic for an application changes. Based on real traces from several service providers, we show that our system is capable of detecting (1) tunneled or wrapped applications, (2) applications that use random ports, and (3) new applications. Moreover, it is robust to routing asymmetry, an important requirement in large ISPs, and has high precision (>97%). Finally, our system is easy to deploy and setup and performs classification in real-time.

Per-user policy enforcement on mobile apps through network functions virtualization

Due to the increasing popularity of smartphones and tablets, mobile apps are becoming the preferred portals for users to access various network services in both residential and enterprise environments. Predominantly using generic HTTP or HTTPS protocols, traffic from different mobile apps is largely indistinguishable. This loss of visibility into mobile app traffic brings new challenges to network management and traffic analysis. It has became very hard to implement network policies based on the differentiation between traffic from compliant and non-compliant mobile apps. This paper presents a system that not only provides network administrators the much desired capability of enforcing policies on mobile app traffic, but also does that at a fine per-user granularity. The proposed system takes a Network Functions Virtualization (NFV) approach and virtualizes an edge router into multiple virtual data planes. Specifically, each data plane serves solely to one particular user and consists of user-specific virtualized network functions. The independence of the virtual data planes facilitates enforcing network policies at the per-user level. To enable policy enforcement on mobile apps, our system includes a sophisticated mobile app identification module to recognize traffic from different apps using preloaded traffic signatures. By exploiting TLS proxying, our system can even enforce policies on those mobile apps adopting traffic encryption. We have implemented a prototype of the proposed system as a wireless access point (AP) using a commodity small form factor PC. Our preliminary experimental evaluations show that the system can scale to modest number of users without much impacting user experience in using the network.

MAPPER: A Mobile Application Personal Policy Enforcement Router for Enterprise Networks

MAPPER is a system for enforcing user-specific policies based on the availability of access nodes that support the capability to dynamically load and execute processing modules on the data path. This work leverages a network access node that, after authenticating a connecting user, loads a set of lightweight virtual machines that process traffic terminated on the user device to implement articulated user-specific access policies. Specifically, we demonstrate how a man-in-the-middle-proxy module, dynamically and opportunistically combined with a module capable of mobile application identification, can implement complex access policies. The man-in-the-middle-proxy module enables MAPPER policies to be applied to both clear and HTTPS traffic, while an intelligent traffic classification system, provides support for policies based on over 250,000 mobile apps spanning both Android and iOS platforms.

Condition Factorization: A Technique for Building Fast and Compact Packet Matching Automata

Rule-based matching on network packet headers is a central problem in firewalls, and network intrusion, monitoring, and access-control systems. To enhance performance, rules are typically compiled into a matching automaton that can quickly identify the subset of rules that are applicable to a given network packet. While deterministic automata provide the best performance, previous research has shown that such automata can be exponential in the size and/or number of rules. Nondeterministic automata can avoid size explosion, but their matching time can increase quickly with the number of rules. In contrast, we present a new technique that constructs polynomial size automata. Moreover, we show that the matching time of our automata is insensitive to the number of rules. The key idea in our approach is that of decomposing and reordering the tests on packet header fields so that the result of performing a test can be utilized on behalf of many rules. Our experiments demonstrate major reductions in space requirements over previous techniques, as well as significant improvements in matching speed. Our technique can uniformly handle prioritized and unprioritized rules, and support applications that require single-match as well as multi-match.

Towards automatic protocol field inference

Security tools have evolved dramatically in the recent years to combat the increasingly complex nature of attacks. However, these tools need to be configured by experts that understand network protocols thoroughly to be effective. In this paper, we present a system called FieldHunter, which automatically extracts fields and infers their types. This information is invaluable for security experts to keep pace with the increasing rate of development of new network applications and their underlying protocols. FieldHunter relies on collecting application messages from multiple sessions. Then, it performs field extraction and inference of their types by taking into consideration statistical correlations between different messages or other associations with meta-data such as message length, client or server IP addresses. We evaluated FieldHunter on real network traffic collected in ISP networks from three different continents. FieldHunter was able to extract security relevant fields and infer their types for well documented network protocols (such as DNS and MSNP) as well as protocols for which the specifications are not publicly available (such as SopCast). Further, we developed a payload-based anomaly detection system for industrial control systems using FieldHunter. The proposed system is able to identify industrial devices behaving oddly, without any previous knowledge of the protocols being used.

Automatic protocol field inference for deeper protocol understanding

Security tools have evolved dramatically in the recent years to combat the increasingly complex nature of attacks, but to be effective these tools need to be configured by experts that understand network protocols thoroughly. In this paper we present FieldHunter, which automatically extracts fields and infers their types; providing this much needed information to the security experts for keeping pace with the increasing rate of new network applications and their underlying protocols. FieldHunter relies on collecting application messages from multiple sessions and then applying statistical correlations is able to infer the types of the fields. These statistical correlations can be between different messages or other associations with meta-data such as message length, client or server IPs. Our system is designed to extract and infer fields from both binary and textual protocols. We evaluated FieldHunter on real network traffic collected in ISP networks from three different continents. FieldHunter was able to extract security relevant fields and infer their nature for well documented network protocols (such as DNS and MSNP) as well as protocols for which the specifications are not publicly available (such as SopCast) and from malware such as (Ramnit).

Botnet protocol inference in the presence of encrypted traffic

Network protocol reverse engineering of botnet command and control (C&C) is a challenging task, which requires various manual steps and a significant amount of domain knowledge. Furthermore, most of todays C&C protocols are encrypted, which prevents any analysis on the traffic without first discovering the encryption algorithm and key. To address these challenges, we present an end-to-end system for automatically discovering the encryption algorithm and keys, generating a protocol specification for the C&C traffic, and crafting effective network signatures. In order to infer the encryption algorithm and key, we enhance state-of-the-art techniques to extract this information using lightweight binary analysis. In order to generate protocol specifications we infer field types purely by analyzing network traffic. We evaluate our approach on three prominent malware families: Sality, ZeroAccess and Ramnit. Our results are encouraging: the approach decrypts all three protocols, detects 97% of fields whose semantics are supported, and infers specifications that correctly align with real protocol specifications.