Pir Abdul Rasool Qureshi

Hybrid model of content extraction

We present a hybrid model for content extraction from HTML documents. The model operates on Document Object Model (DOM) tree of the corresponding HTML document. It evaluates each tree node and associated statistical features like link density and text distribution across the node to predict significance of the node towards overall content provided by the document. Once significance of the nodes is determined, the formatting characteristics like fonts, styles and the position of the nodes are evaluated to identify the nodes with similar formatting as compared to the significant nodes. The proposed hybrid model is derived from two different models, i.e., one is based on statistical features and other on formatting characteristics and achieved the best accuracy. We describe the validity of model with the help of experiments conducted on the standard data sets. The results revealed that the proposed model outperformed other existing content extraction models. We present a browser based implementation of the proposed model as proof of concept and compare the implementation strategy with various state of art implementations. We also discuss various applications of the proposed model with special emphasis on open source intelligence.

EWaS: Novel Approach for Generating Early Warnings to Prevent Terrorist Attacks

No terrorist attacks are performed at once. The attacks are usually carried out after a well planned analysis composed of complete requirements analysis and resource allocation phases. Presently, a consensus is found that the Internet is being used by terrorists for implementation of their plans, and many ideas of early warning systems have been appearing recently. These early warning systems are based on the principles of retrieving and extracting information available online usually in the form of news items or similar pages. The idea behind such warning systems is mainly frequency-based (i. e., the number of appearances of any entity over a particular time span). The rest of the logic revolves around that entity. In this paper, we discuss some important issues related to this type of approach. We also present functional requirements for such early warning systems along with some design constraints. Finally, we introduce a novel approach to address these issues, which is currently being implemented in the Early Warning System (EWaS).

Statistical Model for Content Extraction

We present a statistical model for content extraction from HTML documents. The model operates on Document Object Model (DOM) tree of the corresponding HTML document. It evaluates each tree node and associated statistical features to predict significance of the node towards overall content of the document. The model exploits feature set including link densities and text distribution across the nodes of DOM tree. We describe the validity of model with the help of experiments conducted on the standard data sets. The results revealed that the proposed model outperformed other state of art models. We also describe the significance of the model in the domain of counterterrorism and open source intelligence.

Harvesting Information from Heterogeneous Sources

The abundance of information regarding any topic makes the Internet a very good resource. Even though searching the Internet is very easy, what remains difficult is to automate the process of information extraction from the available online information due to the lack of structure and the diversity in the sharing methods. Most of the times, information is stored in different proprietary formats, complying with different standards and protocols which makes tasks like data mining and information harvesting very difficult. In this paper, an information harvesting tool (hetero Harvest) is presented with objectives to address these problems by filtering the useful information and then normalizing the information in a singular non hypertext format. We also discuss state of the art tools along with the shortcomings and present the results of an analysis carried out over different heterogeneous formats along with performance of our tool with respect to each format. Finally, the different potential applications of the proposed tool are discussed with special emphasis on open source intelligence.

Detecting Terrorism Evidence in Text Documents

The paper presents a model to detect terrorism evi- dence in textual documents. The model pre-processes domain spe- cific documents to extract the general patterns of text associated with the domain. The model then incorporates the Conditional Random Field (CRF) model for detection of sentences containing patterns of terrorism evidence. For incorporation of CRF model, the features are selected from generalized patterns rather than the text itself. We prepared a small data set of manually tagged instances of terrorism evidence for training and testing the model. We found that the proposed model achieves better results than other models such as Hidden Markov Model or conventional CRF which are directly applied to text. The proposed model can be applied for improvement of terrorism event extraction and ontology creation systems, especially with the focus towards their effective role in Open Source Intelligence. We describe briefly the existing systems along with possible improvements with incorporation of the presented model at different levels. Index Terms—Open Source Intelligence, Terrorism Evidence Detection, Conditional Random Fields, Terrorism Ontology, Ter- rorism Event Extraction

Practical algorithms for subgroup detection in covert networks

In this paper, we present algorithms for subgroup detection and demonstrated them with a real-time case study of USS Cole bombing terrorist network. The algorithms are demonstrated in an application by a prototype system. The system finds associations between terrorist and terrorist organisations and is capable of determining links between terrorism plots occurred in the past, their affiliation with terrorist camps, travel record, funds transfer, etc. The findings are represented by a network in the form of an Attributed Relational Graph (ARG). Paths from a node to any other node in the network indicate the relationships between individuals and organisations. The system also provides assistance to law enforcement agencies, indicating when the capture of a specific terrorist will more likely destabilise the terrorist network. In this paper, we discuss the important application area related to subgroups in a terrorist cell using filtering of graph algorithms. The novelty of the algorithms can be easily found from the results they produce.

EWAS: Modeling Application for Early Detection of Terrorist Threats

This paper presents a model and system architecture for an early warning system to detect terrorist threats. The paper discusses the shortcomings of state-of-the-art systems and outlines the functional requirements that must to be met by an ideal system working in the counterterrorism domain. The concept of generation of early warnings to predict terrorist threats is presented. The model relies on data collection from open data sources, information retrieval, information extraction for preparing structured workable data sets from available unstructured data, and finally detailed investigation. The conducted investigation includes social network analysis, investigative data mining, and heuristic rules for the study of complex covert networks for terrorist threat indication. The presented model and system architecture can be used as a core framework for an early warning system.

heteroHarvest: Harvesting information from heterogeneous sources

The abundance of information regarding any topic makes the Internet a very good resource. Even though searching the Internet is very easy, what remains difficult is to automate the process of information extraction from the available online information due to the lack of structure and the diversity in the sharing methods. Most of the times, information is stored in different proprietary formats, complying with different standards and protocols which makes tasks like data mining and information harvesting very difficult. In this paper, an information harvesting tool (heteroHarvest) is presented with objectives to address these problems by filtering the useful information and then normalizing the information in a singular non hypertext format. Finally we describe the results of experimental evaluation. The results are found promising with an overall error rate equal to 6.5% across heterogeneous formats.

LanguageNet: A novel framework for processing unstructured text information

In this paper we present LanguageNet—a novel framework for processing unstructured text information from human generated content. The state of the art information processing frameworks have some shortcomings: modeled in generalized form, trained on fixed (limited) data sets, and leaving the specialization necessary for information consolidation to the end users. The proposed framework is the first major attempt to address these shortcomings. LanguageNet provides extended support of graphical methods contributing added value to the capabilities of information processing. We discuss the benefits of the framework and compare it with the available state of the art. We also describe how the framework improves the information gathering process and contribute towards building systems with better performance in the domain of Open Source Intelligence.

Retracted: Exploring the Evolution of Terrorist Network

This paper discusses advancements and new trends in terrorist networks. We investigate a case regarding a recent terror plan that took place in Denmark and we present the analysis of the thwarted plot. Analyzing covert networks after an incident is practically easy for trial purposes. Mapping clandestine networks to thwarted terrorist activities is much more complicated. The network involved in the recent Danish terror plan is studied through publicly available information. Based on that information we mapped a part of the network centered on David Headley, who recently confessed to have planned a terrorist attack to take place on Danish soil. Despite its deficiencies, the map gives us an insight into new trends in terrorist organizations and people involved in terrorist plots.