Mohammad Ghasemzadeh

Applying reinforcement learning for web pages ranking algorithms

Ranking web pages for presenting the most relevant web pages to users queries is one of the main issues in any search engine. In this paper, two new ranking algorithms are offered, using Reinforcement Learning (RL) concepts. RL is a powerful technique of modern artificial intelligence that tunes agents parameters, interactively. In the first step, with formulation of ranking as an RL problem, a new connectivity-based ranking algorithm, called RL_Rank, is proposed. In RL_Rank, agent is considered as a surfer who travels between web pages by clicking randomly on a link in the current page. Each web page is considered as a state and value function of state is used to determine the score of that state (page). Reward is corresponded to number of out links from the current page. Rank scores in RL_Rank are computed in a recursive way. Convergence of these scores is proved. In the next step, we introduce a new hybrid approach using combination of BM25 as a content-based algorithm and RL_Rank. Both proposed algorithms are evaluated by well known benchmark datasets and analyzed according to concerning criteria. Experimental results show using RL concepts leads significant improvements in raking algorithms.

A novel real-time routing protocol in wireless sensor networks

One of the most important and challenging issues in real-time applications of resource-constrained wireless sensor networks (WSNs) is providing end-to-end delay requirement. To address such an issue a few QoS routing protocols have been proposed. THVR (Two-Hop based Velocity based routing protocol)is newly proposed real-time protocol is based on the concept of using two-hop neighbor information for routing decision. In this paper we propose a novel real-time Power-Aware Two-Hop (PATH) based routing protocol. PATH improves real-time performance by means of reducing the packet dropping in routing decisions. PATH is based on the concept of using two-hop neighbor information and power-control mechanism. The former is used for routing decisions and the latter is deployed to improve link quality as well as reducing the delay. PATH dynamically adjusts transmitting power in order to reduce the probability of packet dropping. Also PATH addresses practical issue like network holes, scalability and loss links in WSNs .We analyze PATH and compare it with THVR. Our analytical results show that PATH can perform better than THVR in term of energy consumption and delay.

K-terminal Network Reliability Evaluation Using Binary Decision Diagram

Binary decision diagram (BDD) is a modern data structure proved to be compact in representation and efficient in manipulation of Boolean formulas. Using binary decision diagram in network reliability analysis has already been investigated by some researchers. In this paper we show how an exact algorithm for network reliability can be improved and implemented efficiently using CUDD - Colorado University decision diagram.

Embedding memoization to the semantic tree search for deciding QBFs

Quantified Boolean formulas (QBFs) play an important role in artificial intelligence subjects, specially in planning, knowledge representation and reasoning [20] In this paper we present ZQSAT (sibling of our FZQSAT [15]), which is an algorithm for evaluating quantified Boolean formulas QBF is a language that extends propositional logic in such a way that many advanced forms of reasoning can be easily formulated and evaluated ZQSAT is based on ZDD, which is a variant of BDD, and an adopted version of the DPLL algorithm The program has been implemented in C using the CUDD package The capability of ZDDs in storing sets of subsets efficiently enabled us to store the clauses of a QBF very compactly and led us to implement the search algorithm in such a way that we could store and reuse the results of all previously solved subformulas with few overheads This idea along some other techniques, enabled ZQSAT to solve some standard QBF benchmark problems faster than the best existing QSAT solvers.

An automatic method for CVSS score prediction using vulnerabilities description

In this paper we introduce an objective method for CVSS score calculation. CVSS is a well known and mostly used method for giving priority to software vulnerabilities. Currently it is being calculated by some slightly subjective methods which require enough skill and knowledge. This research shows how we can benefit from natural language description of vulnerabilities for CVSS calculation. The data that were used for implementation and evaluation of the proposed models consists of the available CVE vulnerability descriptions and their corresponding CVSS scores from the OSVDB database. First, feature vectors were extracted using text mining tools and techniques, and then the SVM and Random-Forest algorithms as well as fuzzy systems were examined to predict the concerned CVSS scores. In spite of the fact that SVM and Random-Forest are mostly used and trusted methods in prediction, results of this research bear a witness that using fuzzy systems can give comparable and even better results. In addition, implementation of the fuzzy based system is much easier and faster. Although so far, there have been so little efforts in using the information embedded in textual materials regarding vulnerabilities, this research shows that it will be valuable to utilize them in systems security establishment.

MA-DSR; Multi Agent based Adaptive DSR protocol with intelligent behavior in realistic environments

A mobile ad-hoc network (MANET) is a collection of wireless mobile nodes forming a temporary Network. These networks do not require any existing infrastructure or central administration. To make MANETs adaptive to different mobility and traffic patterns, this paper proposes a novel routing scheme which is utilized mobile agents and attempts to develop DSR routing protocol in MANETs with simple node-level management behavior resulting in overall system optimization. We develop a probabilistic multi-path routing algorithm and incorporates factors like signal strength into the route metrics so as to predict link breaks before they actually occur. in addition to signal strength and shortest-path metrics, our algorithm updates the goodness of choosing a particular path based on congestion measurement and energy level in each node. By All of the above improvements in DSR, the simulation results testify that our new robust algorithm can be suitable in ad-hoc situations. We refer to the protocol as the multi agents based adaptive DSR (MA-DSR).

A novel ranking algorithm based on Reinforcement Learning

People are very interested in finding most related web pages when they are looking for some information about a topic in the web. This is the responsibility of the search engine to use effective ranking methods to find the most related web pages for a given query. In this paper, we propose a novel ranking method which is based on connectivity and also benefits from the Reinforcement Learning (RL) concepts. RL problems are structured around estimating value functions. In our algorithm, each web page is considered as a state and its score is as value function of the state. So, the key elements in our method are agent, value function and considered interstate transition rewards. Also we inspected a hybrid ranking algorithm which combined results of several basic ranking algorithms. The proposed algorithms are evaluated by using well known benchmark data sets and they are analyzed according to concerning criteria. Experimental results show that applying reinforcement learning method leads to considerable improvements and the hybrid algorithm can outperform all other basic ranking algorithms.

Condition aware robust routing algorithm with cross layer technique for ad hoc situations

Abstract In a MANET, mobile nodes must cooperate to dynamically establish routes using multi-hop wireless links. Some of the main challenges in this area related to routing protocols. These networks do not require any existing infrastructure or central administration. To make MANETs adaptive to different mobility and traffic patterns, this paper proposes a novel routing scheme which is utilized mobile agents and attempts to develop DSR protocol in MANETs with simple node-level management behavior resulting in overall system optimization. We develop a probabilistic multi-path routing algorithm and incorporates factors like signal strength into the route metrics so as to predict link breaks before they actually occur. in addition to signal strength and shortest-path metrics, our algorithm updates the goodness of choosing a particular path based on congestion measurement and energy level in each node. By All of the above improvements in DSR, the simulation results testify that our new Robust algorithm can be suitable in ad-hoc situations. We refer to the protocol as the Multi Agent based Adaptive DSR (MA-DSR). Finally, this paper proposes a new cross layer approach and some modifications in the MAC layer for RERR Analysis.. By this way, we can detect reason of RERR reception and apply proper mechanisms for performance improvement. We introduce yellow, red and black lists for decrease escaped, low energy, congested and insecure nodes chances in routing strategy. We expect that new condition aware MA-DSR, shows better performance.

Adaptive DSR Protocol with Cooperative Agents for Different Mobility and Traffic Patterns

The autonomous character of mobile ad-hoc networks (MANET) poses significant challenges on network communications. Some of the main challenges in this area related to routing protocols. To make MANETs adaptive to different mobility and traffic patterns, this paper proposes a novel routing scheme which is utilized cross layer design and attempts to develop DSR routing protocol in MANETs with simple node-level management behavior resulting in overall system optimization. We develop a probabilistic multipath routing algorithm and incorporates factors like signal strength into the route metrics so as to predict link breaks before they actually occur. In addition to signal strength and shortest-path metrics, our algorithm updates the goodness of choosing a particular path based on congestion measurement and energy level in each node. By All of the above improvements in conventional DSR, the simulation results testify that our new Robust DSR algorithm can be suitable in ad-hoc situations.

Remote Control and Overall Administration of Computer Networks, Using Short Message Service

There is always a great concern for network administrators, how to deal with network failure in reasonable time span. Network failure can happen any time of the day and administrator may not be informed in time and even then would not be able to enforce the remedy effectively. This scenario will arise when physical access to the network devices is not possible; i.e. the administrator is away from network locations. In this paper a new approach has been adopted in such a way that, the information on network status is gathered by an application, using SNMP protocol and is sent to network administrator by SMS. This approach also will allow network administrator to apply all necessary commands, using SMS. The short messages are then converted to be used by SNMP protocol to rectify faults, causing network failure.