Maryam Shokri

Using reinforcement learning for image thresholding

This paper introduces a reinforcement-learning concept to find the optimal threshold for digital images. The proposed approach can integrate human expert knowledge in an objective or subjective way to overcome the shortcomings of existing methods.

Q(/spl Lambda/)-based image thresholding

One of the problems in image processing is finding an appropriate threshold in order to convert an image to a binary one. In this paper we introduce a new method for image thresholding. We use reinforcement learning as an effective way to find the optimal threshold. Q(Λ) is implemented as a learning algorithm to achieve more accurate results. The reinforcement agent uses objective rewards to explore/exploit the solution space. It means that there is not any experienced operator involved and the reward and punishment function must be defined for the agent. The results show that this method works successfully and can be trained for any particular application.

Knowledge of opposite actions for reinforcement learning

Abstract: Reinforcement learning (RL) is one of the machine intelligence techniques with several characteristics that make it suitable for solving real-world problems. However, RL agents generally face a very large state space in many applications. They must take actions in every state many times to find the optimal policy. In this work, a special type of knowledge about actions is employed to improve the performance of the off-policy, incremental, and model-free reinforcement learning with discrete state and action space. One of the components of RL agent is the action. For each action, its associate opposite action is defined. The actions and opposite actions are implemented in the framework of reinforcement learning to update the value function resulting in a faster convergence. The effects of opposite action on some of the reinforcement learning algorithms are investigated.

A reinforcement agent for threshold fusion

Finding an optimal threshold in order to segment digital images is a difficult task in image processing. Numerous approaches to image thresholding already exist in the literature. In this work, a reinforced threshold fusion for image binarization is introduced which aggregates existing thresholding techniques. The reinforcement agent learns the optimal weights for different thresholds and segments the image globally. A fuzzy reward function is employed to measure object similarities between the binarized image and the original gray-level image, and provide feedback to the agent. The experiments show that promising improvement can be obtained. Three well-established thresholding techniques are combined by the reinforcement agent and the results are compared using error measurements based on ground-truth images.

Tradeoff between exploration and exploitation of OQ(λ) with non-Markovian update in dynamic environments

This paper presents some investigations on tradeoff between exploration and exploitation of opposition-based Q(lambda) with non-Markovian update (NOQ(lambda)) in a dynamic environment. In the previous work the authors applied NOQ(lambda) to the deterministic GridWorld problem. In this paper, we have implemented the NOQ(lambda) algorithm for a simple elevator control problem to test the behavior of the algorithm for non-deterministic and dynamic environment. We also extend the NOQ(lambda) algorithm by introducing the opposition weight to find a better tradeoff between exploration and exploitation for the NOQ(lambda) technique. The value of the opposition weight increases as the number of steps increases. Hence, it has more positive effects on the Q-value updates for opposite actions as the learning progresses. The performance of NOQ(lambda) method is compared with Q(lambda) technique. The experiments indicate that NOQ(lambda) performs better than Q(lambda).

The Concept of Opposition and Its Use in Q-Learning and Q(λ) Techniques

Reinforcement learning (RL) is a goal-directed method for solving problems in uncertain and dynamic environments. RL agents explore the states of the environment in order to find an optimal policy which maps states to reward-bearing actions. This chapter discusses recently introduced techniques to expedite some of the tabular RL methods for off-policy, step-by-step, incremental and model-free reinforcement learning with discrete state and action space. The concept of opposition-based reinforcement learning has been introduced for Q-value updating. Based on this concept, the Q-values can be simultaneously updated for action and opposite action in a given state. Hence, the learning process in general will be accelerated.Several algorithms are outlined in this chapter. The OQ(λ) has been introduced to accelerate Q(λ) algorithm in discrete state spaces. The NOQ(λ) method is an extension of OQ(λ) to operate in a broader range of non-deterministic environments. The update of the opposition trace in OQ(λ) depends on the next state of the opposite action (which generally is not taken by the agent). This limits the usability of this technique to the deterministic environments because the next state should be known to the agent. NOQ(λ) is presented to update the opposition trace independent of knowing the next state for the opposite action. The primary results show that NOQ(λ) can be employed in non-deterministic environments and performs even faster than OQ(λ).

Oppositional target domain estimation using grid-based simulation

In this paper we address the problem of estimating the target domain for search and navigation problems. We propose oppositional target domain estimation by modeling the search and navigation environment as a grid. Typically real-world applications exhibit an environment that is extremely large, dramatically diminishing the usability of intelligent agents for search and navigation. The reduction of the size of environment, hence, can help to increase the efficiency and applicability of the agents. We address this issue by modeling the environment as a grid and estimating the target domain inside the environment. The target domain is a reduced space which includes the target. The proposed technique is specifically concerned with reducing the environment using the concept of opposition. Experimental results show significant reduction of the environment size resulting in a shorter search time.

Reinforcement Agents for E-Learning Applications

Advanced computer systems have become pivotal components for learning. However, we are still faced with many challenges in e-learning environments when developing reliable tools to assist users and facilitate and enhance the learning process. For instance, the problem of creating a user-friendly system that can learn from interaction with dynamic learning requirements and deal with largescale information is still widely unsolved. We need systems that have the ability to communicate and cooperate with the users, learn their preferences and increase the learning efficiency of individual users. Reinforcement learning (RL) is an intelligent technique with the ability to learn from interaction with the environment. It learns from trial and error and generally does not need any training data or a user model. At the beginning of the learning process, the RL agent does not have any knowledge about the actions it should take. After a while, the agent learns which actions yield the maximum reward. The ability of learning from interaction with a dynamic environment and using reward and punishment independent of any training data set makes reinforcement learning a suitable tool for e-learning situations, where subjective user feedback can easily be translated into a reinforcement signal.