B.J. Oommen

Discretized pursuit learning automata

The problem of a stochastic learning automaton interacting with an unknown random environment is considered. The fundamental problem is that of learning, through interaction, the best action allowed by the environment (i.e. the action that is rewarded optimally). By using running estimates of reward probabilities to learn the optimal action, an extremely efficient pursuit algorithm (PA), which is presently among the fastest algorithms known, was reported in earlier works. The improvements gained by rendering the PA discrete are investigated. This is done by restricting the probability of selecting an action to a finite and, hence, discrete subset of (0, 1). This improved scheme is proven to be epsilon -optimal in all stationary environments. Furthermore, the experimental results seem to indicate that the algorithm presented is faster than the fastest nonestimator learning automata reported to date, and also faster than the continuous pursuit automaton. >

Generalized pursuit learning schemes: new families of continuous and discretized learning automata

The fastest learning automata (LA) algorithms currently available fall in the family of estimator algorithms introduced by Thathachar and Sastry (1986). The pioneering work of these authors was the pursuit algorithm, which pursues only the current estimated optimal action. If this action is not the one with the minimum penalty probability, this algorithm pursues a wrong action. In this paper, we argue that a pursuit scheme that generalizes the traditional pursuit algorithm by pursuing all the actions with higher reward estimates than the chosen action, minimizes the probability of pursuing a wrong action, and is a faster converging scheme. To attest this, we present two new generalized pursuit algorithms (GPAs) and also present a quantitative comparison of their performance against the existing pursuit algorithms. Empirically, the algorithms proposed here are among the fastest reported LA to date.

Continuous and discretized pursuit learning schemes: various algorithms and their comparison

A learning automaton (LA) is an automaton that interacts with a random environment, having as its goal the task of learning the optimal action based on its acquired experience. Many learning automata (LAs) have been proposed, with the class of estimator algorithms being among the fastest ones, Thathachar and Sastry, through the pursuit algorithm, introduced the concept of learning algorithms that pursue the current optimal action, following a reward-penalty learning philosophy. Later, Oommen and Lanctot extended the pursuit algorithm into the discretized world by presenting the discretized pursuit algorithm, based on a reward-inaction learning philosophy. In this paper we argue that the reward-penalty and reward-inaction learning paradigms in conjunction with the continuous and discrete models of computation, lead to four versions of pursuit learning automata. We contend that a scheme that merges the pursuit concept with the most recent response of the environment, permits the algorithm to utilize the LAs long-term and short-term perspectives of the environment. In this paper, we present all four resultant pursuit algorithms, prove the E-optimality of the newly introduced algorithms, and present a quantitative comparison between them.

Deterministic learning automata solutions to the equipartitioning problem

Three deterministic learning automata solutions to the problem of equipartitioning are presented. Although the first two are epsilon -optimal, they seem to be practically feasible only when a set of W objects is small. The last solution, which uses a novel learning automaton, demonstrates an excellent partitioning capability. Experimentally, this solution converges an order of magnitude faster than the best known algorithm in the literature. >

Graph partitioning using learning automata

Given a graph G, we intend to partition its nodes into two sets of equal size so as to minimize the sum of the cost of the edges having end points in different sets. This problem, called the uniform graph partitioning problem, is known to be NP complete. We propose the first reported learning automaton based solution to the problem. We compare this new solution to various reported schemes such as the B.W. Kernighan and S. Lins (1970) algorithm, and two excellent recent heuristic methods proposed by E. Rolland et al. (1994; 1992)-an extended local search algorithm and a genetic algorithm. The current automaton based algorithm outperforms all the other schemes. We believe that it is the fastest algorithm reported to date. Additionally, our solution can also be adapted for the GPP in which the edge costs are not constant but random variables whose distributions are unknown.

Continuous learning automata solutions to the capacity assignment problem

The Capacity Assignment (CA) problem focuses on finding the best possible set of capacities for the links that satisfies the traffic requirements in a prioritized network while minimizing the cost. Most approaches consider a single class of packets flowing through the network, but, in reality, different classes of packets with different packet lengths and priorities are transmitted over the networks. In this paper, we assume that the traffic consists of different classes of packets with different average packet lengths and priorities. We shall look at three different solutions to this problem. K. Marayuma and D.T. Tang (1977) proposed a single algorithm composed of several elementary heuristic procedures. A. Levi and C. Ersoy (1994) introduced a simulated annealing approach that produced substantially better results. In this paper, we introduce a new method which uses continuous learning automata to solve the problem. Our new schemes produce superior results when compared with either of the previous solutions and is, to our knowledge, currently the best known solution.

Random Early Detection for Congestion Avoidance in Wired Networks: A Discretized Pursuit Learning-Automata-Like Solution

In this paper, we present a learning-automata-like (LAL) mechanism for congestion avoidance in wired networks. Our algorithm, named as LAL random early detection (LALRED), is founded on the principles of the operations of existing RED congestion-avoidance mechanisms, augmented with a LAL philosophy. The primary objective of LALRED is to optimize the value of the average size of the queue used for congestion avoidance and to consequently reduce the total loss of packets at the queue. We attempt to achieve this by stationing a LAL algorithm at the gateways and by discretizing the probabilities of the corresponding actions of the congestion-avoidance algorithm. At every time instant, the LAL scheme, in turn, chooses the action that possesses the maximal ratio between the number of times the chosen action is rewarded and the number of times that it has been chosen. In LALRED, we simultaneously increase the likelihood of the scheme converging to the action, which minimizes the number of packet drops at the gateway. Our approach helps to improve the performance of congestion avoidance by adaptively minimizing the queue-loss rate and the average queue size. Simulation results obtained using NS2 establish the improved performance of LALRED over the traditional RED methods which were chosen as the benchmarks for performance comparison purposes.

epsilon -optimal discretized linear reward-penalty learning automata

Variable-structure stochastic automata (VSSA) are considered which interact with an environment and which dynamically learn the optimal action that the environment offers. Like all VSSA the automata are fully defined by a set of action-probability updating rules. However, to minimize the requirements on the random-number generator used to implement the VSSA, and to increase the speed of convergence of the automation, the case in which the probability-updating functions can assume only a finite number of values. These values discretize the probability space (0, 1) and hence they are called discretized learning automata. The discretized automata are linear because the subintervals of (0, 1) are of equal length. The authors prove the following results: (a) two-action discretized linear reward-penalty automata are ergodic and epsilon -optimal in all environments whose minimum penalty probability is less than 0.5; (b) there exist discretized two-action linear reward-penalty automata that are ergodic and epsilon -optimal in all random environments, and (c) discretized two-action linear reward-penalty automata with artificially created absorbing barriers are epsilon -optimal in all random environments. >

Enhancing prototype reduction schemes with LVQ3-type algorithms

Abstract Various prototype reduction schemes have been reported in the literature. Foremost among these are the prototypes for nearest neighbor (PNN), the vector quantization (VQ), and the support vector machines (SVM) methods. In this paper, we shall show that these schemes can be enhanced by the introduction of a post-processing phase that is related, but not identical to, the LVQ3 process. Although the post-processing with LVQ3 has been reported for the SOM and the basic VQ methods, in this paper, we shall show that an analogous philosophy can be used in conjunction with the SVM and PNN rules. Our essential modification to LVQ3 first entails a partitioning of the respective training sets into two sets called the Placement set and the Optimizing set, which are instrumental in determining the LVQ3 parameters. Such a partitioning is novel to the literature. Our experimental results demonstrate that the proposed enhancement yields the best reported prototype condensation scheme to-date for both artificial data sets, and for samples involving real-life data sets.

The asymptotic optimality of discretized linear reward-inaction learning automata

The automata considered have a variable structure and hence are completely described by action probability updating functions. The action probabilities can take only a finite number of prespecified values. These values linearly increase and the interval [0, 1] is divided into a number of equal length subintervals. The probability is updated by the automata only if the environment responds with a reward and hence they are called discretized linear reward-inaction automata. The asymptotic optimality of this family of automata is proved for all environments.