Stelios Timotheou

Random neural networks with synchronized interactions

Large-scale distributed systems, such as natural neuronal and artificial systems, have many local interconnections, but they often also have the ability to propagate information very fast over relatively large distances. Mechanisms that enable such behavior include very long physical signaling paths and possibly saccades of synchronous behavior that may propagate across a network. This letter studies the modeling of such behaviors in neuronal networks and develops a related learning algorithm. This is done in the context of the random neural network (RNN), a probabilistic model with a well-developed mathematical theory, which was inspired by the apparently stochastic spiking behavior of certain natural neuronal systems. Thus, we develop an extension of the RNN to the case when synchronous interactions can occur, leading to synchronous firing by large ensembles of cells. We also present an O(N3) gradient descent learning algorithm for an N-cell recurrent network having both conventional excitatory-inhibitory interactions and synchronous interactions. Finally, the model and its learning algorithm are applied to a resource allocation problem that is NP-hard and requires fast approximate decisions.

The Random Neural Network

The random neural network (RNN) is a recurrent neural network model inspired by the spiking behaviour of biological neuronal networks. Contrary to most artificial neural network models, neurons in the RNN interact by probabilistically exchanging excitatory and inhibitory spiking signals. The model is described by analytical equations, has a low complexity supervised learning algorithm and is a universal approximator for bounded continuous functions. The RNN has been applied in a variety of areas including pattern recognition, classification, image processing, combinatorial optimization and communication systems. It has also inspired research activity in modelling interacting entities in various systems such as queueing and gene regulatory networks. This paper presents a review of the theory, extension models, learning algorithms and applications of the RNN.

Emergency response simulation using wireless sensor networks

During emergency response situations, decisions have to be made in a timely manner. Multiple entities have to be optimally coordinated and numerous resources must be allocated efficiently, creating a very interesting and challenging technical problem. In this paper we present a simulation system that models the evacuation of a multi-storey building. Autonomous intelligent agents are used to represent various types of actors that interact inside a virtual physical world. We also model virtual hazards, such as fire, that spread inside the building evacuation simulator. A real wireless sensor network is used to monitor the spread of the hazards while an external event generator provides input to the sensors. We study the effect of different disaster scenarios and agent behaviours, such as human behaviour during an emergency, on the result of the evacuation procedure. Our initial results indicate that the safety of the evacuees and the evacuation time depend on local interactions between the participants and are affected by the actors decisions. The integration with the wireless sensor network gives us the opportunity to investigate the effect of sensed information on resource allocation and allows us to study the impact of network issues on the decision making process.

Autonomous networked robots for the establishment of wireless communication in uncertain emergency response scenarios

During a disaster, emergency response operations can benefit from the establishment of a wireless ad hoc network. We propose the use of autonomous robots that move inside a disaster area and establish a network for two-way communication between trapped civilians with uncertain locations and an operation centre. Our aim is to maximise the number of civilians connected to the network. We present a distributed algorithm which involves clustering possible locations of civilians according to their expected shortfall; clustering facilitates both connectivity within groups of civilians and exploration that is based on the uncertainty of these locations. To achieve efficient allocation in terms of time and energy, we also develop a modified algorithm according to which the robots consider the graph that the cluster centres form and follow its minimum spanning tree. We conduct simulations and discuss the efficiency and appropriateness of the two algorithms in different situations.

Fast Distributed Near-Optimum Assignment of Assets to Tasks

We investigate the assignment of assets to tasks where each asset can potentially execute any of the tasks, but assets execute tasks with a probabilistic outcome of success. There is a cost associated with each possible assignment of an asset to a task, and if a task is not executed, there is also a cost associated with the non-execution of the task. Thus, any assignment of assets to tasks will result in an expected overall cost which we wish to minimize. We formulate the allocation of assets to tasks in order to minimize this expected cost, as a nonlinear combinatorial optimization problem. A neural network approach for its approximate solution is proposed based on selecting parameters of a random neural network (RNN), solving the network in equilibrium, and then identifying the assignment by selecting the neurons whose probability of being active is the highest. Evaluations of the proposed approach are conducted by comparison with the optimum (enumerative) solution as well as with a greedy approach over a large number of randomly generated test cases. The evaluation indicates that the proposed RNN-based algorithm is better in terms of performance than the greedy heuristic, consistently achieving on average results within 5% of the cost obtained by the optimal solution for all problem cases considered. The RNN-based approach is fast and is of low polynomial complexity in the size of the problem, while it can be used for decentralized decision making.

Synchronized Interactions in Spiked Neuronal Networks

The study of artificial neural networks has originally been inspired by neurophysiology and cognitive science. It has resulted in a rich and diverse methodology and in numerous applications to machine intelligence, computer vision, pattern recognition and other applications. The random neural network (RNN) is a probabilistic model which was inspired by the spiking behaviour of neurons, and which has an elegant mathematical treatment that provides both its steady-state behaviour and offers efficient learning algorithms for recurrent networks. Second-order interactions, where more than one neuron jointly act upon other cells, have been observed in nature; they generalize the binary (excitatory–inhibitory) interaction between pairs of cells and give rise to synchronous firing (SF) by many cells. In this paper, we develop an extension of the RNN to the case of synchronous interactions, which are based on two cells that jointly excite a third cell; this local behaviour is in fact sufficient to create SF by large ensembles of cells. We describe the system state and derive its stationary solution as well as a O(N3) gradient descent learning algorithm for a recurrent network with N cells when both standard excitatory–inhibitory interactions, as well as SF, are present.

Robotic wireless network connection of civilians for emergency response operations

Mobile robots equipped with wireless devices can prove very useful during emergency response operations. We envision such robots that locate trapped civilians and initiate an ad hoc network connection between them and the rescuers, so that the latter can better assess the situation and plan the rescue operation accordingly. We present a centralised formulation for the novel problem of optimally allocating robots so that they connect as many civilians as possible, while maintaining their multi-hop connection with a static wireless sink. This formulation stems from a combination of characteristics typically found in assignment and network flow optimisation problems. We have also developed a distributed heuristic with which the robots start from the location of the sink and move autonomously trying to connect the civilians while maintaining connectivity. We evaluate our distributed heuristic using a building evacuation simulator and compare it with the centralised approach.

A novel weight initialization method for the random neural network

In this paper, we propose a novel weight initialization method for the random neural network. The method relies on approximating the signal-flow equations of the network to obtain a linear system of equations with nonnegativity constraints. For the solution of the formulated linear nonnegative least squares problem we have developed a projected gradient algorithm. It is shown that supervised learning with the developed initialization method has better performance in terms of both solution quality and execution time than learning with random initialization when applied to a combinatorial optimization emergency response problem.

A random neural network approach to an assets to tasks assignment problem

We investigate the assignment of assets to tasks where each asset can potentially execute any of the tasks, but assets execute tasks with a probabilistic outcome of success. There is a cost associated with each possible assignment of an asset to a task, and if a task is not executed there is also a cost associated with the nonexecution of the task. Thus any assignment of assets to tasks will result in an expected overall cost which we wish to minimise. We propose an approach based on the Random Neural Network (RNN) which is fast and of low polynomial complexity. The evaluation indicates that the proposed RNN approach comes at most within 10% of the cost obtained by the optimal solution in all cases.

Nonnegative Least Squares Learning for the Random Neural Network

In this paper, a novel supervised batch learning algorithm for the Random Neural Network (RNN) is proposed. The RNN equations associated with training are purposively approximated to obtain a linear Nonnegative Least Squares (NNLS) problem that is strictly convex and can be solved to optimality. Following a review of selected algorithms, a simple and efficient approach is employed after being identified to be able to deal with large scale NNLS problems. The proposed algorithm is applied to a combinatorial optimization problem emerging in disaster management, and is shown to have better performance than the standard gradient descent algorithm for the RNN.