Kiichi Urahama

K-winners-take-all circuit with O(N) complexity

Presents a k-winners-take-all circuit that is an extension of the winner-take-all circuit by Lazzaro et al. (1989). The problem of selecting the largest k numbers is formulated as a mathematical programming problem whose solution scheme, based on the Lagrange multiplier method, is directly implemented on an analog circuit. The wire length in this circuit grows only linearly with the number of elements, and the circuit is more suitable for real-time processing than the Hopfield networks because the present circuit produces the solution almost instantaneously-in contrast to the Hopfield network, which requires transient convergence to the solution from a precise initial state. The selection resolution in the present circuit is, however, only finite in contrast to the almost infinite resolution in the Hopfield networks.

Gradient descent learning of nearest neighbor classifiers with outlier rejection

Abstract The nearest neighbor classification rule is extended to reject outlier data and is implemented with an analog electronic circuit. A continuous membership function is derived from an optimization formulation of the classification rule. A learning algorithm is then presented for arranging prototype patterns to their optimal places and adjusting the radius of outlier rejection. The place of prototypes and the rejection radius are incrementally updated at every presentation of training patterns in the steepest descent direction of the error of the membership of the presented pattern from its correct value. Some elementary experiments examplify the convergence of the present learning algorithm.

Analog circuit for solving assignment problems

A novel analog electronic circuit for solving assignment problems is presented. Total length of wiring in the proposed circuit amounts to at most O(n/sup 2/) with n being the number of variables in contrast to O(n/sup 4/) required for previously developed circuits based on the Hopfield neural networks. Moreover, its power dissipation is extremely small by virtue of subthreshold operation of MOS transistors. >

Analog circuit implementation and learning algorithm for nearest neighbor classifiers

Abstract Analog electronic circuits are implemented and learning algorithms are presented for Nearest Neighbor (NN) and f -NN. classifiers on the basis of the probabilistic formulation of these classifiers. Electronic networks are compact subthreshold MOS transistor circuits. In the learning algorithm, the place of prototypes and the variance of the probability distribution are optimized by using the steepest descent method for the Kullback-Leiblers information between the network output and the correct membership.

A gradient system solution to Potts mean field equations and its electronic implementation.

A gradient system solution method is presented for solving Potts mean field equations for combinatorial optimization problems subject to winner-take-all constraints. In the proposed solution method the optimum solution is searched by using gradient descent differential equations whose trajectory is confined within the feasible solution space of optimization problems. This gradient system is proven theoretically to always produce a legal local optimum solution of combinatorial optimization problems. An elementary analog electronic circuit implementing the presented method is designed on the basis of current-mode subthreshold MOS technologies. The core constituent of the circuit is the winner-take-all circuit developed by Lazzaro et al. Correct functioning of the presented circuit is exemplified with simulations of the circuits implementing the scheme for solving the shortest path problems.

Equivalence between some dynamical systems for optimization

It is shown by the derivation of solution methods for an elementary optimization problem that the stochastic relaxation in image analysis, the Potts neural networks for combinatorial optimization and interior point methods for nonlinear programming have common formulation of their dynamics. This unification of these algorithms leads us to possibility for real time solution of these problems with common analog electronic circuits.

Neural algorithms for placement problems

Two improved neural algorithms are presented for solving a placement problem which is a familiar class of NP-hard quadratic assignment problems. Formulation of the problem as a zero-one integer programming leads to an improved form of the Hopfield networks, while a mixed integer programming formulation results in an analogue algorithm similar to the elastic nets. The outermost loop in these algorithms performs an automatically scheduled deterministic annealing. This gives us a natural interpretation of the annealing procedure derived directly from the mathematical programming framework. Experiments reveal that the adaptive elastic net algorithm outperforms the adaptive Hopfield method.

CONSTRAINED POTTS MEAN FIELD SYSTEMS AND THEIR ELECTRONIC IMPLEMENTATION

The Potts mean field approach for solving combinatorial optimization problems subject to winner-takes-all constraints is extended for problems subject to additional constraints. Extra variables corresponding to the Lagrange multipliers are incorporated into the Potts formulation for the additional constraints to be satisfied. The extended Potts equations are solved by using constrained gradient descent differential systems. This gradient system is proven theoretically to always produce a legal local optimum solution of the constrained combinatorial optimization problems. An analog electronic circuit implementing the present method is designed on the basis of the previous Potts electronic circuit. The performance of the present method is theoretically evaluated for the constrained maximum cut problems. The lower bound of the cut size obtained with the present method is proven to be the same as that of the basic Potts scheme for the unconstrained maximum cut problems.

Performance evaluation of Hopfield network for simple examples

The performance guarantees of the Hopfield network are given for two simple graph problems. A lower bound of cutsize is evaluated for the maximum cut problem through the analysis of eigenvalues at equilibrium states. The condition of constraint satisfaction and an upper bound of the cutsize are given for the graph bipartitioning problem. In addition, an effective numerical scheme is proposed to integrate the differential equations of the Hopfield network by using the backward Euler formula with one-step Gauss-Seidel relaxation. Theoretical estimates of the performance of the algorithm are verified experimentally.<<ETX>>

PERFORMANCE OF THE RELAXATION ALGORITHM FOR MAXIMUM-CUT PROBLEMS

A relaxation algorithm is presented for solving a class of combinatorial optimization problems called set-partitioning tasks. The convergence property of the presented algorithm is investigated theoretically. A performance guarantee is derived theoretically for the present algorithm applied to an NP-hard example problem called the maximum-cut graph partitioning. The experimental examination of its performance manifests its superiority in computational speed to the conventional gradient method.