Minesh B. Amin

A scalable parallel formulation of the backpropagation algorithm for hypercubes and related architectures

We present a new technique for mapping the backpropagation algorithm on hypercube and related architectures. A key component of this technique is a network partitioning scheme called checkerboarding. Checkerboarding allows us to replace the all-to-all broadcast operation performed by the commonly used vertical network partitioning scheme, with operations that are much faster on the hypercubes and related architectures. Checkerboarding can be combined with the pattern partitioning technique to form a hybrid scheme that performs better than either one of these schemes. Theoretical analysis and experimental results on nCUBE and CM5 show that our scheme performs better than the other schemes, for both uniform and nonuniform networks. >

Generalization by neural networks

The authors discuss the requirements of learning for generalization, where the traditional methods based on gradient descent have limited success. A stochastic learning algorithm based on simulated annealing in weight space is presented. The authors verify the convergence properties and feasibility of the algorithm. An implementation of the algorithm and validation experiments are described. >

ZAMBEZI: a parallel pattern parallel fault sequential circuit fault simulator

Sequential circuit fault simulators use the multiple bits in a computer data word to accelerate simulation. We introduce, and implement, a new sequential circuit fault simulator, a parallel pattern parallel fault simulator, ZAMBEZI, which simultaneously simulates multiple faults with multiple vectors in one data word. ZAMBEZI is developed by enhancing the control flow, of existing parallel pattern algorithms. For a very wide range of benchmark circuits, compared to parallel fault and parallel pattern simulators, ZAMBEZI offers either the best, or very close to the best, uniprocessor performance. ZAMBEZI also offers superior performance when parallelized.

The C31 parallel benchmark suite - introduction and preliminary results

Current parallel benchmarks, while appropriate for scientific applications, lack the defense relevance and representativeness for developers who are considering parallel computers for their Command, Control, Communication, and Intelligence (C3I) systems. We present a new set of compact application benchmarks which are specific to the C3I application domain. The C3I Parallel Benchmark Suite (C3IPBS) program is addressing the evaluation of not only machine performance, but also the software implementation effort. Our methodology currently draws heavily from the PARKBENCH[2] and NAS Parallel Benchmarks[1]. The paper presents the benchmarking methodology, introduces the benchmarks, and reports initial results and analysis. Finally, we describe the lessons that we have learned so far from formulating and implementing the C3I benchmarks.

Workload Distribution in Fault Simulation

Simulation at the gate level is computationally very expensive.Parallel processing is one technique to reduce simulation time.Possessing knowledge of the distribution of computational activity insimulation can aid in parallelizing it efficiently. We present a newcharacterization of the distribution of the computational workload infault simulation. An empirical analysis shows that the workloaddistribution is circuit specific, and is largely independent of thevector set being simulated. An inexpensive method to predict theworkload distribution is also discussed.

Customizing parallel formulations of backpropagation learning algorithm to neural network architectures: a summary of result

Several generic parallel formulations of the backpropagation learning algorithm have been proposed recently. Further speedups are possible by customizing parallel formulations to the architecture of the neural network. The paper addresses the issue of customizing parallel formulations of the backpropagation learning algorithm to a given neural network architecture on multiprocessors with hypercube-like communication topology. We introduce a new parallel formulation called rectangular checkerboarding which adapts to the network architecture and can provide performance gains for non-uniform neural networks, where the number of nodes vary across the layers. Algebraic analysis shows that each instance of rectangular checkerboarding (using a specific rectangular processor grid) is optimal for an important family of network architectures. Experiments on CM-5 show that customizing to network architecture can provide significant (/spl sim/50%) performance gains for many interesting non-uniform neural network architectures, which are currently used in important applications. We also introduce the staircase framework, which can use different processor grids for different layers of a neural network.<<ETX>>

Data parallel fault simulation

Fault simulation is a compute intensive problem. Data parallel simulation on multiple processors is one method to reduce fault simulation time. We discuss a novel technique to partition the fault set for data parallel fault simulation. When applied statically, the technique can scale well for up to eight processors. The fault set partitioning technique is simple, can itself be parallelized, and can be implemented with extreme ease. Therefore, the technique can be used on a low cost parallel resource, such as a network of workstations.

Zamlog : a parallel algorithm for fault simulation based on Zambezi

We present a new multiprocessor sequential circuit fault simulator, Zamlog, based on a novel uniprocessor simulator, Zambezi. Both the fault and test sets are partitioned for multiprocessor simulation. The parallelization technique, designed to preserve the efficiency of Zambezi, is simple to implement and has low communication requirements. Experimental results indicate that Zamlog can obtain speedups of up to 95. The speedups obtained and the scalability are between 3 and 10 times better than any reported in the literature. Furthermore, the speed-ups obtained are with respect to a uniprocessor algorithm which is superior, by an average of 40%, to those used to gauge the speed-ups of previous parallel systems.

Generalization Performance of Feed-Forward Neural Networks

Abstract The success of neural networks with recognition problems has opened the door for more ambitious applications such as generalization problems, where a network is expected to correctly predict output for inputs previously unseen during learning. Learning algorithms need to avoid underfitting and overfitting to successfully generalize. We evaluate the ability of backpropagation and stochastic backpropagation learning algorithms to generalize via a set of controlled experiments. We describe a tool, GNET, which facilitates the design and execution of these experiments. Backpropagation and stochastic backpropagation learning algorithms with smaller networks are able to generalize to linear, quadratic and logarithmic functions. However, these learning algorithms have difficulty in learning non-monotonic functions. We demonstrate the effect of noise on the learning ability of these algorithms. The experiments also characterize the overfitting phenomenon associated with generalization in analyzing learning ability.

Data parallel sequential circuit fault simulation

Sequential circuit fault simulation is a compute-intensive problem. Parallel simulation is one method to reduce fault simulation time. In this paper, we discuss a novel technique to partition the fault set for the fault parallel simulation of sequential circuits on multiple processors. When applied statically, the technique can scale well for up to thirty two processors on an ethernet. The fault set partitioning technique is simple and can itself be parallelized. Processors working in parallel require no communication with one another. An existing uniprocessor algorithm can be used for parallel simulation without modification. Therefore, this system can be used effectively on a low-cost parallel resource, such as a network of workstations.