Torsten Suel

Efficient communication using total-exchange

A central question in parallel computing is to determine the extent to which one can write parallel programs using a high-level, general-purpose, and architecture-independent programming language and have them executed on a variety of parallel and distributed architectures without sacrificing efficiency. A large body of research suggests that, at least in theory, general-purpose parallel computing is indeed possible provided certain conditions are met: an excess of logical parallelism in the program, and the ability of the target architecture to efficiently realize balanced communication patterns. The canonical example of a balanced communication pattern is an h-relation, in which each processor is the origin and destination of at most h messages. A plethora of protocols has been designed for routing h-relations in a variety of networks. The goal has been to minimize the value of h while guaranteeing delivery of the messages within a time constant factor from optimal. In this paper we describe protocols that meet the most stringent efficiency requirement, namely delivery of messages within time that is a lower order additive term from the best achievable. Such protocols are called 1-optimal. While these protocols achieve 1-optimality only for heavily loaded networks, that is, for large values of h, they are remarkable for their simplicity in that they only use the total-exchange communication primitive. The total-exchange can be realized in many networks using very simple, contention-free, and extremely efficient schemes. The technical contribution of this paper is a protocol to route random h-relations in an N-processor network using /sup h///sub N/(1+o(1))+O(log log N) total-exchange rounds with high probability. Using message duplication, we can improve the bound to /sup h///sub N/(1+o(1))+O(log*N). This improves upon the /sup h///sub N/(1+o(1))+O(log N) bound of Gerbessiotis and Valiant. While our theoretical improvements are modest, our experimental results show an improvement over the protocol of A. Gerebessiotis and L.G. Valiant.<<ETX>>

Routing and sorting on meshes with row and column buses

Gives improved deterministic algorithms for permutation routing and sorting on meshes with row and column buses. Among our results, we obtain a fairly simple algorithm for permutation routing on two-dimensional meshes with buses that achieves a running time of n+o(n) and a queue size of 2. We also describe an algorithm for routing on r-dimensional networks with a running time of (2/spl minus/1/r)n+o(n) and a queue size of 2, and show how to obtain deterministic algorithms for sorting whose running times match those for permutation routing. An interesting feature of our algorithms is that they can be implemented on a wide variety of different models of meshes with buses within the same bounds on time and queue size. Finally, we also study the performance of meshes with buses on dynamic routing problems, and propose fast routing schemes under several different assumptions about the properties of the bus system.<<ETX>>

On probabilistic networks for selection, merging, and sorting

We study comparator networks for selection, merging, and sorting that output the correct result with high probability, given a random input permutation. We prove tight bounds, up to constant factors, on the size and depth of probabilistic (n,k)-selection networks. In the case of (n, n/2)-selection, our result gives a somewhat surprising bound of \(\Theta(n \log \log n)\) on the size of networks of success probability in \([\delta, 1-1/\mbox{poly}(n)]\) , where δ is an arbitrarily small positive constant, thus comparing favorably with the best previously known solutions, which have size \(\Theta(n\log n)\) . We also prove tight bounds, up to lower-order terms, on the size and depth of probabilistic merging networks of success probability in \([\delta, 1-1/\mbox{poly}(n)]\) , where δ is an arbitrarily small positive constant. Finally, we describe two fairly simple probabilistic sorting networks of success probability at least \(1-1/\mbox{poly}(n)\) and nearly logarithmic depth.

Lower Bounds for Shellsort

We show lower bounds on the worst-case complexity of Shellsort. In particular, we give a fairly simple proof of an ?(n(lg2n)/(lglgn)2) lower bound for the size of Shellsort sorting networks for arbitrary increment sequences. We also show an identical lower bound for the running time of Shellsort algorithms, again for arbitrary increment sequences. Our lower bounds establish an almost tight trade-off between the running time of a Shellsort algorithm and the length of the underlying increment sequence.

Lower bounds for sorting networks

We establish a lower bound of (1.12 – o(l)) n log n on the size of any n-input sorting network; this is the first lower bound that improves upon the trivial information-theoretic bound by more than a lower order term. We then extend the lower bound to comparator networks that approximately sort a certain fraction of all input permutations. We also prove a lower bound of (c – o(l)) log n, where c N 3.27, on the depth of any sorting network; the best previous result of approximately (2.41 – O(1)) log n was established by Yao in 1980. Our result for size is based on a new technique that lower bounds the number of “O-1 collisions” in the network; we provide strong evidence that the technique will lead to even better lower bounds. 1Part of this work was done while the author was at DIMACS. 2XEROX Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CA 943o4. Partially supported by the NSF under grant CCR-9404113. Email: kahale@parc. xerox. corn. 3Department of Mathematics and Laboratory for Computer Science, MIT, Cambridge, MA 02139. Supported by ARPA Contracts NOO014-91-J-1698 and NOO014-92-J-1799. Email: ftl@math. mit. edu. 4Department of Computer Science, Stanford University, Stanford, CA 94305. Supported by an NSF Mathematical Sciences Postdoctoral Research Fellowship. Part of this work was done while the author was at MIT and supported by ARPA Contracts NOO014-91-J-1698 and NOO014-92-J-1799. Email: yuan@cs. stanford. edu. 5Department of Computer Science, University of Texas at Austin, Austin, TX 78712. Supported by the Texas Advanced Research Program under Grant No. ARP-93-O03658461. Email: plaxton@cs .utexas. edu. 6NEC Research Institute, 4 Independence Way, Princeton, NJ 08540. This work was done while the author was at the University of Texas at Austin, and was supported by the Texas Advanced Research Program under Grant No. ARP-93-O03658461. Email: torsten@research .nj .nec. corn. TDePartment of Computer Science, Rutgers University! ‘iscataway, NJ 08855. Supported by ARPA under contract DABT63-93-C-0064. Part of this work was done at the University of Paderborn, Germany. Email: szemered@cs. rutgers. edu. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct mmmercial advantage, the ACM copyright notice and the title of the publication and IS date appear, and notice is given that copyin is by permission of the Association of Computing Machinery. o cop otherwise or to republish, requires a fee andh!spacilc perrnissiAn. STOC’ 95, Las V as, Nevada, USA ?’ 01995 ACM O-89 91-718-9/95/0005..

A lower bound for sorting networks based on the shuffle permutation

3 .50

Improved bounds for routing and sorting on multi-dimensional meshes

We prove an omega (lg2 n/lg lg n) lower bound for the depth of sorting networks based on the shuffle permutation. This settles an open question posed by Knuth, up to a theta (lg lg n) factor. The proof technique employed in the lower bound argument may be of separate interest.

A Super-Logarithmic Lower Bound for Hypercubic Sorting Networks

We show improved bounds for 1–1 routing and sorting on multi-dimensional meshes and tori. In particular, we give a fairly simple deterministic algorithm for sorting on the <italic>d</italic>-dimensional mesh of side length <italic>n</italic> that achieves a running time of <italic>3dn</italic>/2+<italic>o</italic>(<italic>n</italic>) for the <italic>d</italic>-dimensional mesh and torus, respectively, that make one copy of each element. We also show lower bounds for sorting with respect to a large class of indexing schemes, under a model of the mesh where each processor can hold an arbitrary number of packets. Finally, we describe algorithms for permutation routing whose running times come very close to the diameter lower bound.

Beyond the Worst-Case Bisection Bound: Fast Sorting and Ranking on Meshes

Hypercubic sorting networks are a class of comparator networks whose structure maps efficiently to the hypercube and any of its bounded degree variants. Recently, n-input hypercubic sorting networks with nearly logarithmic depth have been discovered. These networks are the only known sorting networks of depth o(lg^2 n) that are not based on expanders, and their existence raises the question of whether a depth of O(lg n) can be achieved by any hypercubic sorting network. In this paper, we resolve this question by establishing a super-logarithmic lower bound on the depth of any n-input hypercubic sorting network. Our lower bound can be extended to certain restricted classes of non-oblivious sorting algorithms on hypercubic machines.

Derandomizing algorithms for routing and sorting on meshes

Sorting is an important subroutine in many parallel algorithms and has been studied extensively on meshes and related networks. If every processor of an n×n mesh is the source and destination of at most k elements, then sorting requires at least k · n/2 steps in the worst-case, and simple algorithms have recently been proposed that nearly match this bound. However, this lower bound does not extend to non-worst-case inputs, or weaker definitions of sorting that are sufficient in many applications. In this paper, we give algorithms and lower bounds for several such problems.