Gabriel Oksa

PARALLEL SOLUTION OF SYMMETRIC POSITIVE DEFINITE TOEPLITZ SYSTEMS

Initially, parallel algorithms were designed by parallelising the existing sequential algorithms for frequently occurring problems on available parallel architectures. More recently, parallel strategies have been identified and utilised resulting in many new parallel algorithms. However, the analysis of such techniques reveals that further strategies can be applied to increase the parallelism. One of these strategies, i.e., increasing the computational work in each processing node, can reduce the memory accesses and hence congestion in a shared memory multiprocessor system. Similarly, when network message passing is minimised in a distributed memory processor system, dramatic improvements in the performance of the algorithm ensue. A frequently occurring computational problem in digital signal processing (DSP) is the solution of symmetric positive definite Toeplilz linear systems. The Levinson algorithm for solving such linear equations is where the Toeplitz matrix property is utilised in the elimination p...

New Dynamic Orderings for the Parallel One–Sided Block-Jacobi SVD Algorithm

Five variants of a new dynamic ordering are presented for the parallel one-sided block Jacobi SVD algorithm. Similarly to the two-sided algorithm, the dynamic ordering takes into account the actual status of a matrix—this time of its block columns with respect to their mutual orthogonality. Variants differ in the computational and communication complexities and in proposed global and local stopping criteria. Their performance is tested on a square random matrix of order 8192 with a random distribution of singular values using p=16, 32, 64, 96 and 128 processors. All variants of dynamic ordering are compared with a parallel cyclic ordering, two-sided block-Jacobi method with dynamic ordering and the ScaLAPACK routine PDGESVD with respect to the number of parallel iteration steps needed for the convergence and total parallel execution time. Moreover, the relative errors in the orthogonality of computed left singular vectors and in the matrix assembled from computed singular triplets are also discussed. It turns out that the variant 3, for which a local optimality in some precisely defined sense can be proved, and its combination with variant 2, are the most efficient ones. For relatively small blocking factors l=2p, they outperform the ScaLAPACK procedure PDGESVD and are about 2 times faster.

Special Issue: A systolic block-Jacobi SVD solver for processor meshes

We design the systolic version of the two-sided block-Jacobi algorithm for the singular value decomposition (SVD) of matrix A∈R m×n , and m, n even. The algorithm involves the class CO of parallel orderings on the two-dimensional toroidal mesh with p processors. The mathematical background is based on the QR decomposition (QRD) of local data matrices and on the triangular Kogbetliantz algorithm (TKA) for local SVDs in the diagonal mesh processors. Subsequent updates of local matrices in the diagonal as well as nondiagonal mesh processors are required. We show that all updates can be realized by orthogonal modified Givens rotations. These rotations can be efficiently pipelined in parallel in the horizontal and vertical rings of processor through the toroidal mesh. Our solution requires, per one mesh processor, systolic processing elements (PEs) and additional delay elements. The time complexity can be estimated as where w is the number of global sweeps in the two-sided block-Jacobi algorithm and Δ is the length of the global synchronization time step. The VLSI area per mesh processor, measured by the number of vertical and horizontal wires required for its construction, can be estimated as and the combined VLSI area–time complexity per mesh processor is The theoretical speedup can be estimated as Using the mesh processors of fixed inner size , even, it is possible to construct the square two-dimensional toroidal mesh and to compute the SVD of matrix A, the size of the which matches the shape of mesh processors, i.e. In this sense, the systolic algorithm is scalable.

PRECONDITIONED PARALLEL BLOCK–JACOBI SVD ALGORITHM

We show experimentally, that the QR factorization with the complete column pivoting, optionally followed by the LQ factorization of the R-factor, can lead to a substantial decrease of the number of outer parallel iteration steps in the parallel block-Jacobi SVD algorithm, whereby the details depend on the condition number and on the shape of spectrum, including the multiplicity of singular values. Best results were achieved for well-conditioned matrices with a multiple minimal singular value, where the number of parallel iteration steps has been reduced by two orders of magnitude. However, the gain in speed, as measured by the total parallel execution time, depends decisively on how efficient is the implementation of the distributed QR and LQ factorizations on a given parallel architecture. In general, the reduction of the total parallel execution time up to one order of magnitude has been achieved.

Parallel SVD Computing in the Latent Semantic Indexing Applications for Data Retrieval

One of the main sources of information in our society is a written word. Since times of Sumerians, a written document became the main tool to inform, to teach, to entertain and to archive the knowledge. Today, some 6000 years after Sumerians, nothing has changed with respect to the importance of a written text. To become widely available, the knowledge must be manipulated in an easy and reliable way, and some type of text encoding on a computer is needed