Massimiliano Rosa

A Cellwise Block-Gauss-Seidel Iterative Method for Multigroup S N Transport on a Hybrid Parallel Computer Architecture

Abstract A Fourier analysis is conducted in two-dimensional (2-D) geometry for the discrete ordinates (SN) approximation of the neutron transport problem solved with Richardson iteration (source iteration) using the cellwise block-Jacobi (bJ) and block-Gauss-Seidel (bGS) algorithms. The results of the Fourier analysis show that convergence of bJ can degrade, leading to a spectral radius ρ equal to 1, in problems containing optically thin cells. For problems containing cells that are optically thick, instead, ρ tends to 0. Hence, in the optically-thick-cell regime, bJ is rapidly convergent even for scattering-dominated problems. Similar conclusions hold for bGS, except bGS approaches the asymptotic, thick-cell regime at convergence rates higher than bJ. Hence, we have implemented the bGS algorithm on the Roadrunner hybrid, parallel computer architecture. A compute node of this massively parallel machine comprises AMD Opteron cores that are linked to a Cell Broadband Engine (Cell/B.E.). LAPACK routines have been ported to the Cell/B.E. in order to make use of its parallel synergistic processing elements (SPEs). The bGS algorithm is based on the LU factorization and solution of a linear system that couples the fluxes for all SN angles and energy groups on a mesh cell. For every cell of a mesh that has been parallel decomposed on the higher-level Opteron processors, a linear system is transferred to the Cell/B.E. and the parallel LAPACK routines are used to compute a solution, which is then transferred back to the Opteron, where the rest of the SN transport computations take place. Compared to standard parallel machines, a one-hundred-fold speedup of the bGS was observed on Roadrunner. Numerical experiments with strong and weak parallel scaling demonstrate that the bGS method is viable and compares favorably to full parallel transport sweeps (FPS) on 2-D unstructured meshes when it is applied to optically thick, multimaterial problems. Specifically, the strong parallel efficiency of accelerated bGS on Roadrunner can achieve 73% at 512 processors, compared with 32 processors, while efficiency is 34% for the (Opteron-only) implementation of FPS. The weak parallel efficiency of bGS is 58% while it reaches 10% for FPS. As expected, however, bGS is not as efficient as FPS in optically thin problems.

Fourier Analysis of Inexact Parallel Block-Jacobi Splitting with Transport Synthetic Acceleration

Abstract A Fourier analysis is conducted for the discrete ordinates, or SN, approximation of the neutron transport problem solved with Richardson iteration (source iteration) and Richardson iteration preconditioned with transport synthetic acceleration (TSA), using the inexact parallel block-Jacobi (IPBJ) algorithm both in slab and two-dimensional Cartesian geometry. Both traditional, or “beta,” TSA (TTSA) and a modified TSA (MTSA), in which only the scattering in the low-order equations is reduced by some nonnegative factor β < 1, are considered. The results for the unaccelerated algorithm show that convergence of IPBJ can degrade, leading in particular to stagnation of the generalized minimum residual method with restart parameter m, GMRES(m), in problems containing optically thin subdomains. The IPBJ algorithm preconditioned with TTSA can be effective, provided the β parameter is properly tuned for a given scattering ratio c, but is potentially unstable. Compared to TTSA, MTSA is less sensitive to the choice of β, more effective for the same computational effort, measured in terms of the effective scattering ratio c′, and it is unconditionally stable.

Properties of the SN-Equivalent Integral Transport Operator in Slab Geometry and the Iterative Acceleration of Neutral Particle Transport Methods

Abstract General expressions for the matrix elements of the discrete SN-equivalent integral transport operator are derived in slab geometry. Their asymptotic behavior versus cell optical thickness is investigated both for a homogeneous slab and for a heterogeneous slab characterized by a periodic material discontinuity wherein each optically thick cell is surrounded by two optically thin cells in a repeating pattern. In the case of a homogeneous slab, the asymptotic analysis conducted in the thick-cell limit for a highly scattering medium shows that the discretized integral transport operator approaches a tridiagonal matrix possessing a diffusion-like coupling stencil. It is further shown that this structure is approached at a fast exponential rate with increasing cell thickness when the arbitrarily high order transport method of the nodal type and zero-order spatial approximation (AHOT-N0) formalism is employed to effect the spatial discretization of the discrete ordinates transport operator. In the case of periodically heterogeneous slab configurations, the asymptotic behavior is realized by pushing apart the cells’ optical thicknesses; i.e., the thick cells are made thicker while the thin cells are made thinner at a prescribed rate. We show that in this limit the discretized integral transport operator is approximated by a pentadiagonal structure. Notwithstanding, the discrete operator is amenable to algebraic transformations leading to a matrix representation still asymptotically approaching a tridiagonal structure at a fast exponential rate bearing close resemblance to the diffusive operator. The results of the asymptotic analysis of the integral transport matrix are then used to gain insight into the excellent convergence properties of the adjacent-cell preconditioner (AP) acceleration scheme. Specifically, the AP operator exactly captures the asymptotic structure acquired by the integral transport matrix in the thick-cell limit for homogeneous slabs of pure-scatterer or partial-scatterer material, and for periodically heterogeneous slabs hosting purely scattering materials. In the above limits the integral transport matrix reduces to a diffusive structure consistent with the diffusive matrix template used to construct the AP. In the case of periodically heterogeneous slabs containing absorbing materials, the AP operator partially captures the asymptotic structure acquired by the integral transport matrix. The inexact agreement is due either to discrepancies in the equations for the boundary cells or to the nondiffusive structure acquired by the integral transport matrix. These findings shed light on the immediate convergence, i.e., convergence in two iterations, displayed by the AP acceleration scheme in the asymptotic limit for slabs hosting purely scattering materials, both in the homogeneous and periodically heterogeneous cases. For periodically heterogeneous slabs containing absorbing materials, immediate convergence is achieved by modifying the original recipe for constructing the AP so that the correct asymptotic structure of the integral transport matrix coincides with the AP operator in the asymptotic limit.

On the Degradation of Cell-Centered Diffusive Preconditioners for Accelerating SN Transport Calculations in the Periodic Horizontal Interface Configuration

Abstract We investigate the degraded effectiveness of diffusion-based acceleration schemes in terms of the adjacent-cell preconditioner (AP) in a periodically heterogeneous limit devised for the two-dimensional (2-D) periodic horizontal interface (PHI) configuration. Specifically, we demonstrate that the diffusive low-order operator employed in the AP scheme lacks the structure of the integral transport operator in the above asymptotic limit since it (1) ignores cross-derivative coupling and (2) incorrectly estimates the strength of intra-layer coupling in the optically thin layers. In order to prove propositions 1 and 2, we derive expressions for the elements of the matrix representing a certain angular (SN) and spatially discretized form of the 2-D neutron transport integral operator. This is the transport operator that produces the full scalar flux solution if it is directly inverted on the once-collided particle source. The properties of this operator’s elements are then investigated in the asymptotic limit for PHI. The results of the asymptotic analysis point to a sparse but nonlocal matrix structure due to long-range coupling of a cell’s average flux with its neighboring cells, independent of the distance between the cells in the spatial mesh. In particular, for a cell in a thin layer, cross-derivative coupling of the cell’s flux to its diagonal neighbors is of the same asymptotic order as self-coupling and coupling with its north/south Cartesian neighbors. Similarly, its coupling with the fluxes in the same thin layer is of the same order, independent of the distance between the cells in the layer, as the coupling with the east/west Cartesian neighbors. We also show that modifying the standard diffusion-based AP can lead to effective acceleration in PHI. Specifically, we devise three novel acceleration schemes, named APB, Optimized-AP (OAP), and Hybrid-AP (HAP), obtained by modifying the original AP formalism in 2-D. In the APB the five-point AP operator is extended to a nine-point stencil that accounts for cross-derivative coupling by including the matrix elements of the integral transport operator B, which couple a cell-averaged scalar flux to its first diagonal neighbors. In the OAP the five-point stencil of the original AP operator is retained while optimizing the value of the elements in the preconditioner that affect the coupling of a cell with its east/west Cartesian neighbors. Specifically, the optimum elements are obtained by minimizing the iteration’s spectral radius and offer a more correct estimate of the strength of intra-layer coupling in a thin layer. Finally, the nine-point HAP operator represents a “hybrid” of the APB and OAP approaches, in the sense that the spectral properties of the optimized five-point OAP are further improved via the inclusion of cross-derivative terms. Fourier analysis of the novel acceleration schemes indicates that robustness of the accelerated iterations can be recovered, in spite of sharp material discontinuities, by accounting for cross-derivative coupling and by optimizing the preconditioner elements. The new acceleration schemes have also been implemented in a 2-D transport code, and numerical tests successfully verify the predictions of the Fourier analysis. However, it is important to emphasize that the modifications attempted in this work are specific to the selected asymptotic limit for PHI and do not translate into new low-order operators for the general heterogeneous-material case. Rather, the above modified operators suggest that it may be possible to eventually derive such a general low-order unconditionally robust operator.