Hybrid MPI-OpenMP programming for parallel OSEM PET reconstruction

To improve the parallel efficiency (PE) of the ordered-subsets expectation-maximization (OSEM) algorithm for three-dimensional (3-D) positron emission tomography (PET) image reconstruction, we focused on reducing the computational imbalance among parallel processes and interprocess data exchange time which were the dominant limiting factors of PE when a large number of networked compute nodes were used. As clusters with multiple processors on each compute node have become increasingly common, we have aimed to take advantage of the load-balancing mechanism and the inherently lower latency of shared memory threads across processors within a single node. We, therefore, implemented the OSEM algorithm with a hybrid message passing interface (MPI) and OpenMP approach on the basis of a standard MPI implementation. The contributing components to the total reconstruction time for the hybrid technique were quantified and compared to that using only MPI. The hybrid MPI-OpenMP technique achieved a consistent PE improvement of approximately 7% to 17% compared to the pure MPI approach on the same number of compute nodes. As clusters of larger shared-memory multiprocessor (SMP) machines continue to become more cost effective, we expect this hybrid MPI-OpenMP approach to be increasingly valuable.