TY - GEN
T1 - PERFORMANCE MODELING AND SCALING OF PETSC BASED DIRECT NUMERICAL SIMULATIONS FOR HYBRID ROCKET BOUNDARY LAYERS
AU - Retfalvi, Kolos
AU - Knepley, Matthew
AU - DesJardin, Paul E.
N1 - Publisher Copyright:
Copyright © 2025 by ASME.
PY - 2025
Y1 - 2025
N2 - This paper presents a comprehensive performance and scaling analysis of direct numerical simulations for reacting boundary layers, focusing on slab burner configurations. Using a PETSc-based finite volume CFD framework, the study evaluates the scalability and computational cost of flow, chemistry, and radiation evaluations across 2D and 3D simulations. Parallel scalability is analyzed for the coupled flow, chemistry, and radiation heat transfer processes. Weak and strong scaling studies are conducted on up to 15,000 computational ranks, revealing robust performance for flow cells exceeding 200 per rank. Chemistry evaluations dominate the computational cost in large 3D simulations, accounting for approximately 40% of the total runtime, while flow processes contribute around 35%, and radiation solver contributions remain below 10% due to reduced evaluation frequencies. GPU accelerate chemistry evaluation, implemented with Zero-RK, demonstrates significant promise, achieving up to a 5× speedup for workloads exceeding 30,000 cells per GPU. This study identifies key challenges, including memory bottlenecks and the effects of domain partitioning on flow scalability, while highlighting the potential of GPU-accelerated chemistry to reduce computational costs. These findings provide realizable run configurations for 2D, 3D, and GPU-accelerated cases, offering insights for optimizing reactive flow solvers similar to ABLATE.
AB - This paper presents a comprehensive performance and scaling analysis of direct numerical simulations for reacting boundary layers, focusing on slab burner configurations. Using a PETSc-based finite volume CFD framework, the study evaluates the scalability and computational cost of flow, chemistry, and radiation evaluations across 2D and 3D simulations. Parallel scalability is analyzed for the coupled flow, chemistry, and radiation heat transfer processes. Weak and strong scaling studies are conducted on up to 15,000 computational ranks, revealing robust performance for flow cells exceeding 200 per rank. Chemistry evaluations dominate the computational cost in large 3D simulations, accounting for approximately 40% of the total runtime, while flow processes contribute around 35%, and radiation solver contributions remain below 10% due to reduced evaluation frequencies. GPU accelerate chemistry evaluation, implemented with Zero-RK, demonstrates significant promise, achieving up to a 5× speedup for workloads exceeding 30,000 cells per GPU. This study identifies key challenges, including memory bottlenecks and the effects of domain partitioning on flow scalability, while highlighting the potential of GPU-accelerated chemistry to reduce computational costs. These findings provide realizable run configurations for 2D, 3D, and GPU-accelerated cases, offering insights for optimizing reactive flow solvers similar to ABLATE.
KW - Computational Fluid Dynamics
KW - Direct Numerical Simulation (DNS)
KW - High Performance Computing
KW - Performance
KW - Reacting Flows
KW - Scalability
UR - https://www.scopus.com/pages/publications/105018465099
U2 - 10.1115/FEDSM2025-158539
DO - 10.1115/FEDSM2025-158539
M3 - Conference contribution
AN - SCOPUS:105018465099
T3 - American Society of Mechanical Engineers, Fluids Engineering Division (Publication) FEDSM
BT - Artificial Intelligence (AI) for Fluids; CFD Methods; CFD Applications; Bio-Inspired and Biomedical Fluid Dynamics; Fluid Measurement and Instrumentation; Energy and Sustainability
PB - American Society of Mechanical Engineers (ASME)
T2 - 2025 ASME Fluids Engineering Division Summer Meeting, FEDSM 2025
Y2 - 27 July 2025 through 30 July 2025
ER -