Navigation and service

Computational fluid engineering

Fluid engineering

AVBP is a parallel Computational Fluid Dynamic (CFD) code that studies the combustion process in gas turbines, targeting its optimisation, impacting stability and pollution reduction. The AVBP code has been ported to all major systems (SGI Altix ICE, CRAY XT4, IBM BlueGene/Q) with excellent performance. However, keeping a good level of scalability and performance on the upcoming HPC systems is challenging: CFD requires all computing cores to communicate frequently, and the physical models often require reduced variables (max/min/mean) over the complete computing partition.

In DEEP, CERFACS (the European Centre for Research and Advanced Training in Scientific Computation) aims at improving AVBP’s scalability by taking advantage of the Cluster-Booster architecture. In order to do that, the bottlenecks caused by the original master/slave approach were removed as a first step. The next step was migrating from a pure MPI approach to a hybrid approach of MPI+OmpSs. The OmpSs model allows exposing additional parallelism, and by using the task-based model, it was possible to implement a version of the application that outperforms and outscales the previous one. Loop refactoring and compiler hints gave an extra edge in performance, as now the vector units are used more efficiently. Lastly, the I/O operations have been offloaded, together with costly global reductions that hindered the scalability of the application, and that now are performed in an overlapped manner in the Cluster, while the simulation continues on the Booster.

"Computational fluid dynamics typically scales easily thanks to their nearest neighbour communication, that avoid complex communication patterns. However, it has strong pre and post processing phases. Our code scale nicely in architectures that resemble the Booster, which makes us think that it will run well on the Booster too. Being able to couple the Booster with the Cluster allows us to keep the I/O phase in the Cluster, sized accordingly to the I/O requirements rather than to the simulation requirements, and at the same time have the possibility of powerful streamlined postprocessing without disturbing the simulation itself." - Gabriel Staffelbach