Meeting Exascale Computing with Source-to-Source Compilers

Future computing platforms are expected to be heterogeneous in architecture, specifically, consisting of conventional CPUs and powerful hardware accelerators. The hardware heterogeneity, combined with the huge scale of these future platforms, will make the task of programming extremely difficult. To overcome the programming challenge for the important class of scientific computations that are based on meshes, this project aims to develop new programming methodologies and fully automated source-to-source compilers. The new methodologies and compilers will help computational scientists to quickly prepare implementations of implicit and explicit mesh-based computations for truly-heterogeneous and resource-efficient execution on CPU+accelerator computing platforms. Real-world simulators from computational cardiology will be used as testbeds of the new programming methodologies and automated compilers. The success of such real-world heterogeneous simulations will not only verify the usefulness of the source-to-source compilers, but more importantly will allow unprecedented resolution and fidelity when investigating the particular topics of heart failure and arrhythmia. During the project's first year, the research focus was on investigating how Intel's first-generation many-core Xeon Phi processors should be effectively programmed. Two research papers were published/accepted, where one of the articles successfully used Tianhe-2: the world's second most powerful supercomputer until summer 2017. The project leader also had a central role in a newly granted INFRASTRUKTUR proposal (eX3), which will provide this IKTPLUSS project with extremely valuable access to bleeding-edge hardware. The project's research activities were during the second year also extended to cover GPUs, currently the dominating hardware accelerators for computing. A particular objective is to enhance FEniCS, which is a software framework for finite element computations based on automated code generation, with GPU-computing capabilities. The project team also started a new collaboration with University of Tokyo, with respect to using Japan's Oakforest PACS supercomputer (a cluster of Intel's second-generation Xeon Phi processors) for simulations of subcellular calcium dynamics. Experiences of programming accelerators from the project had delivered important inputs to the design of the eX3-infrastructure. The third year of the project saw results from the fruitful collaboration with the Japanese side, in the form of new programming methodologies and simulation results, which were obtained through using the Oakforest PACS supercomputer. The other focus area of the third year was on using the cutting-edge GPUs from Nvidia, currently the most powerful commodity hardware available. Thanks to numerous experiments enabled by the eX3 infrastructure, it was demonstrated how a DGX-2 machine, consisting of 16 Nvidia V100 GPUs, can be used to simulate the electrical activity inside the human heart, using realistic geometric and physiological data. Such multi-GPU enabled simulations are much faster than those using a state-of-the-art simulator based on CPUs. Moreover, a prototype code translator has been built, inside the FEniCS framework, to automatically compile a high-abstraction-level description of a solver of partial differential equations into a GPU-capable simulation code. The project has also contributed to an increased Norwegian participation in international collaboration projects centering around HPC.

The cardiac-simulation research results produced by this project have helped the host institution (Simula Research Laboratory) to reinforce its internationally leading position in cardiac modeling and simulation, through efficient use of heterogeneous supercomputers. Moreover, the expertise acquired by the project's core research team in performance modeling, heterogeneous parallel programming and source-to-source compilation, has led to several successful international joint research proposals with Simula being the key partner. Nationally, this project has considerably contributed to the establishment and operation of the national eX3 infrastructure (Experimental Infrastructure for Exploration of Exascale Computing), hosted at Simula, funded by RCN's Forskningsinfrastruktur program (Project 270053). One of the core members of this project has become the recipient of two researcher projects from RCN.

Huge-scale heterogeneous clusters, which consist of CPUs and hardware accelerators, are expected to be the dominating architecture of future supercomputers. To break the programming barrier, this project proposes a radical strategy which uses automated source-to-source compilers to completely replace the tedious and error-prone manual programming. Another distinctive feature of this new strategy is to simultaneously make use of the computational power of both CPUs and accelerators, instead of the currently prevailing strategy that offloads some work tasks completely from the CPUs to the accelerators. The focus area of this project will be on mesh-based computations related to solving partial differential equations, a central subject in the entire field of computational science. Two fully automated source-to-source compilers will be developed. One targets implicit computations and allows user input as a high-level description of the equations to be solved. The other compiler targets explicit computations and accepts user input as plain, serial code with a few annotations in the style of OpenMP. The internal code analyzers of both compilers will, in combination with the backends, automate the implementation of performance-critical optimizations such as task pipelining and asynchronous data transfers. Two simulators from computational cardiology will be used as real-life testbeds of the two compilers, aiming to automatically generate hardware-compatible heterogeneous implementations. These will not only advance cardiac research on the particular topics of heart failure and arrhythmia, but demonstrate the usefulness of the two source-to-source compliers to all computational scientists who use mesh-based computations, thereby enabling them to welcome the future arrival of exascale computing.