Enabling Graph Neural Networks at Exascale

Where do fake news come from, and how do they spread? Can we identify misinformation spreaders in social networks? Which roads will be blocked by traffic jams tomorrow? What are the possible side effects of drugs? Many crucial questions require predictions in a connected world, and Graph Neural Networks (GNNs) allow us to make such predictions. While current AI systems can understand and generate text and images, GNNs can understand the structure of a connected world. They can find better solutions for abstract problems which can then be used for direct improvements in many diverse areas, from traffic planning to industrial optimization. In social networks GNNs can learn and understand patterns of information spread by analyzing network connections and messages contents at the same time. These capacities are crucial to combat online misinformation. As the networks to be analyzed grow larger, more powerful computers are needed to run these AI systems. By using novel, innovative computing devices, we can limit the energy consumption of computing GNNs. Thereby we ensure that progress in artificial intelligence happens in an environmentally friendly manner.

- 18 papers - 10 master theses - 3 misinformation datasets - multiple GNN implementations, including on GPUs - New understanding of the possibilities and limitations when using GNNs for combinatorial problems - New understanding of the possibilities and limitations when using GNNs for social network analysis - Foundational work on using Graphcore Intelligence Processing Units (IPU processors) for graph problems and GNNs - Foundational work on incremental and temporal clustering algorithms - A new type of GNN that alleviates the over-squashing problem - New national and international collaborations

Graph neural networks (GNNs), which extend the successful ideas of deep learning to irregularly structured data, are a recent addition to the field of artificial intelligence. While traditional deep learning has focused on regular inputs such as images composed of pixels in two-dimensional space, graph neural networks can analyze and learn from unstructured connections between objects. This gives GNNs the ability to tackle completely new classes of problems, such as analyzing social networks and power grids, or uncovering molecule structures in computational chemistry. Some experts in the field also believe that graph networks, due to their capacity for combinatorial generalization, represent an important next step towards the development of general artificial intelligence. However, such tasks require vast amounts of computation, which can only be provided by parallel processing. Is well known that parallel computation for irregular problems is much more challenging than for regular ones, and GNNs are no exception. While traditional deep learning has been scaled up to run on entire supercomputers efficiently, GNNs currently do not scale to multiple processors. This proposal aims to overcome this limitation by drawing upon decades of experience in scalable graph algorithms and sparse linear algebra, and adapting techniques that are proven to be effective for distributing graph computations over large parallel systems to GNNs. We aim to create a new computational framework which allows users to specify a GNN while the framework handles the task of distributing graphs over parallel machines, as well as selecting and running the algorithms that are best suited for the computation automatically. Recently, frameworks such as TensorFlow, have made traditional deep neural networks accessible for a large number of users. In the same way, our goal is to create a proof-of-concept framework that will be a crucial factor to the successful GNNs real-world appliance.