Artificial intelligence (AI) systems are becoming ubiquitous and disruptive to industries such as healthcare, transportation, manufacturing, robotics, retail, banking, and energy. According to a recent European study, AI could contribute up to EUR 13.3 trillion to the global economy by 2030. Most of the recent AI breakthroughs can be attributed to the subfield of deep learning (DL), which has achieved state-of-the-art performance in tasks traditionally challenging for machines, such as object and speech recognition, and beating human champions at the game of Go. However, the complex and unintuitive black-box nature of DL models results in lack of interpretability, lack of robustness, and inability to generalize to situations beyond those encountered during training. These issues must be addressed in order to make AI systems trustworthy and deployable in social environments, industry and business-critical applications.
In the EXAIGON project, the Norwegian University of Science and Technology (NTNU), SINTEF Digital and international research partners will address these challenges by developing user-centric Explainable AI (XAI) methods for understanding how black-box models make their predictions and what are their limitations. Due to the data-driven nature of the topic, the research within EXAIGON will be driven by use cases (including datasets, models, and expert knowledge) provided by seven industry collaborators: DNB, DNV, Embron, Equinor, Kongsberg, Telenor and TrønderEnergi. The main focus areas are supervised learning, deep reinforcement learning, deep Bayesian networks and human-machine co-behaviour.
EXAIGON will create an XAI ecosystem around the Norwegian Open AI-Lab, and its outcomes will benefit the research community, the industry and high-level policy makers, who are concerned about the impact of deploying AI systems to the real world in terms of efficiency, safety, and respect for human rights.
By end of 2025, the project has graduated 26 MSc students, contributed to numerous dissemination activities and published several scientific results. Two PhD students have by now completed their work (in 2023 and 2025) and three more are in the process of finalizing their dissertations. Two of the project’s publications have received best papers awards:
• Bjøru et al.: A Divide and Conquer Approach for Solving Structural Causal Models. 12th International Conference on Probabilistic Graphical Models, PMLR 246, 348-360
• Calero et. al.: An XAI Approach on the Capacity of Transformers to Learn Time Dependencies in Time Series Forecasting (Presented during EXPLAINABILITY 2024, The First International Conference on Systems Explainability, held in Valencia, Spain during November 17 - 21, 2024
EXAIGON has had extensive international collaboration with several institutes, including research stays abroad. One PhD student spent 2 months in Australia, as part of a research stay at the group of Professor Tim Miller at the University of Melbourne. Prof. Tim Miller is an international expert within XAI, and has co-authored two EXAIGON publications. Two PhD students (IDI/NTNU) have had extended international collaboration with researchers at the Complutense University of Madrid (Spain) and Universidad de Almeria (Spain), including research stays at those universities in 2024. The project’s postdoctoral researcher (IDI/NTNU) is collaborating with the University of Washington (USA) and the Aalto University school of Business (Finland). There is also visit of a PhD researcher (ITK/NTNU) at ETH Zurich between Oct 2024 – Feb 2025. All visits have resulted in academic publications in collaborations with the collaborating institutes.
The project’s researchers have so far had several interactions with the industry collaborators, and one postdoc maintains regular collaboration with 3 of the partners.
In June 2025, a major contribution by the project was presented at a webinar with strong industry participation, co-organized between EXAIGON and SFI AutoShip. In the webinar, full-scale sea trial results of a deep reinforcement learning controller with real-time explanations were presented for the first time.
The EXAIGON project has delivered substantial scientific, industrial, and competence-related impact, in line with the anticipated outcomes defined at project start.
From a scientific perspective, the project has contributed significantly to the state of the art in explainable AI (XAI). More than 30 peer-reviewed scientific publications (a few still under review) have been produced, addressing multiple aspects of XAI, including explainable deep reinforcement learning, explainability in deep neural networks, Bayesian approaches, and human–machine interaction. The project established strong international research collaborations through research stays abroad, resulting in several co-authored publications with leading international partners. Two of the project’s publications received best paper awards, underlining the scientific quality and relevance of the results.
The project has also achieved clear industrial impact through close collaboration with several industry partners, including Kongsberg Digital, Zeabuzz, Embron, and DNV. These collaborations were centered around concrete industrial use cases, with demonstrations of XAI-based methods in realistic and operational environments. The relevance of the project is further reinforced by the fact that several EXAIGON graduates are now employed in industry, where they continue to develop and apply methods and approaches related to explainable AI.
In terms of human capital and competence building, EXAIGON has educated a new generation of experts in trustworthy and explainable AI. Two PhD candidates have completed their doctoral degrees during the project period, and three additional PhD candidates are in the final stages of their dissertations. In addition, 26 MSc students have graduated with theses related to the project, several of whom are now working in industry. The scientific outcomes of the project have also been integrated into education, and several of the developed methods are now taught in NTNU courses, including TTK4192 – Mission Planning for Autonomous Systems.
The project has benefited from and contributed to research infrastructure at NTNU, in particular through extensive testing on NTNU’s autonomous passenger ferry, milliAmpere1. Full-scale experiments provided valuable practical insight into the deployment of XAI methods in real-world autonomous systems and informed upgrades of the ferry’s sensor suite, which will further support future research.
Finally, the project has generated emergent impact beyond its original scope. Several results and collaborations from EXAIGON are now being actively pursued in follow-up initiatives, including SFI AutoShip and the newly established Maritime AI centre, demonstrating the project’s lasting relevance and impact.
The recent advances of Artificial Intelligence (AI) hold promise for significant benefits to society in the near future. However, in order to make AI systems deployable in social environments, industry and business-critical applications, several challenges related to their trustworthiness must be addressed first.
Most of the recent AI breakthroughs can be attributed to the subfield of Deep Learning (DL), but, despite their impressive performance, DL models have drawbacks, with some of the most important being a) lack of transparency and interpretability, b) lack of robustness, and c) inability to generalize to situations beyond their past experiences.
Explainable AI (XAI) aims at remedying these problems by developing methods for understanding how black-box models make their predictions and what are their limitations. The call for such solutions comes from the research community, the industry and high-level policy makers, who are concerned about the potential of deploying AI systems to the real world in terms of efficiency, safety, and respect for human rights.
EXAIGON will advance the state-of-the-art in XAI by conducting research in four areas:
1. Supervised learning models
2. Deep reinforcement learning models
3. Deep Bayesian networks
4. Human-machine co-behaviour.
Areas 1-3 involve design of new algorithms and will interact continuously with Area 4, to ensure the developed methods provide explanations understandable to humans. The developed methodologies will be evaluated in close collaboration with 7 industry partners, who have provided the consortium with business-critical use cases, including data, models and expert knowledge.
The consortium includes two international partners from the University of Wisconsin-Madison, and University of Melbourne, respectively, who have conducted and published outstanding research in relevant areas over the last few years.
