Modern AI models are highly data driven and are often trained from many (often thousands) examples of annotated data. By annotated data, we refer to data where we know the content of each sample. A weakness of such data driven models is they lack commonsense knowledge in the form of understanding constraints, physics, and other rules that describe the real world. Therefore, when we apply the models, we often observe strange predictions that may contradict well known rules and constraints.
The goal of this project is to advance deep learning models, a popular type of AI model, by exploiting and integrating human knowledge, constraints, and physics into the models. We will then utilize these novel models to provide a leap forward in automatic analysis of Earth observation image data. The use of such prior knowledge will make the training of the deep learning models more efficient. This means we will need fewer samples to train the models well or to make them more accurate. It also opens new possibilities for applying unlabeled data (i.e., data with no annotation) for training the deep learning models.
To utilize commonsense knowledge, so-called knowledge graphs is particularly interesting. A knowledge graph is a structured representation of facts. Critical R&D challenges include design of knowledge graphs and how such graphs can be efficiently integrated into DL models.
The project focuses on two Earth observation challenges: mapping of wetlands and mapping of snow cover from satellite image data. The expected results are improved methods for mapping of land cover. This will contribute to improved management of our wetland ecosystems. Improved maps of snow cover will provide better estimates of water resources available for hydropower production and better monitoring of climate changes in the cryosphere.
The project starts July 1, 2023, is a collaboration between NR, UiT, NIBIO and Edinsights.
Earth observation (EO) systems offer unprecedented volumes of data. To utilize this information, automated analysis of EO data is an ongoing research activity. Deep learning (DL) has revolutionized the analysis of image data and is currently state-of-the art for solving a wide range of remote sensing tasks. However, although DL can fit a relatively accurate mapping of the input data, it can easily fail to capture underlying rules and constraints.
The project goals are to integrate human knowledge, constraints and physics into DL models, to develop semi-supervised learning (SSL) methods that utilizes human knowledge, and to develop methodology to estimate the corresponding pixel-wise uncertainties. Critical R&D challenges include design of knowledge graphs and how such graphs can be efficiently integrated into DL models, how to perform SSL for multi-sensor and multi-temporal data where modalities may be missing, how to utilize integrated knowledge for improved learning capabilities, and how to efficiently model the pixel-wise uncertainties.
Methodology that enables the inclusion of human knowledge are of great interest within the research community and will have a big impact on next generation DL methods. This includes improved test accuracy, learning capabilities, trust, reliability, and understanding behind the reasoning. The methodology developed is generic and will therefore be applicable to other scientific challenges, outside of the stakeholders’ applications.
The project focuses on two mapping challenges: wetland and snow cover. The outcome of this project will directly enhance our understanding of wetlands in Norway, and will contribute to reduced loss of wetland areas and maintenance of biodiversity and ecosystem services. Improved maps of snow cover will provide better estimates of water resources available for hydropower production. Moreover, improved snow cover estimates will enable better monitoring of climate changes in the cryosphere.