A Stochastic ContROL approach to machine Learning with applications to Environmental Risk models (SCROLLER)

The main idea in the SCROLLER project is to study the connections between stochastic analysis, risk theory and machine learning. Stochastic analysis is the part of mathematics that analyzes uncertainty over time. In particular, stochastic optimal control theory is a tool for making optimal decisions over time under uncertainty. The reason we work with models with uncertainty (stochastic models) instead of models without uncertainty (deterministic models) is that most problems in reality are affected by uncertain events. Weather, politics, climate change and human behavior are all sources of uncertainty. When we choose applications for the SCROLLER project, we will focus on issues related to environmental and climate risks. For example, we will work with stochastic models for the decay and maintenance of systems exposed to weather and wind. We have used so-called environmental contours to assess the safety of systems that are exposed to extreme environmental influences. Due to climate change, there is more extreme weather, and generally greater uncertainty regarding the future. We hope that this project can contribute to risk assessments that take these changes into account. So far in the SCROLLER project, we have worked on the link between environmental contours and optimal design. In this connection, we have found a representation of the design optimization problem via environmental contours for two different risk measures (value-at-risk and conditional-value-at-risk) under very general conditions, i.e. processes that are not necessarily stationary, such as weather variables. Furthermore, we have looked at the optimal design of experiments via the practically feasible solution method sequential Bayesian optimization, and compared this with the theoretically optimal solution via stochastic optimal control theory. We are also in the process of studying self-exciting jumping processes and solving stochastic control problems for jumping processes with default. Furthermore, we have looked at a practical application of a stochastic control problem: How to control the spread of HIV/AIDS in an optimal way using a vaccine called PReP. We have also studied reinforcement learning and dynamic programming in connection with so-called probabilistic digital twins. These are digital copies of physical systems used for risk assessment. In connection with these works, we have completed several articles that have been submitted to peer-reviewed journals.

The SCROLLER-project studies the connections between stochastic analysis, risk theory and machine learning. Our results will be applied to risk management and reliability analysis with a focus on environmental variables as risk factors and models related to climate. The project consists of 4 scientific work packages: WP1 - Reinforcement learning and stochastic optimal control: We investigate whether it is possible to use the ideas and theoretical results from the stochastic maximum principle in connection with reinforcement learning. Can we draw inspiration from the martingale methods and the maximum principle theory of stochastic optimal control to relax the Markovian assumption of reinforcement learning? We will compare ML methods for stochastic optimal control problems (e.g., reinforcement learning and deep neural networks) to classical dynamic programming techniques. WP2 - Constrained risk management and connections to machine learning: We study an optimal consumption problem with a weighted value at risk constraint (WVaR) as well as other kinds of trading constraints (for instance no shortselling). We would also like to include the WVaR concept when doing risk analysis of stochastic networks. WP3 - Stochastic process modeling of degradation caused by environmental risk factors: We use self-exciting processes with positive jumps to model degradation caused by sporadic shocks with clustering behaviour. We will study the limiting process when the jump intensity increases at the same time as jump size decreases and compare this to the gamma process and the inverse Gaussian process. We also study optimal maintenance. WP4 - Augmented environmental contours: We extend environmental contours with the hope of contributing to better safety assessments of structures exposed to environmental risk factors. In particular, we study design optimisation with buffering and see how WVaR can be included in the environmental contour framework. We also include time in the model.