The main idea of the SCROLLER-project is to study the connections between stochastic analysis, risk theory and machine learning.
Stochastic analysis is the mathematical study of uncertainty over time. In particular, stochastic optimal control theory is a tool for making optimal decisions over time under uncertainty. The reason for working with stochastic models, as opposed to deterministic ones, is that most real-life problems are influenced by uncertain factors. Weather, politics, climate change and human actions are potential sources of uncertainty.
During the last decade, there has been a vast technological development and growth in computational power. In addition, digitalisation implies that big data is available in many different settings. Machine learning is a set of mathematical algorithms and techniques which enable computers to improve at performing tasks with experience. Examples of ML algorithms are neural networks and reinforcement learning.
Machine learning algorithms can lead to wrong conclusions if we are not careful in understanding the underlying mathematics. Though the experimental results of machine learning are good, there is still a lack of understanding of the mathematical reasons for these results. In particular, the literature concerning the connections between machine learning and stochastic analysis is sparse. The main purpose of the SCROLLER-project is to study these connections.
In choice of applications throughout the SCROLLER-project, we will focus on problems related to environmental and climate risks. For instance, we will work on degradation models with respect to environmental risk factors. We will use environmental contours for safer risk assessment of structures exposed to extreme environmental events. Due to climate change, there is more extreme weather, and in general more uncertainty regarding the future. We hope that this project can contribute to derive suitable risk assessments which take this change into account.
So far during the SCROLLER project, we have worked on the connection between environmental contours and optimal design. In this connection, we have found a representation of a design optimisation problem via environmental contours for two different risk measures (value-at-risk and conditional value-at-risk). We have also studied optimal design of experiments via the practically feasible solution method of sequential Bayesian optimisation, and compared this to the theoretically optimal approach of stochastic optimal control. Furthermore, we have begun our studies of self-exciting jump processes and stochastic control problems for default jump processes. Finally, we have studied a practical application of stochastic optimal control: Optimally controlling the spread of HIV/AIDS through the so-called PReP vaccine. We have also studied reinforcement learning and dynamic programming i connection to probabilistic digital twins. These are digital copies of physical systems which can be used for risk assessment.
In connection to these works, we have completed several papers which have been submitted to peer review journals.
The SCROLLER-project studies the connections between stochastic analysis, risk theory and machine learning. Our results will be applied to risk management and reliability analysis with a focus on environmental variables as risk factors and models related to climate. The project consists of 4 scientific work packages:
WP1 - Reinforcement learning and stochastic optimal control:
We investigate whether it is possible to use the ideas and theoretical results from the stochastic maximum principle in connection with reinforcement learning. Can we draw inspiration from the martingale methods and the maximum principle theory of stochastic optimal control to relax the Markovian assumption of reinforcement learning? We will compare ML methods for stochastic optimal control problems (e.g., reinforcement learning and deep neural networks) to classical dynamic programming techniques.
WP2 - Constrained risk management and connections to machine learning:
We study an optimal consumption problem with a weighted value at risk constraint (WVaR) as well as other kinds of trading constraints (for instance no shortselling). We would also like to include the WVaR concept when doing risk analysis of stochastic networks.
WP3 - Stochastic process modeling of degradation caused by environmental risk factors:
We use self-exciting processes with positive jumps to model degradation caused by sporadic shocks with clustering behaviour. We will study the limiting process when the jump intensity increases at the same time as jump size decreases and compare this to the gamma process and the inverse Gaussian process. We also study optimal maintenance.
WP4 - Augmented environmental contours:
We extend environmental contours with the hope of contributing to better safety assessments of structures exposed to environmental risk factors. In particular, we study design optimisation with buffering and see how WVaR can be included in the environmental contour framework. We also include time in the model.