Back to search

IKTPLUSS-IKT og digital innovasjon

MBT4CPS: Evolving, Modelling and Testing Realistic and Risky Uncertain Behaviours of Cyber-Physical Systems

Alternative title: Model-based Testing of Cyber-Physical Systems (MBT4CPS)

Awarded: NOK 7.2 mill.

We use so-called Cyber-Physical Systems in our day-to-day life either knowingly or unknowingly. There is a broad spectrum of their applications ranging from simple applications as smart wristbands (e.g., recording health data) to complex applications such as systems deployed on the seabed for oil extraction. Undoubtedly, we envisage these systems to work reliably even when confronted with uncertain circumstances. However, the existing state-of-the-art system lacks approaches to deal with such a challenge. This project targets precisely this challenge, i.e., assuring that the applications of Cyber-Physical Systems (CPSs) work safely and reliably even in the existence of uncertain circumstances. As a starting point, the project aims to fill a small gap in the literature. To solve this challenge we rely on devising new automated testing methods for the applications of CPSs facing uncertain situations. Given the breadth and depth of the topic we emphasized on dealing with the following two types of uncertain situations: First, we focused on uncertainties related to the security features implemented in CPSs. Such types of security features include confidentiality, authorization, and integrity, to name a few. Any loophole in the implementation of these features may lead to uncertain behavior of a CPS and consequently, could result in unwanted circumstances. We focused on testing the following two types of uncertainties: 1) Uncertainties in the security features that lead to uncertain operations of CPSs; 2) The behavior of security features in the presence of other types of uncertainties that may compromise the security of CPSs. We have defined a taxonomy to understand and classify security-related uncertainties in CPSs. Based on this taxonomy, we have developed a novel model-based security testing approach. Our approach explicitly considers security-related uncertainties during testing. Tests generated with our approach discover unknown security-related uncertainties, in addition to finding security-related faults under uncertainty. To validate our work, we used a smart grids case study. Secondly, we focused on the uncertainties in the self-healing behaviors implemented in the applications of CPSs. CPSs are increasingly becoming intelligent systems and thus can heal themselves in the case of faulty situations. Self-healing behaviors are critical for the applications of CPSs since such types of behaviors allow them to recover from faults and avoid any harmful situations gracefully. We focus on testing self-healing behaviors in the existence of uncertain situations to assure that when these systems will be safe in real life applications, where uncertainty is unavoidable. We have developed specialized test modeling approach for creating executable test models for self-healing behaviors together with environmental uncertainties. To execute such models efficiently to find faults, we have devised new active machine learning-based test strategies to test drone applications. In addition, we have developed a testing approach to discover to discover unknown CPS behaviors under a variety of network conditions. In conclusion, we have developed new methods of testing CPSs with which one can systematically and automatically test CPSs under uncertainties in their applications. Once these well-tested systems are installed in real life these systems will be secure and safe and subsequently more reliable.

The following are the key scientific outcomes/impacts of the project: 1) Novel automated testing methods for testing the self-healing behaviours and security aspects of cyber-physical systems (CPS). Such methods have shown to be cost-effective when applied to a set of CPS case studies. 2) Software testing tools implementing a subset of these tools are publicly available for further future extensions by researchers; 3) The data from the experiments are also made publicly available to facilitate replication studies and further comparisons with the future testing methods; 4) The results are published in well-reputed peer-reviewed journals and conferences; 5) CPS have applications in many societal applications such as in welfare technologies provided by City of Oslo, and maritime domain (e.g., DNV-GL, Norway). Our results have potential to ensure safe and secure operation of these societal applications.

English Version: Our daily life is increasingly relying on Cyber-Physical Systems (CPSs). We can find such systems in many safety and mission-critical domains such as robotics, logistics, healthcare, aerospace, and maritime applications. Tackling uncertainty in CPSs at an acceptable cost is essential to prevent severe failures in real life. Ensuring that CPSs will provide correct behaviors in the presence of uncertainty, we need to demonstrate that CPSs achieve a satisfactory level of safety, security, and robustness at an acceptable cost. To ensure the quality of the systems, we need to develop cost-effective methods for verification and validation of CPSs such as based on systematic and automated model-based testing (MBT). Our goal is to improve the dependability (e.g., security and robustness) of CPSs with cost-effective model and search-based testing methods with a particular focus on known and unknown risky uncertainty. Unknown uncertainty is the one risk factor that can only be observed during the real operation of a CPS as opposed to the ones that are known at the design time. The consequence of risky uncertainty will be unwanted circumstances leading to potentially hazardous situations. We will achieve our goal by advancing the current state-of-art by developing new solutions for testing Cyber-Physical Systems based on sound theoretical foundations in the following steps. First, we will develop a solution for minimalistic modeling of different types of known risky uncertainties. Second, we will develop methods to intelligently evolve known models towards realistic and risky unknown uncertainty models (evolved models) using search algorithms (e.g., genetic algorithms mimicking natural selection). Third, we will develop methods to automatically generate test cases from the evolved models to test a CPS under unknown uncertainty. With our solutions, we will ensure that the CPSs will continue to operate appropriately even in uncertainty and possibly at a reduced quality of operation, rather than to fail. Norwegian Version: Vårt daglige liv er avhengig av Cyber-Fysiske Systemer (CFS-er) i økende grad. Vi kan finne slike systemer i mange sikkerhets- og samfunnskritiske domener som roboter, logistikk, helsetjenester, luftfart og sjøfart. Det er essensielt å håndtere usikkerhet i CFS-er til en akseptabel kostnad hvis man skal forhindre katastrofale feil i det virkelige liv. For å sikre at CFS-er vil oppføre seg korrekt i møte med usikkerheter må vi demonstrere at systemene kan være tilfredsstillende trygge, sikre og robuste, til en akseptabel kostnad. For å sikre denne kvaliteten må vi utvikle kostnadseffektive metoder for å verifisere og validere CFS-er, slik som metoden basert på systematisk, modellbasert testing (MBT). Målet er å forbedre påliteligheten (sikkerheten og robustheten) til CFS-ene med kostnadseffektive modellbaserte testmetoder som spesielt tar for seg kjente og ukjente uforutsigbarheter. Ukjente uforutsigbarheter er den ene risikofaktoren som bare kan observeres under virkelige kjøringer av Cyber-Fysiske Systemer, i motsetning til de uforutsigbarhetene som er kjent ved utviklingsstadiet. Konsekvensen av risikable uforutsigbarheter er at uønskede omstendigheter fører til potensielt farlige situasjoner. Vi ønsker å nå målet vårt ved å videreutvikle dagens forskning og utvikle nye løsninger for å teste CFS-er basert på et solid teoretisk grunnlag. Først vil vi utvikle en løsning for minimalistisk modellering av ulike typer risikable uforutsigbarheter. Deretter vil vi utvikle metoder for intelligent evolusjon av kjente modeller mot realistiske og risikable ukjente usikkerhetsmodeller (utviklede modeller) ved å bruke søkealgoritmer som minner om naturlig utvalg. Det tredje steget går ut på å utvikle metoder som automatisk genererer eksempeltester fra de utviklede modellene for å teste en CFS for ukjente uforutsigbarheter. Med våre løsninger skal vi sikre at CFS-ene vil fortsette å være i korrekt drift selv om de skulle møte på uforutsigbarheter. Systemene vil muligens kjøre med redusert kapasitet, men driftsstans kommer ikke til å inntreffe i møte med risiko.

Funding scheme:

IKTPLUSS-IKT og digital innovasjon