Educational assessments of the 21st century: Measuring and understanding students adaptability in complex problem solving situations

All life is problem solving. We are often faced with situations, in which the information that is needed to solve a problem is not immediately obvious and in which things can change quite rapidly. For instance, imagine the software on your smartphone went through an update and has therefore novel features, designs, and functionality. You will find yourself in a new and perhaps unfamiliar situation that requires you to learn and adjust your previous thinking about how your smartphone works. The ability to cope with such situations is referred to as "adaptability". Given the rapid development of modern information and communication technology in our society, this ability has become as important as never before. But how exactly does adaptability work? Which steps need to be taken to become adaptable? How does adaptability relate to the success in solving a problem? And, most importantly, how can we know that we are adaptable? This study has taken an educational perspective on these questions. During the first project period, it became clear that adaptability does not only refer to flexible thinking but also to emotional agility and affective adjustments. This knowledge informed the first step of developing an assessment of adaptability and resulted in a self-report scale. The scale showed sufficient psychometric properties and differentiated well between cognitive-behavioural and affective-emotional aspects of adaptability. The project revealed during the second period that students' beliefs about science and scientific knowledge is related to the beliefs about their own adaptability. In other words, students see changes in the world of science and scientific knowledge through "their own lenses". In the last project period, a computer-based test of adaptability was developed and administered to approximately 1200 students at the transition to university. This test contained tasks about health, nutrition, and sleep and assessed basic problem-solving skills and the skills to adapt to novel or changing information. Students? performance on these tasks was positively related to their ability of abstract reasoning. Multiple adaptability profiles of students existed and ranged from hardly to highly adaptable. These profiles largely differed in the mindsets students had reported as either being fixed or growth mindsets. Overall, this project resulted in a self-report and a performance-based assessment of adaptability in complex problem-solving situations and has shown that individual differences in adaptability exist are related to students' mindsets, curiosity, and beliefs about science.

The project results have an impact on the understanding of adaptability. So far, adaptability has largely been considered a trait that mainly requires cognitive flexibility and abstract reasoning; however, our project has shown that it also requires affective and emotional skills and beliefs. In fact, adaptability comprises a complex skillset and belief system that involve self-beliefs, beliefs about scientific knowledge, and mindsets. These novel perspectives on adaptability have informed, to some extent, the frameworks of other international projects, such as the problem-solving framework of the second PIAAC cycle and the computational thinking framework of the TEACH21st project. The project's respective publications have highlighted the different components of adaptability and have informed the scientific community as well as contributed to the extant body of research. Finally, the setup of the assessments can be used to map students' adaptability broadly.

Given the rapid development of modern information and communication technology in our society, the skill demands in education and workplaces have recently shifted toward non-routine, dynamic, and complex skills. From an educational perspective on this shift, there is an increasing need for valid and innovative assessments that are able to capture the complexity of these key skills. The project addresses this need by developing and validating an innovative, computer-based assessment of students' ability to creatively adjust their thinking, behavior, and drive to changes and novelty (i.e., adaptability) in complex problem solving (CPS) situations. Building on existing assessments of CPS, this new assessment requires students to react on dynamic changes and novel information that determine the problem situation. The resulting data provide unique opportunities to study students' behavior with respect to adaptability. Besides looking at the cognitive and behavioral components of adaptability, the project will evaluate indicators of an affective dimension to provide a differentiated perspective on this key construct. The study will be conducted in grade levels 8 to 10 (N=1,450), a developmental phase in which students need to deal with a number of novel and dynamic situations; the data will be analyzed using advanced latent variable modeling. To further understand the nature of adaptability as a construct that may assist students in academic and non-academic contexts, and to examine the validity of the new assessment, the relations to cognate constructs such as fluid intelligence and working memory capacity, motivational constructs such as openness and self-concept, and to educational criteria such as school achievement are investigated. The project will generate new knowledge about adaptability in CPS situations as a critical skill in 21st century education, and will advance national competences in developing innovative, computer-based assessments.