Latent Variable Factor Mixture models to track Longitudinal Differentiation Patterns

Comparisons to a meaningful baseline are key to evaluate the quality of any measured quantity. If a student chooses the correct response option for 5 out of 10 yes/no quiz items, is that a good or bad performance? In an absolute sense, the student got half correct, but when you realize that someone who randomly guesses the answer is also expected to get half correct, the student's performance is in fact less than impressive. Similar comparative principles prove to be useful in other more complex settings: From evaluations of the quality of popular statistical models to evaluations of the response quality of self-report data in student questionnaires of international large-scale comparative educational research. In the former setting, we zone in on the exact meaning of baseline model comparisons in the evaluation of so-called structural equation models that are hugely popular in social sciences, clarifying why current practice in the field is far from ideal. In the latter setting, we zone in on the use of mixtures of statistical models to distinguish between and identify individuals that are genuinely answering the questionnaire versus individuals that provided responses in a more haphazard random fashion as if they were disregarding what is being asked from them. This methodology can be used for measurement validity and data quality checks and is specifically relevant to survey assessments with a low-stakes character in which participants have nothing to gain or lose from participating in the survey.

Target: Quantitative researchers in social sciences - Increased awareness about meaning of fit indices in SEM & the limited applicability of common rules of thumb; - Code+Examples to more properly use & report on incremental fit indices; - Dissemination of mixture IRT to model response heterogeneity and patterns in questionnaire data with examples+code to implement the suggested procedures. Target: Stakeholders in international large-scale comparative studies - Increased awareness about invalid response behavior (e.g., prevalence, consequences, ...) on the student questionnaire components of studies such as TIMSS and PISA. - Statistical modeling procedures allowing the mapping of such behavior & running sensitivity studies to check for inferential robustness - Potential inclusion of these procedures as survey quality indicators by the organizing parties (e.g., IEA, OECD) Note. Deviation from originally planned longitudinal application context due to COVID19-related issues.

Developmental processes are key in the social sciences, with individual progression being a core issue. Usually, progression addresses the question how far you moved along a ruler and assumes that as long as the same ruler (i.e., measurement instrument) is used, scores can naturally be compared across time. The technical term for this assumption is longitudinal measurement equivalence (LME). If the ruler would change or what the ruler is trying to measure changed throughout the process, the common ground for comparisons disappears. Hence, you risk comparing apples and oranges. This is problematic when using individual progress profiles for selection decisions in schools and companies. From this perspective, LME is a threat that needs to be averted, yet in some cases it might actually be an essential sign of appropriate development. The assessment of competence acquisition for student-teachers might be a good example. While at the start of the learning process the different competences might be one undifferentiated pile mainly reflecting general skill, a more differentiated and specialized competence structure is expected to surface through experience and learning opportunities. Hence, for learning progressions of student-teachers it is not only a question of how much the competences changed (i.e., sliding along the same ruler), but of in what way their competences changed. Instead of quantitative growth, focus is on qualitative evolution where the level of differentiation is used as a measure of progression. To study such change patterns, the current project aims to develop sound statistical procedures to accommodate the tracking of intra-individual longitudinal differentiation by creative use of latent variable mixture models that account for inequivalent progress trajectories and individual differences in development. This will foster new research in all scientific domains where individual progression, change, and development is of interest.