Latent Variable Factor Mixture models to track Longitudinal Differentiation Patterns

Sammenligninger med en meningsfull 'baseline' er nøkkelen til å evaluere kvalitet på en bestemt mengde. Hvis en elev svarer riktig på 5 av 10 ja/nei-spørsmål, er det da bra eller dårlig? Snakker man absolutt, fikk eleven halvparten riktig, men når man innser at en elev som tilfeldig gjetter svaret også forventes å få halvparten rett, da er elevens prestasjon mindre imponerende. Lignende komparative prinsipper viser seg å være nyttige i andre mer komplekse settinger: Fra evalueringer av kvaliteten på populære statistiske modeller, til evalueringer av responskvaliteten i studentspørreskjemaer fra den internasjonale stor-skala komparative utdanningsforskningen. I den første settingen soner vi inn på den eksakte betydningen av sammenligning av baselinemodeller i evalueringen av såkalte strukturelle ligningsmodeller som er veldig populære innen samfunnsvitenskap, og klargjør hvorfor dagens praksis på feltet er langt fra ideell. I sistnevnte sett soner vi inn på bruken av blandinger av statistiske modeller for å skille mellom og identifisere individer som genuint svarte på spørreskjemaet versus individer som ga svar på en mer tilfeldig måte som om de ignorerte det som ble spurt om. Denne metodikken kan brukes til målingsvaliditet og datakvalitetssjekker og er spesifikt relevant for undersøkelsesvurderinger med lavinnsatskarakter der deltakerne ikke har noe å vinne eller tape på å delta i undersøkelsen.

Target: Quantitative researchers in social sciences - Increased awareness about meaning of fit indices in SEM & the limited applicability of common rules of thumb; - Code+Examples to more properly use & report on incremental fit indices; - Dissemination of mixture IRT to model response heterogeneity and patterns in questionnaire data with examples+code to implement the suggested procedures. Target: Stakeholders in international large-scale comparative studies - Increased awareness about invalid response behavior (e.g., prevalence, consequences, ...) on the student questionnaire components of studies such as TIMSS and PISA. - Statistical modeling procedures allowing the mapping of such behavior & running sensitivity studies to check for inferential robustness - Potential inclusion of these procedures as survey quality indicators by the organizing parties (e.g., IEA, OECD) Note. Deviation from originally planned longitudinal application context due to COVID19-related issues.

Developmental processes are key in the social sciences, with individual progression being a core issue. Usually, progression addresses the question how far you moved along a ruler and assumes that as long as the same ruler (i.e., measurement instrument) is used, scores can naturally be compared across time. The technical term for this assumption is longitudinal measurement equivalence (LME). If the ruler would change or what the ruler is trying to measure changed throughout the process, the common ground for comparisons disappears. Hence, you risk comparing apples and oranges. This is problematic when using individual progress profiles for selection decisions in schools and companies. From this perspective, LME is a threat that needs to be averted, yet in some cases it might actually be an essential sign of appropriate development. The assessment of competence acquisition for student-teachers might be a good example. While at the start of the learning process the different competences might be one undifferentiated pile mainly reflecting general skill, a more differentiated and specialized competence structure is expected to surface through experience and learning opportunities. Hence, for learning progressions of student-teachers it is not only a question of how much the competences changed (i.e., sliding along the same ruler), but of in what way their competences changed. Instead of quantitative growth, focus is on qualitative evolution where the level of differentiation is used as a measure of progression. To study such change patterns, the current project aims to develop sound statistical procedures to accommodate the tracking of intra-individual longitudinal differentiation by creative use of latent variable mixture models that account for inequivalent progress trajectories and individual differences in development. This will foster new research in all scientific domains where individual progression, change, and development is of interest.