Back to search

FINNUT-Forskning og innovasjon i utdanningssektoren

Conversation Analytic innovation for Teacher Education (CAiTE). Research-based training for enhancing assessment and feedback quality

Alternative title: Trening av samtaleferdigheter i lærerutdanningen (CAiTE). Forskningsbasert rollespill som metode for å styrke vurderingspraksiser i skolen

Awarded: NOK 9.1 mill.

How can we train teachers to be effective in their assessment practices? CAiTE's goal (Conversation Analytic innovation for Teacher Education) was to strengthen the quality of teacher education and improve teachers' assessment practices by developing, evaluating and implementing a new research-based instruction method (CARM - Conversation Analytic Role-play Method) to train teacher students' conversational skills. Based on video analyzes of oral exams and teachers' supervision of group work classrooms, the project has developed, piloted, implemented and evaluated a CARM workshop for teacher students. CAiTE has provided significant research-based insight into teachers' teaching and assessment practices, both in the way teachers support students' learning in the classroom, and how oral exams are conducted. In particular, the research has mapped the communicative practices that provide immediate support to students. The project has for the first time shown how teachers and examiners translate guidelines for oral exams into practice, and how specific ways of asking questions in oral exams affect students' ability to show understanding and knowledge. CAiTE has contributed to an expanded understanding of how large variation in examination practice can have an impact on students. The findings showed that there are very limited guidelines and advice for assessment interviews. Recommendations and advice were largely based on personal experience, was theoretical, or anecdotal, rather than based on what happens in the classrooms. The results helped to legitimize CAiTE's purpose, which was to examine real teacher practices to see how they fit with guidelines, and then provide training and communication advice based on research results. There is considerable variation in time spent on the various parts of the oral examination. The oral exam consists of five activity phases that are clearly structured. There is great variation in how much time is spent on the different phases. There is also variation in how much of the curriculum is covered during student presentation and subject discussion. There is considerable variation in the number of questions asked and topics covered in the subject conversations between examiners and candidates. The subject conversation is a strongly asymmetrical conversation genre where the student's room for maneuver is very limited. The examiner's design of questions provides guidelines for what the candidates have the opportunity to answer. The examiners 'open, leading and multiple questions during the oral examination provide strong guidelines for what it is possible to answer, and these questions can have both a scaffolding-building and inhibiting effect on the candidates' opportunities to demonstrate competence. Sometimes candidates give an answer that is marked as incorrect. Then the examiners correct the candidate and come up with the correct answer for them, or they try to get the candidate to correct themselves by initiating repair in more or less direct ways. This gives the candidate a new opportunity to answer correctly, but often leads to long side sequences where the candidate is not necessarily shown as much knowledge. The grading discussions are strongly consensus-oriented and have a high level of agreement. In case of disagreement, the grade is negotiated through a step-by-step decision-making process. When examiners initially disagree, they can avoid giving support to an unwanted grade proposal to get support for their own point of view, and when they still change their minds and give in to an unwanted grade, it is justified by general (and unwritten) standards for assessment Analyzes of teachers' guidance in the classroom show that teachers use nuance in intonation to support students in finding adequate answers to questions, and how teachers and students use explicit and implicit tools to offer and seek help in group work at the lower secondary level. All parties are heavily dependent on body language or non-verbal communication to determine if teacher assistance is needed. Immediate problem solving can take place without the students' work process being unnecessarily interrupted, and the students use the teacher's movement in the classroom as a resource to ask for help with minor problems. We developed an intervention based on the CARM to improve interactional awareness among teacher students. The results from this intervention showed that there were statistically significant different intervention group (participant who performed CARM workshop before self-reflection evaluation) and control group (participants who did not perform self-reflection evaluation before CARM workshop) on self-reflection. There was also a high degree of acceptance of the course: the participants assessed the course as useful for their own development as a teacher and assessed the course content and degree of difficulty as suitable for teacher students.

The CAiTE project has developed strong pathways to impact via its partnerships with teacher education and policymakers in the Vestfold region. Our partners have been able to feed into the process of data collection, analysis, and training delivery. The research has significance for the total national competence of collaborative group work and oral examinations, which has been understudied. The research impact is significant since it underpins new, targeted, evidence-based training materials for teacher practice and professional development. The research findings have been used in communications skills training (CARM training), which was used with teachers and showed via the randomized controlled trial to be effective. Teacher students can now, early in their training, access teacher practice through engaging with authentic examples and reflect upon how their conduct affect interactional outcomes in the classroom and during oral exams. Our results have been used in different educational settings, including professional conferences, masters’ courses, and professional development for teacher moderation. We also used our results in a CAiTE end seminar in which practitioners (teachers and teacher educators) and policy makers (Utdanningsdirektoratet and Vestfold fylkeskommune) attended. Our research, and the intervention contributes to knowledge and awareness on how language and interactional conduct affects the ensuing interaction in classrooms and during oral examinations. The results are also significant for policy makers in Norwegian education policy. An emerging concern for CAiTE was to investigate actual assessment practices between teachers, examiners, and students, and compare them to existing formal and informal educational guidelines for conducting and evaluating assessments. Some of our results can justify changes and adjustments to the guidelines issued by KD/Udir. We had two meetings with Udir during the project period and have an ongoing collaboration with Vestfold County to arrange sensor moderation seminars based on our research results.

This project aims to develop, evaluate and implement a new research-based instruction method for training feedback skills in teacher education at the University College of Southeast Norway (USN). The research group comprises three researchers from USN, two academic entrepreneurs and partners from Loughborough University (UK), three partner schools in the district, praxis schools, and an advisory board of the most experienced researchers in the field. The proposed project includes a research stage and an intervention stage. The former will entail analysis of video-recorded teacher-student interactions to identify feedback practices that support learning. The intervention stage will include translation of the research findings into role-play training workshops for teacher education, using the Conversation Analytic Role-play Method (CARM). CARM workshops involve viewing and/or listening to anonymised audio or video clips and transcripts synchronously and line by line, enabling participants to live through actual practice episodes without knowing what will happen next. The participants will then role-play what they might do next to handle the situation. The workshops will be delivered prior to the teacher students' second praxis period, and we will evaluate whether these workshops have the intended effect on teacher students' learning outcomes and self-efficacy. We will evaluate our intervention with questionnaires addressing student teachers' perceived self-efficacy, insight and confidence related to the delivery of student feedback. The first questionnaire will be based on The Norwegian Teacher Self-Efficacy Scale, a standardised questionnaire used to measure teachers' self-efficacy and for which a strong factorial invariance and test-retest reliability has been demonstrated. The proposed project will implement the CARM method with the close involvement of school teachers, teacher students, teacher educators and researchers from the educational sector.

Publications from Cristin

Funding scheme:

FINNUT-Forskning og innovasjon i utdanningssektoren