Back to search

FINNUT-Forskning og innovasjon i utdanningssektoren

Measuring competence development in vocational education and training (MECVET)

Awarded: NOK 6.1 mill.

Background. Competence based curricula and instructional models have changed vocational education and training in several ways both in Norway and in other countries: Teaching, learning trajectories and research on assessment and learning. Among these initiatives is the development of new tools for international comparisons of students and apprentices' competence development.Three approaches have been focused in recent years. One is to identify competencies in vocational education and training based on job requirements and specific work tasks. The second strategy is represented by an international initiative (which Norway also attended) that would develop a so-called PISA for vocational education and training (VET-LSA). This approach combined measurement of specific professional competencies and generic skills in jobs such as social competence. The attempt failed because the test tasks lacked relevance. The project "Measuring competence development in vocational education and training" (MECVET) based on a third approach to large-scale assessment developed in Germany (University of Bremen) under the acronym KOMET. This model aims at measuring holistic professional competencies, and test construction here differs significantly from the above approaches. While the first put together professional profiles from an international database, the KOMET platform prescribes the organization of workshops (expert workshops) and project teams with vocational experts who define the profession's core tasks. These form the basis for the development of open written test tasks completed by pupils / apprentices and rated by the judgment of professional experts supported by given criteria - a logic for large-scale assessment that is recognized in the OECD prototypes AHELO (generic skills higher education) and PIAAC (adult competence) . MECVET is conducted as a longitudinal trial of the KOMET platform for students in the 2nd year of school (N=150), apprentices after their1st year in companies (N=150) and 2nd year in companies (N=150)). The vocations are industrial mechanics, electricians and health care workers. Similar studies are conducted in Germany, China and South Africa. Results / experiences with MECVET (4 years overview) o The development of open test tasks could only to a limited extent be translated from the German versions as was done in China and South Africa. This particularly applied to the technical vocations where the revisions were done in collaboration with vocational experts. o Compared with experiences / results from Germany the variation in test conditions and student achievement is greater in the Norwegian trial, which could be attributed to the degree of voluntariness and standardization of test conditions. o The comparative data show that the Norwegian apprentices in industrial mechanics and electrical installation/maintenance have a lower score on vocational competence than similar German - especially an understanding of work processes, recycling, health/environment/security, innovation in jobs. The differences were lower in terms of technical-vocational understanding. o In the industrial mechanics and health vocations there is a distinct progress in professional competence from 1. year to 2. year as Apprentice. A similar trend is not identified for the apprentices in electrical installation/ maintenance. The progress in the other two vocations correlates with reported high quality in work place training and the provisions from the training offices. These interactions seem stronger for the Norwegian results compared with similar German (and Chinese) results. o The international profile of MECVET was reinforced by the research group succeeding in an application to the UTFORSK program (coordinated by SIU) for the development of collaboration between higher education and research connected to MECVET. In this project, HIVE, one of MECVETs partners, the South African Cape University is key participant. o The professional competence scores on the KOMET-test reflect to a large extent differences in writing skills (so called generic competence) which question the valdidity of such texts for the diagnostics of professional competence. The MECVET-results suggest that disse patterns vary strongly from one vocation to another. Dissemination activities. In the MECVET project the dissemination activities have been pursued on three main arenas: Collegial communication in Norway, collegial communication internationally and dissemination to stakeholders in schools and workplaces

Competence-based education has become a favourite model for innovations in VET and stimulated research on assessment for learning. Efforts have also been made to construct tools for large scale assessment - along three main lines. One is to identify compe tencies on the basis of job requirements and specific work tasks. The second strategy is represented by a German initiative to develop a so called PISA-VET or VET-LSA that combined the previous one with an attempt to measure cross-occupational competenci es like social skills. A third alternative is to be found in the COMET-model which is a platform for competence diagnostics that underlines the need for a comprehensive (holistic) measurement instrument in the field of VET. The COMET and the VET-LSA-proje cts differ in terms of principles for test construction. Whereas the later selects samples of pre-defined occupational tasks from a database, the procedure in the COMET-project is to construct a limited set of multidimensional tasks for each occupations in workshops. The output is open task assignments and a set-up for rating based on judgmental consensus and criterion-related guidance - a logic also to be found in recent OECD prototypes like AHELO (higher education generic skills) and PIAAC (adult comp etencies). In the MECVET we will address the methodological challenges associated with prototyping a measurement tool for competence development that builds on the COMET-model and introduce methods for validating both the problem solving strategies of tra inees and the judgments of the raters. Our study will have a cross over longitudinal design over the 2., 3. and 4. year in VET and the learning trajectories of the students/trainees will be compared with questionnaire data on background factors, vocation al commitment, school environment, and the work place as learning environment.

Funding scheme:

FINNUT-Forskning og innovasjon i utdanningssektoren