Back to search

IKTPLUSS-IKT og digital innovasjon

AI4Users: Responsible Use of Artificial Intelligence through Design for Accountability and Intelligibility

Alternative title: AI4Users: Ansvarlig bruk av kunstig intelligens gjennom design (KI4Brukere)

Awarded: NOK 16.0 mill.

The AI4Users project takes a human-centred perspective for the development of tools and design principles to help users trust artificial intelligence (AI) solutions. The novelty of AI4Users is that it targets specifically non-experts extending beyond AI and data scientists. AI refers to machines performing the cognitive functions typically associated with humans, including perceiving, reasoning, and learning. It is not a specific technology but an umbrella term that includes a multitude of different technologies. For instance, AI technologies include machine learning algorithms that process data to identify patterns and make predictions or classifications. Humans are ultimately responsible for AI and they need to be involved in a non-superficial way, ensuring meaningful human control. The responsible use of AI entails ensuring intelligibility and accountability. Intelligibility means that the AI applications must be intelligible as to their state, the information they possess, their capabilities and limitations. This is linked to the overall problem of AI explainability. Accountability is another key concern and relates to the whole AI-life cycle spanning development, use and performance monitoring. The project started by reviewing extant research on accountability for AI. The analysis revealed that this body of literature is mostly conceptual with limited empirical grounding. Furthermore, it was interesting to find that the latest research engages with the same issues covered by research performed during the 1980´s (ensuring transparency, training users to understand AI limits, determining legal status and use scope). This is an indication that the same problems persist but also, points to gaps in covering emergent issues. The ongoing commoditizing machine learning and the practices of using data for purposes different from the original collection purpose creates new accountability challenges related to the interplay between data, models, and human experts. We also looked at new action possibilities offered by AI-enabled digital assistants (chatbots) for public services. Understanding new action possibilities is key for overcoming known limitations of fully automated services by leveraging AI for human augmentation. Further project activities are performed following the Action Design Research (ADR) approach. ADR is an iterative and adaptive research method closely linked to practice. The method emphasizes collaboration and is evaluation driven. The project includes a) design, prototyping and assessing tools enabling different categories of non-experts to maintain insight into AI applications, b) formalization of the design knowledge generated in actionable design principles, c) capacity building through collaborations between academia and the public sector nationally and internationally.

Infusing public services with AI solutions can contribute to efficiency and effectiveness improvements but this may come with an increase in opaqueness. This opaqueness can pose limits on involving humans in shaping, operating and monitoring the arrangements in place ensuring meaningful human control. The responsible use of AI entails ensuring algorithmic intelligibility and accountability. Intelligibility means that algorithms in use must be intelligible as to their state, the information they possess, their capabilities and limitations. Accountability means that it is possible to trace and identify responsibility for the results of algorithms. Both are required for using algorithms under human oversight. The AI4Users project will contribute to the responsible use of AI through the design and assessment of software tools and the formalisation of design principles for algorithmic accountability and intelligibility. The project takes a human-centred perspective addressing the needs of different groups implicated in AI-infused public services: citizens, case handlers at the operational level, middle managers and policy makers. The novelty of the AI4Users is that it targets specifically non-experts extending the reach of research beyond AI experts and data scientists. The use cases to be employed by the project will address different oversight scenarios including human-in-the-loop, human-on-the-loop and human-in-command. The User Organisation will be NAV and the project will be associated with NAV’s AI lab. The project research will be linked to NAV´s ongoing AI work and specific AI solutions under deployment. The project will seek access to case handlers in local NAV offices and NAV´s permanent local and national user committees (NAV Brukermedvirkning lokalt & nasjonalt). The overall aim is to advance the public infrastructures and contribute to introducing human-friendly and trustworthy artificial intelligence in practice.

Activity:

IKTPLUSS-IKT og digital innovasjon

Thematic Areas and Topics

IKT forskningsområdeKunstig intelligens, maskinlæring og dataanalysePolitikk- og forvaltningsområderDigitaliseringPortefølje Industri og tjenestenæringerLTP2 Innovasjon i stat og kommuneLTP2 Utenforskap, inkludering, kulturmøter og migrasjonBransjer og næringerIKT-næringenIKT forskningsområdeDigitalisering og bruk av IKTArbeidArbeidslivKjønnsperspektiver i forskningKjønn som perspektiv i problemstillingKjønnsperspektiver i forskningDigitalisering og bruk av IKTOffentlig sektorArbeidIKT forskningsområdeVisualisering og brukergrensesnittInternasjonaliseringInternasjonalt prosjektsamarbeidAnvendt forskningBransjer og næringerAnnen tjenesteytingPortefølje Naturvitenskap og teknologiPortefølje Humaniora og samfunnsvitenskapPortefølje Demokrati, styring og fornyelseLTP2 Et kunnskapsintensivt næringsliv i hele landetPolitikk- og forvaltningsområderLikestilling og inkluderingGrunnforskningPortefølje Velferd, kultur og samfunnLTP2 Fagmiljøer og talenterLTP2 Muliggjørende og industrielle teknologierLTP2 Samfunnssikkerhet og samhørighetLTP2 Styrket konkurransekraft og innovasjonsevneLTP2 IKT og digital transformasjonBransjer og næringerPolitikk- og forvaltningsområderNæring og handelInternasjonaliseringFornyelse og innovasjon i offentlig sektorForskning for fornyelse av offentlig sektorPolitikk- og forvaltningsområderOffentlig administrasjon og forvaltningPortefølje Muliggjørende teknologierIKT forskningsområdeMenneske, samfunn og teknologiPolitikk- og forvaltningsområderArbeidsliv - Politikk og forvaltningPolitikk- og forvaltningsområderFornyelse og innovasjon i offentlig sektorLTP2 Utvikle fagmiljøer av fremragende kvalitetPolitikk- og forvaltningsområderForskning