Evaluation optics of the nation state: The past, present and future of public documentation.

Independent evaluation and audit is a cornerstone of Norwegian democracy, from the high-profile reports of the National Audit Office of Norway to the myriad of evaluation processes organized within ministries, directorates and civil society organizations. This system builds upon the fundamental principles of transparency and accountability, which hold that citizens should be able to see that state funds are spent in the best possible way. Yet a growing research literature shows that evaluation is currently practiced in ways that may have unfortunate side-effects, because it shifts our focus towards that which is possible to measure and visualize. The image we gain from evaluations may thus impact upon how we understand and govern society. This means that evaluation reports not only describe society, they may over time also contribute to change that which is being evaluated. This dilemma has served as the starting point for the EVALUNATION project, where we have asked: How do new evaluation methods - what we term the "optics of evaluation" - contribute to change our nation's understanding of what is valuable and important? Through interdisciplinary research based in practice-oriented document analysis and ethnographic fieldwork we have studied how evaluations are being done, what characterizes this practice field and how this field is now changing due to digitalisation and artificial intelligence. Our main sites of study have been evaluation of development assistance (foreign aid) in an historical perspective and the report-writing processes of the National Audit Office. We have in addition engaged in ongoing methodological experiments of connecting historical, document-analytical, ethnographic and digital methods as a means to better understand how evaluation documents are produced, circulated, used and stored.

EVALUNATION-prosjektets oppnådde virkninger og effekter er i hovedsak: (1) vi har fått langt mer forskningsbasert kunnskap om et underbelyst samfunnsområde av stor relevans; (2) vi har fått etablert god og varig kontakt mellom forskningsfronten og praksisfeltet på et fagområde der det til nå har vært lite forskning; (3) gjennom solid grunnforskning har vi kunnet drive tverrfaglig metodeutvikling innenfor praksis-orientert dokumentanalyse som har truffet et stort behov både i Norge og internasjonalt.

Independent evaluation of public spending is a cornerstone of contemporary democracies, including Norway. The act of documenting that desired results are achieved as planned has become both an integral part of government and public administration and also an explicit public expectation. Given the critical democratic function and widespread deployment of both evaluation and performance audit in Norway, the lack of academic attention and scholarly analysis of these phenomena is surprising. International scholarship has demonstrated critical adverse effects of evaluation and audit upon public administration through analysis of the social and organizational aspects of evaluation and audit, yet consentrated less attention on the historical and technological developments that in practice have enabled and transformed both evaluation and audit. EVALUNATION is designed to fill this research gap by making the practical tools, methods, and infrastructures of evaluation - what the PI proposes to conceptualize as the 'optics of evaluation' - the main empirical object. A key premise of EVALUNATION is that evaluation involves large amounts of texts, and that analyzing the writing, circulation, and use of these texts is indispensable for understanding the role of evaluation in society. EVALUNATION aims to contribute substantially to the establishment of a distinct methodology for the study of how large-scale document-centered institutions and processes are being transformed by digitalization. The project team and work packages are composed to enable this radical and experimental interdisciplinarity by transgressing the dichotomies of historical/contemporary, qualitative/quantitative, and material/digital. Integrated into these high academic ambitions is a detailed plan for interacting with the professional evaluation community in Norway as a means for both gaining best possible data, verifying preliminary analyses, and enabling exploitation of project results.