PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems

Perceval addresses the human perception of audio and video quality. Early in the project, a collaborative effort led to a sensory focus group, with the required transcription and summary of the recorded discussion. The established quality expressions and terminologies were carried onward in the project. In response to earlier methods that use rating scales and short stimuli, or objective metrics, a novel subjective assessment methodology was designed for long-duration quality evaluations of audiovisual content. The new method allows direct adjustments of video impairments, making it possible to study user behaviour over extended periods of time. The methodology has been thoroughly tested in different application contexts and with audio and video artifacts. Results demonstrate the influence of extended presentations on the perceived quality of audiovisual media. A large study on individual differences focused on how sensitivity to video quality impairments may very depending on personal characteristics. Related to this pursuit, a Norwegian version of a Questionnaire for measuring Cognitive Style (OSIVQ) was developed and evaluated. Subsequent eye-tracking experiments further explored the topic for viewing behaviour. Findings suggest that an individual's cognitive style can to an extent explain differences in video quality sensitivity, but expertise and interests are likely more important. Another eye-tracking experiment explored how quality assessment tasks and video artifacts can influence video saliency. Visual salience is defined by the visual attention directed at a region, and this type of study has previously been confined to still images. Under way is a new method to interpret eye-tracking data for videos. With a series of studies on perceived audiovisual synchrony, the project introduces direct investigations of the effects of audio and video quality on multi-modal perception. The studies shed light on the robustness of the perceptual system; severe distortion has little impact on the temporal integration of audio and video, although perceived synchrony does vary across content types. A temporal artifact common to videconference systems stands out as an exception; in the presence of reverberation, a change is observed in the temporal integration of short auditory and visual speech excerpts.

Today, there exists very few methodology and software to assess the complex effects of human audiovisual quality perception. Especially the combination of multi-modal perception where audio and video stimuli interact are largely unexplored. Current object ive and subjective assessment methods approach the problem mainly from a technical perspective considering frame rates, spatial resolutions and distortion artifacts. The perceptual and cognitive side of quality experience is thereby frequently ignored. M ost methods also require a controlled laboratory environment, restrict stimuli duration to a few seconds and demand full attention from assessors. This makes them inadequate for long-term evaluation of user sensations in real application scenarios where perceptual, affective and cognitive factors are of paramount importance. In response, the proposed research project aims to develop a novel approach to multi-modal quality assessment for audiovisual systems by considering cognitive, perceptual and affe ctive quality attributes. We approach the challenges with an interdisciplinary research team, consisting of researches from cognitive psychology, sensory analysis of food and multimedia technology. Although this research may be applicable to many multi-mo dal applications, we focus on two key scenarios, the World Opera project and scalable video coding. Mid-term goal of the project are robust subjective assessment methods for multi-modal quality evaluation that allow long-term evaluation sessions an requi re minimal attention from assessors. The final goal is an objective metric for multi-modal quality assessment that can be used for automated quality tests in many audiovisual applications.