Back to search

IKTPLUSS-IKT og digital innovasjon

Spectral Skin Modelling and Rendering for Realistic Digital Faces

Alternative title: Spektral hudmodellering og rendering for realistiske digitale ansikter

Awarded: NOK 7.9 mill.

The creation of realistic digital faces plays a very important role in digital effects for movies, for instance to create the digital double of an actor. Besides acquiring a faithful geometry of the face, recreating a realistic appearance of human skin is extremely important. In fact, human perception creates a strong revulsion toward things that appear almost human, but not quite: a phenomenon named the ?uncanny valley?. Human skin has a complex structure, with several layers containing blood vessels, connective tissues, fat and hair follicles. However, skin color is mostly due to the concentration of some molecules (skin chromophores), such as melanin and hemoglobin. Each skin chromophore can be present in two different variants, and their relative ratio has different effects on the color of the skin. A further complication is that their concentration changes with time, and depends on the emotional state. As a result, facial skin color and appearance undergo continuous changes. Our project, in collaboration with experts in computer graphics at Imperial College London, will extend the state of the art on both acquisition and rendering of facial appearance and human skin, in both static and dynamic conditions. We will use novel spectral image acquisition devices available at NTNU and at Imperial College to measure important skin data, which will be used to create a new skin appearance model for computer graphics applications, thus increasing the photo-realism of digital faces in movies and video games. Furthermore, novel techniques to estimate skin chromophores from images will be devised. Potentially, this will have additional applications in cosmetology, such as the selection of the appropriate colors to personalize makeup, or the selection of the most suitable sunscreen lotion. More important, the accurate estimation of some skin chromophores could find applications in medicine, since it has important consequences on skin cancer. The Spectraskin project started in April 2019. The Colourlab website (http://www.colourlab.no) provides the latest updates and details of the activities in the project while the research published till now can be followed on Cristin (https://app.cristin.no/projects/show.jsf?id=678057).

A longstanding goal of computer graphics is the creation of realistic digital faces. The difficulties behind this task partly lie in human perception, which creates a strong revulsion toward things that appear almost human (the uncanny-valley). Moreover, the complex structure of the skin, and the dependency of facial appearance on both the physical and emotional state, cause continuous changes in the color of facial skin. Skin color is mostly determined by the concentrations of the chromophores melanin and hemoglobin, each of them present in two different forms. Previous work focused on the estimation of some of these parameters, generally from a patch of skin in static conditions, used to synthesize the skin for the whole face. Our project will extend the state of the art on both acquisition and rendering, thanks also to the collaboration with a world-leading scholar in computer graphics, Prof. Ghosh at Imperial College London. We will use the hyper-spectral acquisition devices available at NTNU and the new multispectral light stage for facial appearance capture at Imperial College, in order to estimate skin reflectance properties and skin chromophores concentration maps, including the ratio between eumelanin and pheomelanin. The latter causes important differences in the skin appearance, and plays a role in both ultraviolet-radiation dependent and independent skin cancer. The novel image-based skin chromophores estimation method will be used along with the light stage to capture the entire face of subjects, recording also changes from static static to dynamic conditions. Furthermore, a new skin model will be derived. Overall, our work will help increasing the photo realism of digital faces in movies and video games, and could have important application also in medicine, dermatology and cosmetology.

Publications from Cristin

No publications found

No publications found

Funding scheme:

IKTPLUSS-IKT og digital innovasjon