Combining spectral and image information in the analysis of hyperspectral imaging data

In the fascinating realm of modern biology and medicine, researchers are unlocking the secrets of biological materials through cutting-edge techniques. One such groundbreaking method gaining prominence is infrared microspectroscopic imaging. This innovative spectroscopic technique enables the chemical characterization of diverse biological materials, providing a unique biochemical fingerprint. Imagine a microscopic world where each image point captures a full infrared spectrum, spanning thousands of wavelengths. This is equivalent to a camera that can capture not only red, green and blue colors, but also colors related to specific chemical attributes such as fats, carbons, proteins and many other biochemical components. This vast amount of data, when collected from numerous samples and patients, results in substantial datasets that are later analyzed by machine learning techniques. While current applications focus on mining chemical information, the DeepHyperSpec project acknowledges the scattering aspect of light-matter interaction and seeks to revolutionize the field by integrating morphological information from infrared microspectroscopic images. DeepHyperSpec aims to combine the intricacies of morphology, scattering, and absorption features in these images, paving the way for advanced data science approaches. The project strives to deepen our understanding of the relationship between these features, leading to precise characterization and classification of biological materials. In a significant breakthrough, the project delved into the analytical description of infrared radiation scattering and absorption in real-world microscopic samples. The analytical approach relies on complex mathematical models of light-matter interaction and inevitably requires simplifications, such as assuming perfect spherical bodies. Understanding that biological cells seldom exhibit perfect spherical shapes, the project explored the impact of morphological deviations from spheres on spectroscopic results. The results affirmed the use of analytical models in the correction of Mie scattering effects, a crucial aspect in the computational retrieval of pure absorbance spectra. An exciting development emerged with the introduction of a 3D diffraction tomographic approach for infrared microspectroscopy. Leveraging deep neural networks, researchers tackled the spectroscopic inverse scattering problem, unveiling spatially resolved chemical compositions with unprecedented accuracy. This approach not only provides insights into the interior and membrane of biological cells but also explores the possibility of 3D imaging beyond the diffraction limit. This new approach has been published in a high-ranking journal. Further, our project has just unveiled a set of cutting-edge neural networks designed to tackle the complex challenge of inverse scattering. By conditioning the space of solutions both chemically and physically, we're navigating uncharted territories. We're especially excited about our hybrid models, where we've combined the power of physical models with the finesse of parameter estimation. Stay tuned, as these groundbreaking findings are set to be published soon, ushering in a new era of understanding in the world of inverse scattering.

Infrared microspectroscopic imaging is a new technique for rapid, label-free and automated diagnosis of various types of cancer. The technique is expected to enter clinical routine analysis in the years coming. The information content in infrared microspectroscopic image data is overwhelming. An infrared microspectroscopic image typically consists of several thousands to several hundred thousands of pixels, with a full infrared spectrum with several thousand frequency readings in every pixel. Today, only chemical information extracted from the spectral domain is used for classification of tissues into healthy tissue and different cancer types. While morphological information is utilized in medical image analysis of histological images without a spectral domain, the morphological information in the analysis of infrared microspectroscopic images is ignored. DeepHyperSpec will combine deep learning methods with multivariate modelling of scattering and absorption in biomedical vibrational spectroscopy in order to develop a new paradigm for the analysis of hyperspectral imaging data. The acquired knowledge and methodology will allow to fully exploit the spectral and the image domain in hyperspectral imaging data and thus substantially increase the precision, interpretability and stability of classification models. The results of DeepHyperSpec will have an impact on other fields employing hyperspectral imaging, such as geospatial hyperspectral imaging and monitoring by satellites and drones. The research will be conducted by the multidisciplinary Biospectroscopy and Data Modelling (BioSpec) Group at the Faculty of Science and Technology/Realtek, NMBU in close collaboration with four internationally renowned research teams.