Multimodal 3D Holographic tool and real-time Guidance System with point-of-care diagnostics for surgical planning and interventions on liver and pancreatic cancers

Perioperative care is an intricate process that encompasses 1) preoperative evaluation, surgical planning and preparation; 2) intraoperative procedure; and 3) postoperative management. Perioperative care depends on clinical decision-making, a continuous process that involves many healthcare professionals (incl. radiologists, surgeons, anaesthesiologists, nurses, and others) who – based on the best available evidence, clinical experience, and the patient's individual circumstances – make informed decisions on the best course of action. Even though advancements in medical imaging modalities (CT, MRI, ultrasound) have greatly improved surgical outcomes, their interpretation towards deciding a specific course of action remains subjective. In addition, these modalities have several limitations when used alone (e.g., missed lesions; cannot provide a 3D visualization of key structures such as vessels), ultimately failing to provide a full picture of the patient’s anatomy that is shared and unambiguously understood by all in the assigned team. Due to these limitations, there are often suboptimal decisions that add to a cumulative risk of surgical complications in perioperative care. HoloSurge is a joint undertaking that brings European scientists and clinicians from high-tech companies, clinical centres, and research institutions in a consortium to deploy an innovative AI-based medical system that: i) enables automatic fusion and segmentation of state-of-the-art imaging modalities for interactive 3D holographic visualization of patients’ organs; ii) provides real-time surgical navigation; and iii) allows real-time cell-level cancer diagnostics. The solution will provide surgical teams with an unmatched holistic view of the patient’s anatomy and pathology in real-time at point of care, before and during surgery. This innovation will be validated for cancer interventions, while creating a trail of better-informed perioperative decision-making that can save countless lives.