Underwater Hyperspectral Imaging (UHI) is a groundbreaking innovation that expands our ability to capture a wide range of wavelengths beyond human perception. This technology offers a comprehensive view of the electromagnetic spectrum, allowing us to identify and analyze subtle differences in the spectral signatures of objects and materials. We need those features to navigate underwater effectively, especially in visually sparse environments like those beneath ice covers, where discernible features are scarce to the naked eye. In the context of underwater navigation, we rely on visual landmarks to create detailed maps of a robot's surroundings and determine its precise location within those maps. Therefore, employing a high-quality sensor like UHI can reveal numerous landmarks that are virtually imperceptible to the human eye. Our pioneering work in underwater robotics and autonomous navigation extends beyond under-ice exploration on Earth. Collaborations with NASA's Jet Propulsion Laboratory (JPL) have shed light on common sub-ice navigation challenges faced on planetary bodies with ice shields, underscoring the broader relevance of our research. Furthermore, Underwater Hyperspectral Imaging (UHI) holds significant potential for Arctic and under-ice research, offering valuable insights into ecological health and seafloor structure in these demanding environments.