Ondřej Herman

Compact hybrid real-time hyperspectral imaging system with high effective spatial, spectral, and temporal resolution

Medical endoscopes for image-guided surgery commonly use standard color image sensors, discarding any more detailed spectral information. Medical spectroscopy devices with various spectral working ranges are specialized to specific medical procedures and in general are not usable for image-guided surgery due to limitations in spatial or temporal resolution. In this paper, we present an initial demonstrator of hyperspectral endoscope, composed of two image sensors with complementing parameters. Using this hybrid approach, combining sensors with different spatial and spectral resolutions and spectral ranges, we obtain improved coverage of all the respective parameters. After digitally processing and merging the video streams, while maintaining the better features of both, we obtain an imaging system providing high effective spatial, spectral, and temporal resolution. The system is based on field programmable gate arrays. It provides real-time video output (60 Hz), which is usable for navigation during image-guided surgery. The flexible system architecture allows for an easy extension of the processing algorithms and enables minimal video signal latency. Physical dimensions and portability of the system are comparable to standard off-the-shelf medical endoscope cameras. The device can output both processed video and standard visible light video signals on one or more video outputs of the system. The resulting processed video signal obtained from the combined image sensor data greatly increases the amount of useful information available to the end user.