Raimund Leitner

Multi-spectral video endoscopy system for the detection of cancerous tissue

Multi-spectral video endoscopy provides considerable potential for early stage cancer detection. Previous multi-spectral image acquisition systems were of limited use for endoscopy due to (i) the necessary spatial scanning of push-broom approaches or (ii) the impractical long switching times of liquid crystal tunable filters. Recent technological advances in the field of tuneable filters, in particular fast acousto-optical tunable filters (AOTF), make switching times below 1ms feasible. Thus, AOTFs represent a suitable technology for the acquisition of hyper-spectral image and multi-spectral video data with excellent spatial and temporal resolution. In this paper, we propose a hyper-spectral imaging endoscope using a fast AOTF synchronized with a highly sensitive EMCCD camera for the detection of cancerous tissue. The setup demonstrates that the acquisition of hyper-spectral image and multi-spectral video data is feasible and enables the augmentation of endoscopic videos with overlays indicating cancerous tissue regions. Using hyper-spectral measurements from biopsies acquired with the setup in a clinical environment it is shown that the spectral characteristic of cancerous regions is tissue dependent. Even a sophisticated classifier such as a Support Vector Machines (SVM) or a Mixture of Gaussian Classifier (MOGC) cannot generalize the discriminative information if the training set contains measurements from different tissue types (e.g. larynx vs. parotid). In contrast, a training data selection scheme that chooses similar training sets for a given test set achieves a better prediction accuracy using an approach based on a Quadratic Discriminant Classifier (QDC) with the important advantage of improved robustness and less liability to overtraining. Combined with an image registration removing motion-based acquisition artefacts, the spectral information allows the augmentation of the video stream with overlays indicating cancerous tissue regions.

High-sensitivity hyperspectral imager for biomedical video diagnostic applications

Video endoscopy allows physicians to visually inspect inner regions of the human body using a camera and only minimal invasive optical instruments. It has become an every-day routine in clinics all over the world. Recently a technological shift was done to increase the resolution from PAL/NTSC to HDTV. But, despite a vast literature on invivo and in-vitro experiments with multi-spectral point and imaging instruments that suggest that a wealth of information for diagnostic overlays is available in the visible spectrum, the technological evolution from colour to hyper-spectral video endoscopy is overdue. There were two approaches (NBI, OBI) that tried to increase the contrast for a better visualisation by using more than three wavelengths. But controversial discussions about the real benefit of a contrast enhancement alone, motivated a more comprehensive approach using the entire spectrum and pattern recognition algorithms. Up to now the hyper-spectral equipment was too slow to acquire a multi-spectral image stack at reasonable video rates rendering video endoscopy applications impossible. Recently, the availability of fast and versatile tunable filters with switching times below 50 microseconds made an instrumentation for hyper-spectral video endoscopes feasible. This paper describes a demonstrator for hyper-spectral video endoscopy and the results of clinical measurements using this demonstrator for measurements after otolaryngoscopic investigations and thorax surgeries. The application investigated here is the detection of dysplastic tissue, although hyper-spectral video endoscopy is of course not limited to cancer detection. Other applications are the detection of dysplastic tissue or polyps in the colon or the gastrointestinal tract.

SEDMI: Saliency based edge detection in multispectral images☆

Abstract Detecting edges in multispectral images is difficult because different spectral bands may contain different edges. Existing approaches calculate the edge strength of a pixel locally, based on the variation in intensity between this pixel and its neighbors. Thus, they often fail to detect the edges of objects embedded in background clutter or objects which appear in only some of the bands. We propose SEDMI, a method that aims to overcome this problem by considering the salient properties of edges in an image. Based on the observation that edges are rare events in the image, we recast the problem of edge detection into the problem of detecting events that have a small probability in a newly defined feature space. The feature space is constructed by the spatial gradient magnitude in all spectral channels. As edges are often confined to small, isolated clusters in this feature space, the edge strength of a pixel, or the confidence value that this pixel is an event with a small probability, can be calculated based on the size of the cluster to which it belongs. Experimental results on a number of multispectral data sets and a comparison with other methods demonstrate the robustness of the proposed method in detecting objects embedded in background clutter or appearing only in a few bands.

Hyperspectral video endoscope for intra-surgery tissue classification using auto-fluorescence and reflectance spectroscopy

This paper presents a hyper-spectral video endoscopy system which utilizes a combination of auto-fluorescence imaging and white-light reflectance spectroscopy for intra-surgery tissue classification. The results of the first clinical study consisting of 59 cases of otolaryngoscopic examinations and thorax surgeries are discussed in this paper. The main focus of this application is the detection of tumor tissue, although hyper-spectral video endoscopy is not limited to cancer detection. The results show that hyper-spectral video endoscopy exhibits a large potential to become an important imaging technology for medical imaging devices that provide additional diagnostic information about the tissue under investigation.

UAV-based environmental monitoring using multi-spectral imaging

Monitoring the soil composition of agricultural land is important for maximizing crop-yields. Carinthian Tech Research, Schiebel GmbH and Quest Innovations B.V. have developed a multi-spectral imaging system that is able to simultaneously capture three visible and two near infrared channels. The system was mounted on a Schiebel CAMCOPTER® S-100 UAV for data acquisition. Results show that the system is able to classify different land types and calculate vegetation indices.

A study on design of object sorting algorithms in the industrial application using hyperspectral imaging

Many industrial object-sorting applications leverage benefits of hyperspectral imaging technology. Design of object sorting algorithms is a challenging pattern recognition problem due to its multi-level nature. Objects represented by sets of pixels/spectra in hyperspectral images are to be allocated into pre-specified sorting categories. Sorting categories are often defined in terms of lower-level concepts such as material or defect types. This paper illustrates the design of two-stage sorting algorithms, learning to discriminate individual pixels/spectra and fusing the per-pixel decisions into a single per-object outcome. The paper provides a case-study on algorithm design in a real-world industrial sorting problem. Four groups of algorithms are studied varying the level of prior knowledge about the sorting problem. Apart of the sorting accuracy, the algorithm execution speed is estimated assuming an ideal implementation. Relating these two performance criteria allows us to discuss the accuracy/speed trade-off of different algorithms.

UAV based Multi-spectral Imaging System for Environmental Monitoring

Abstract An airborne multi-spectral imaging system is demonstrated for agricultural applications. A compact multi-spectral image capture system was mounted on an unmanned autonomous vehicle (UAV) and used to survey a landscape. Five spectral bands (three visible, two near infrared bands) were imaged, processed and analyzed to measure the quantity of vegetation in the scene. The UAV equipped with the multi-spectral imaging system finished the test flights successfully and the results of these experiments suggest that this type of system would be useful for agricultural applications as precision farming or crop yield management. Zusammenfassung Diese Arbeit beschreibt ein luftgestütztes multi-spektrales bildgebendes System für landwirtschaftliche Applikationen. Ein kompaktes multi-spektrales Aufnahmesystem wurde für Landschaftsuntersuchungen auf ein unbemanntes Luftfahrzeug (UAV) montiert. Fünf spektrale Kanäle (drei im sichtbaren, zwei im Nahinfrarot-Bereich) wurden aufgenommen, verarbeitet und analysiert, um den Zustand der Vegetation zu bestimmen. Das mit dem multi-spektralen bildgebenden System ausgestattete UAV absolvierte die Testflüge erfolgreich und die Ergebnisse des Experiments, zeigen dass das vorgeschlagene System für landwirtschaftliche Anwendungen wie Precision Farming und Ertragsmanagement gut geeignet ist.

Quantitative Ethylene Measurements with MOx Chemiresistive Sensors at Different Relative Air Humidities

The sensitivity of two commercial metal oxide (MOx) sensors to ethylene is tested at different relative humidities. One sensor (MiCS-5914) is based on tungsten oxide, the other (MQ-3) on tin oxide. Both sensors were found to be sensitive to ethylene concentrations down to 10 ppm. Both sensors have significant response times; however, the tungsten sensor is the faster one. Sensor models are developed that predict the concentration of ethylene given the sensor output and the relative humidity. The MQ-3 sensor model achieves an accuracy of ±9.2 ppm and the MiCS-5914 sensor model predicts concentration to ±7.0 ppm. Both sensors are more accurate for concentrations below 50 ppm, achieving ±6.7 ppm (MQ-3) and 5.7 ppm (MiCS-5914).

UAV-based multi-spectral environmental monitoring

This paper describes an airborne multi-spectral imaging system which is able to simultaneously capture three visible (400-670nm at 50% FWHM) and three near infrared channels (670-1000nm at 50% FWHM). The rst prototype was integrated in a Schiebel CAMCOPTER®S-100 VTOL (Vertical Take-O and Landing) UAV (Unmanned Aerial Vehicle) for initial test ights in spring 2010. The UAV was own over land containing various types of vegetation. A miniaturized version of the initial multi-spectral imaging system was developed in 2011 to t into a more compact UAV. The imaging system captured six bands with a minimal spatial resolution of approx. 10cm x 10cm (depending on altitude). Results show that the system is able to resist the high vibration level during ight and that the actively stabilized camera gimbal compensates for rapid roll/tilt movements of the UAV. After image registration the acquired images are stitched together for land cover mapping and ight path validation. Moreover the system is able to distinguish between dierent types of vegetation and soil. Future work will include the use of spectral imaging techniques to identify spectral features that are related to water stress, nutrient deciency and pest infestation. Once these bands have been identied, narrowband lters will be incorporated into the airborne system.

UAV-based multispectral environmental monitoring

Accurate and up-to-date soil monitoring allow farmers to take rapid, targeted action in case of nutrient deficiencies and pest infestation. This is essential for the maximization of crop-yields. This paper describes an airborne multi-spectral imaging system which is able to simultaneously capture three visible and two near infrared channels. The system was integrated in a Schiebel CAMCOPTER® S-100 UAV for data acquisition. Results show that the system is able to classify different types of vegetation by calculating vegetation indices.