Stefan Franke

Intervention time prediction from surgical low-level tasks

OBJECTIVE Effective time and resource management in the operating room requires process information concerning the surgical procedure being performed. A major parameter relevant to the intraoperative process is the remaining intervention time. The work presented here describes an approach for the prediction of the remaining intervention time based on surgical low-level tasks. MATERIALS AND METHODS A surgical process model optimized for time prediction was designed together with a prediction algorithm. The prediction accuracy was evaluated for two different neurosurgical interventions: discectomy and brain tumor resections. A repeated random sub-sampling validation study was conducted based on 20 recorded discectomies and 40 brain tumor resections. RESULTS The mean absolute error of the remaining intervention time predictions was 13 min 24s for discectomies and 29 min 20s for brain tumor removals. The error decreases as the intervention progresses. DISCUSSION The approach discussed allows for the on-line prediction of the remaining intervention time based on intraoperative information. The method is able to handle demanding and variable surgical procedures, such as brain tumor resections. A randomized study showed that prediction accuracies are reasonable for various clinical applications. CONCLUSION The predictions can be used by the OR staff, the technical infrastructure of the OR, and centralized management. The predictions also support intervention scheduling and resource management when resources are shared among different operating rooms, thereby reducing resource conflicts. The predictions could also contribute to the improvement of surgical workflow and patient care.

Multi-perspective workflow modeling for online surgical situation models

INTRODUCTION Surgical workflow management is expected to enable situation-aware adaptation and intelligent systems behavior in an integrated operating room (OR). The overall aim is to unburden the surgeon and OR staff from both manual maintenance and information seeking tasks. A major step toward intelligent systems behavior is a stable classification of the surgical situation from multiple perspectives based on performed low-level tasks. MATERIAL AND METHODS The present work proposes a method for the classification of surgical situations based on multi-perspective workflow modeling. A model network that interconnects different types of surgical process models is described. Various aspects of a surgical situation description were considered: low-level tasks, high-level tasks, patient status, and the use of medical devices. A study with sixty neurosurgical interventions was conducted to evaluate the performance of our approach and its robustness against incomplete workflow recognition input. RESULTS A correct classification rate of over 90% was measured for high-level tasks and patient status. The device usage models for navigation and neurophysiology classified over 95% of the situations correctly, whereas the ultrasound usage was more difficult to predict. Overall, the classification rate decreased with an increasing level of input distortion. DISCUSSION Autonomous adaptation of medical devices and intelligent systems behavior do not currently depend solely on low-level tasks. Instead, they require a more general type of understanding of the surgical condition. The integration of various surgical process models in a network provided a comprehensive representation of the interventions and allowed for the generation of extensive situation descriptions. CONCLUSION Multi-perspective surgical workflow modeling and online situation models will be a significant pre-requisite for reliable and intelligent systems behavior. Hence, they will contribute to a cooperative OR environment.

OR.NET: multi-perspective qualitative evaluation of an integrated operating room based on IEEE 11073 SDC

PurposeClinical working environments have become very complex imposing many different tasks in diagnosis, medical treatment, and care procedures. During the German flagship project OR.NET, more than 50 partners developed technologies for an open integration of medical devices and IT systems in the operating room. The aim of the present work was to evaluate a large set of the proposed concepts from the perspectives of various stakeholders.MethodThe demonstration OR is focused on interventions from the head and neck surgery and was developed in close cooperation with surgeons and numerous colleagues of the project partners. The demonstration OR was qualitatively evaluated including technical as well as clinical aspects. In the evaluation, a questionnaire was used to obtain feedback from hospital operators. The clinical implications were covered by structured interviews with surgeons, anesthesiologists and OR staff.ResultsIn the present work, we qualitatively evaluate a subset of the proposed concepts from the perspectives of various stakeholders. The feedback of the clinicians indicates that there is a need for a flexible data and control integration. The hospital operators stress the need for tools to simplify risk management in openly integrated operating rooms.ConclusionThe implementation of openly integrated operating rooms will positively affect the surgeons, the anesthesiologists, the surgical nursing staff, as well as the technical personnel and the hospital operators. The evaluation demonstrated the need for OR integration technologies and identified the missing tools to support risk management and approval as the main barriers for future installments.

Rule-based medical device adaptation for the digital operating room

A workflow-driven cooperative operating room needs to be established in order to successfully unburden the surgeon and the operating room staff very time-consuming information-seeking and configuration tasks. We propose an approach towards the integration of intraoperative surgical workflow management and integration technologies. The concept of rule-based behavior is adapted to situation-aware medical devices. A prototype was implemented and experiments with sixty recorded brain tumor removal procedures were conducted to test the proposed approach. An analysis of the recordings indicated numerous applications, such as automatic display configuration, room light adaptation and pre-configuration of medical devices and systems.

Connecting the clinical IT infrastructure to a service-oriented architecture of medical devices

Abstract The new medical device communication protocol known as IEEE 11073 SDC is well-suited for the integration of (surgical) point-of-care devices, so are the established Health Level Seven (HL7) V2 and Digital Imaging and Communications in Medicine (DICOM) standards for the communication of systems in the clinical IT infrastructure (CITI). An integrated operating room (OR) and other integrated clinical environments, however, need interoperability between both domains to fully unfold their potential for improving the quality of care as well as clinical workflows. This work thus presents concepts for the propagation of clinical and administrative data to medical devices, physiologic measurements and device parameters to clinical IT systems, as well as image and multimedia content in both directions. Prototypical implementations of the derived components have proven to integrate well with systems of networked medical devices and with the CITI, effectively connecting these heterogeneous domains. Our qualitative evaluation indicates that the interoperability concepts are suitable to be integrated into clinical workflows and are expected to benefit patients and clinicians alike. The upcoming HL7 Fast Healthcare Interoperability Resources (FHIR) communication standard will likely change the domain of clinical IT significantly. A straightforward mapping to its resource model thus ensures the tenability of these concepts despite a foreseeable change in demand and requirements.

From SOMDA to application – integration strategies in the OR.NET demonstration sites

Abstract The effective development and dissemination of the open integration for the next generation of operating rooms require a comprehensive testing environment. In this paper, we present the various challenges to be addressed in demonstration applications, and we discuss the implementation approach, the foci of the demonstration sites and the evaluation efforts. Overall, the demonstrator setups have proven the feasibility of the service-oriented medical device architecture (SOMDA) and real-time approaches with a large variety of example applications. The applications demonstrate the potentials of open device interoperability. The demonstrator implementations were technically evaluated as well as discussed with many clinicians from various disciplines. However, the evaluation is still an ongoing research at the demonstration sites. Technical evaluation focused on the properties of a network of medical devices, latencies in data transmission and stability. A careful evaluation of the SOMDA design decisions and implementations are essential to a safe and reliable interoperability of integrated medical devices and information technology (IT) system in the especially critical working environment. The clinical evaluation addressed the demands of future users and stakeholders, especially surgeons, anesthesiologists, scrub nurses and hospital operators. The opinions were carefully collected to gain further insights into the potential benefits of the technology and pitfalls in future work.

Surgical instrument similarity metrics and tray analysis for multi-sensor instrument identification

A robust identification of the instrument currently used by the surgeon is crucial for the automatic modeling and analysis of surgical procedures. Various approaches for intra-operative surgical instrument identification have been presented, mostly based on radio-frequency identification (RFID) or endoscopic video analysis. A novel approach is to identify the instruments on the instrument table of the scrub nurse with a combination of video and weight information. In a previous article, we successfully followed this approach and applied it to multiple instances of an ear, nose and throat (ENT) procedure and the surgical tray used therein. In this article, we present a metric for the suitability of the instruments of a surgical tray for identification by video and weight analysis and apply it to twelve trays of four different surgical domains (abdominal surgery, neurosurgery, orthopedics and urology). The used trays were digitized at the central sterile services department of the hospital. The results illustrate that surgical trays differ in their suitability for the approach. In general, additional weight information can significantly contribute to the successful identification of surgical instruments. Additionally, for ten different surgical instruments, ten exemplars of each instrument were tested for their weight differences. The samples indicate high weight variability in instruments with identical brand and model number. The results present a new metric for approaches aiming towards intra-operative surgical instrument detection and imply consequences for algorithms exploiting video and weight information for identification purposes.

The intelligent OR: design and validation of a context-aware surgical working environment

PurposeInteroperability of medical devices based on standards starts to establish in the operating room (OR). Devices share their data and control functionalities. Yet, the OR technology rarely implements cooperative, intelligent behavior, especially in terms of active cooperation with the OR team. Technical context-awareness will be an essential feature of the next generation of medical devices to address the increasing demands to clinicians in information seeking, decision making, and human–machine interaction in complex surgical working environments.MethodsThe paper describes the technical validation of an intelligent surgical working environment for endoscopic ear–nose–throat surgery. We briefly summarize the design of our framework for context-aware system’s behavior in integrated OR and present example realizations of novel assistance functionalities. In a study on patient phantoms, twenty-four procedures were implemented in the proposed intelligent surgical working environment based on recordings of real interventions. Subsequently, the whole processing pipeline for context-awareness from workflow recognition to the final system’s behavior is analyzed.ResultsRule-based behavior that considers multiple perspectives on the procedure can partially compensate recognition errors. A considerable robustness could be achieved with a reasonable quality of the recognition. Overall, reliable reactive as well as proactive behavior of the surgical working environment can be implemented in the proposed environment.ConclusionsThe obtained validation results indicate the suitability of the overall approach. The setup is a reliable starting point for a subsequent evaluation of the proposed context-aware assistance. The major challenge for future work will be to implement the complex approach in a cross-vendor setting.

Video-based detection of device interaction in the operating room.

Abstract The establishment of modern workflow management technologies requires the integration of dated devices. The extraction of the essential device data and usage time spans is a central requirement for an integrated OR environment. Therefore, methods are required that extract such information from the output provided by older generation devices, namely video signals. We developed a four-level approach for video-based device information extraction. Usually, video streams contain all relevant patient data and device usage information. We propose an approach consisting of defining regions of interest, grabbing video signals, analyzing the signals and storing the data in a centralized and structured location. The analysis considers textual information and graphical visualization. A prototype of the analysis approach was implemented and applied to a neurosurgical case. An evaluation study was conducted to measure the performance of the approach on video recordings of real interventions. Three medical devices were considered: intraoperative ultrasound, neuro-navigation and microscope. Overall, recognition rates for device usage higher than 95% were obtained. The approach is not limited to a single surgical discipline and does not require modification of medical devices. Furthermore, the analysis of microscopic video streams expands the detectable aspects of the surgical workflow beyond the recognition of device usage.

Towards structuring contextual information for workflow-driven surgical assistance functionalities

Abstract A workflow-driven cooperative working environment needs to be established in order to successfully unburden the surgeon and the OR staff from technical configuration and information-seeking tasks. An important prerequisite for autonomous situationaware adaptation of medical devices is a comprehensive representation of the operating context regarding the surgical process and situation. We propose a hierarchical structuring of process-related and situation-related information entities and include assessment scores that intraoperative workflow information systems may provide via OR networks. The conducted experiments on the proposed assessment scores included sixty recorded brain tumour removal procedures and considered 344 distinguishable surgical situations. A comprehensive modelling of surgical situations and process context will be a significant pre-requisite for reliable autonomous adaptation of medical devices and systems in digital operating rooms.