Carsten Röcker

Extending the Design Space in Industrial Manufacturing Through Mobile Projection

This paper illustrates design opportunities for future systems in industrial manufacturing based on mobile projectors. We present a projection-based augmented reality (AR) assembly assistance system that supports users in the production process by projecting picking and assembly information into the physical workspace. Based on insights gained with a first prototype system using a stationary projector, we consider the availability of mobile projectors and the extended design space as a huge opportunity for AR applications in industrial manufacturing. Within this paper, we describe design ideas for extending our concept and for applying mobile projection in the context of industrial manufacturing.

From Smart Health to Smart Hospitals

Prolonged life expectancy along with the increasing complexity of medicine and health services raises health costs worldwide dramatically. Advancements in ubiquitous computing applications in combination with the use of sophisticated intelligent sensor networks may provide a basis for help. Whilst the smart health concept has much potential to support the concept of the emerging P4-medicine (preventive, participatory, predictive, and personalized), such high-tech medicine produces large amounts of high-dimensional, weakly-structured data sets and massive amounts of unstructured information. All these technological approaches along with “big data” are turning the medical sciences into a data-intensive science. To keep pace with the growing amounts of complex data, smart hospital approaches are a commandment of the future, necessitating context aware computing along with advanced interaction paradigms in new physical-digital ecosystems. In such a system the medical doctors are supported by their smart mobile medical assistants on managing their floods of data semi-automatically by following the human-in-the-loop concept. At the same time patients are supported by their health assistants to facilitate a healthier life, wellness and wellbeing.

Using Head-Mounted Displays and In-Situ Projection for Assistive Systems: A Comparison

The increasing demand to customize products affects production workers in many industries, as assembly tasks become more complex due to higher product variety. Assistive systems providing instructions at the workplace have been proposed to overcome increasing cognitive demand during assembly tasks. Commercially available assistive systems provide spatially registered instructions, either by using in-situ projections or head-mounted displays (HMDs). As there is little empirical knowledge about the individual advantages and disadvantages of both approaches, we are interested in comparing both types of systems. Through a user study at a manual assembly workplace, we compare both approaches to a paper baseline. Our results reveal that both in-situ instructions and paper instructions lead to significantly faster task completion times and significantly fewer errors than HMDs. Using additional questionnaires and interviews, we are able to identify the shortcomings of HMD-based instructions and discuss the possibilities of using flexible in-situ instructions for worker assistance.

The Design Space of Augmented and Virtual Reality Applications for Assistive Environments in Manufacturing: A Visual Approach

Research on how to take advantage of Augmented Reality and Virtual Reality applications and technologies in the domain of manufacturing has brought forward a great number of concepts, prototypes, and working systems. Although comprehensive surveys have taken account of the state of the art, the design space of industrial augmented and virtual reality keeps diversifying. We propose a visual approach towards assessing this space and present an interactive, community-driven tool which supports interested researchers and practitioners in gaining an overview of the aforementioned design space. Using such a framework we collected and classified relevant publications in terms of application areas and technology platforms. This tool shall facilitate initial research activities as well as the identification of research opportunities. Thus, we lay the groundwork, forthcoming workshops and discussions shall address the refinement.

User interfaces for cyber-physical systems

Abstract In this paper, we analyze the specific requirements of interacting with cyber-physical systems and propose a design approach that is driven by user needs and makes use of an expanded toolbox that contains state-of-the-art interaction technologies including Smart Glasses and Wearables. We present several examples of assistance systems in industrial production that use these interaction technologies and discuss the corresponding usability and implementation aspects.

smARt.Assembly – Projection-Based Augmented Reality for Supporting Assembly Workers

In this paper we present smARt.assembly – a projection-based augmented reality (AR) assembly assistance system for industrial applications. Our system projects digital guidance information in terms of picking information and assembly data into the physical workspace of a user. By using projections, we eliminate the use of smart glasses that have drawbacks such as a limited field of view or low wearing comfort. With smARt.assembly, users are able to assemble products without previous knowledge and without any other assistance.

User Interfaces for Cyber-Physical Systems: Challenges and Possible Approaches

Catchwords such as “Cyber-Physical-Systems” and “Industry 4.0” describe the current development of systems with embedded intelligence. These systems can be characterized by an increasing technical complexity that must be addressed in the user interface. In this paper we analyze the specific requirements posed by the interaction with cyber-physical-systems, present a coordinated approach to these requirements and illustrate our approach with a practical example of an assistance system for assembly workers in an industrial production environment.

Reasoning Under Uncertainty: Towards Collaborative Interactive Machine Learning

In this paper, we present the current state-of-the-art of decision making (DM) and machine learning (ML) and bridge the two research domains to create an integrated approach of complex problem solving based on human and computational agents. We present a novel classification of ML, emphasizing the human-in-the-loop in interactive ML (iML) and more specific on collaborative interactive ML (ciML), which we understand as a deep integrated version of iML, where humans and algorithms work hand in hand to solve complex problems. Both humans and computers have specific strengths and weaknesses and integrating humans into machine learning processes might be a very efficient way for tackling problems. This approach bears immense research potential for various domains, e.g., in health informatics or in industrial applications. We outline open questions and name future challenges that have to be addressed by the research community to enable the use of collaborative interactive machine learning for problem solving in a large scale.

A context-aware assistance system for maintenance applications in smart factories based on augmented reality and indoor localization

The term Industrie 4.0 carries the vision of smart factories, which automatically adapt to changes and assist the human as much as possible during operation and maintenance. This includes smart human machine interfaces, which reduce the chances of errors and help to make the right decisions. This paper presents an approach to equip the maintenance software running on a tablet PC with augmented reality functionality to be able to place virtual sticky notes at production modules. Additionally, these sticky notes are enriched with position information. The central element of this approach is an ontology-based context-aware framework, which aggregates and processes data from different sources. As a result, a tablet PC application was implemented, which allows displaying maintenance information as well as live plant process data in the form of augmented reality. More than 100 of those sticky notes can be placed using this system, whereas each note requires a file size of 12 to 16 kilo bytes. After placing a sticky note, the system recognizes it even if the cameras position is not exactly the same as during the placing process.

Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm

This paper presents a prototype of an intelligent assistive system for workers in stationary manual assembly using projection-based augmented reality (AR) and intelligent hand tracking. By using depth cameras, the system can track the hands of the user and makes the user aware of wrong picking actions or errors in the assembly process. The system automatically adapts the digital projection-based overlay according to the current work situation. The main research contribution of our work is the presentation of a novel hand-tracking algorithm. In addition, we present the results of an user study of the system that shows the challenges and opportunities of our system and the hand-tracking algorithm in particular. We assume that our results will inform the future design of assistive systems in manual assembly.