Sebastian Büttner

Extending the Design Space in Industrial Manufacturing Through Mobile Projection

This paper illustrates design opportunities for future systems in industrial manufacturing based on mobile projectors. We present a projection-based augmented reality (AR) assembly assistance system that supports users in the production process by projecting picking and assembly information into the physical workspace. Based on insights gained with a first prototype system using a stationary projector, we consider the availability of mobile projectors and the extended design space as a huge opportunity for AR applications in industrial manufacturing. Within this paper, we describe design ideas for extending our concept and for applying mobile projection in the context of industrial manufacturing.

Using Head-Mounted Displays and In-Situ Projection for Assistive Systems: A Comparison

The increasing demand to customize products affects production workers in many industries, as assembly tasks become more complex due to higher product variety. Assistive systems providing instructions at the workplace have been proposed to overcome increasing cognitive demand during assembly tasks. Commercially available assistive systems provide spatially registered instructions, either by using in-situ projections or head-mounted displays (HMDs). As there is little empirical knowledge about the individual advantages and disadvantages of both approaches, we are interested in comparing both types of systems. Through a user study at a manual assembly workplace, we compare both approaches to a paper baseline. Our results reveal that both in-situ instructions and paper instructions lead to significantly faster task completion times and significantly fewer errors than HMDs. Using additional questionnaires and interviews, we are able to identify the shortcomings of HMD-based instructions and discuss the possibilities of using flexible in-situ instructions for worker assistance.

The Design Space of Augmented and Virtual Reality Applications for Assistive Environments in Manufacturing: A Visual Approach

Research on how to take advantage of Augmented Reality and Virtual Reality applications and technologies in the domain of manufacturing has brought forward a great number of concepts, prototypes, and working systems. Although comprehensive surveys have taken account of the state of the art, the design space of industrial augmented and virtual reality keeps diversifying. We propose a visual approach towards assessing this space and present an interactive, community-driven tool which supports interested researchers and practitioners in gaining an overview of the aforementioned design space. Using such a framework we collected and classified relevant publications in terms of application areas and technology platforms. This tool shall facilitate initial research activities as well as the identification of research opportunities. Thus, we lay the groundwork, forthcoming workshops and discussions shall address the refinement.

User interfaces for cyber-physical systems

Abstract In this paper, we analyze the specific requirements of interacting with cyber-physical systems and propose a design approach that is driven by user needs and makes use of an expanded toolbox that contains state-of-the-art interaction technologies including Smart Glasses and Wearables. We present several examples of assistance systems in industrial production that use these interaction technologies and discuss the corresponding usability and implementation aspects.

smARt.Assembly – Projection-Based Augmented Reality for Supporting Assembly Workers

In this paper we present smARt.assembly – a projection-based augmented reality (AR) assembly assistance system for industrial applications. Our system projects digital guidance information in terms of picking information and assembly data into the physical workspace of a user. By using projections, we eliminate the use of smart glasses that have drawbacks such as a limited field of view or low wearing comfort. With smARt.assembly, users are able to assemble products without previous knowledge and without any other assistance.

Reasoning Under Uncertainty: Towards Collaborative Interactive Machine Learning

In this paper, we present the current state-of-the-art of decision making (DM) and machine learning (ML) and bridge the two research domains to create an integrated approach of complex problem solving based on human and computational agents. We present a novel classification of ML, emphasizing the human-in-the-loop in interactive ML (iML) and more specific on collaborative interactive ML (ciML), which we understand as a deep integrated version of iML, where humans and algorithms work hand in hand to solve complex problems. Both humans and computers have specific strengths and weaknesses and integrating humans into machine learning processes might be a very efficient way for tackling problems. This approach bears immense research potential for various domains, e.g., in health informatics or in industrial applications. We outline open questions and name future challenges that have to be addressed by the research community to enable the use of collaborative interactive machine learning for problem solving in a large scale.

Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm

This paper presents a prototype of an intelligent assistive system for workers in stationary manual assembly using projection-based augmented reality (AR) and intelligent hand tracking. By using depth cameras, the system can track the hands of the user and makes the user aware of wrong picking actions or errors in the assembly process. The system automatically adapts the digital projection-based overlay according to the current work situation. The main research contribution of our work is the presentation of a novel hand-tracking algorithm. In addition, we present the results of an user study of the system that shows the challenges and opportunities of our system and the hand-tracking algorithm in particular. We assume that our results will inform the future design of assistive systems in manual assembly.

Towards a Framework for Assistance Systems to Support Work Processes in Smart Factories

Increasingly, production processes are enabled and controlled by Information Technology (IT), a development being also referred to as “Industry 4.0”. IT thereby contributes to flexible and adaptive production processes, and in this sense factories become “smart factories”. In line with this, IT also more and more supports human workers via various assistance systems. This support aims to both support workers to better execute their tasks and to reduce the effort and time required when working. However, due to the large spectrum of assistance systems, it is hard to acquire an overview and to select an adequate system for a smart factory based on meaningful criteria. We therefore synthesize a set of comparison criteria into a consistent framework and demonstrate the application of our framework by classifying three examples.

A Checklist Based Approach for Evaluating Augmented Reality Displays in Industrial Applications

The selection of suitable display technologies for industrial augmented reality (AR) applications is becoming increasingly relevant as such applications move from the proof-of-concept to the application stage. To support project managers, designers and developers in the critical selection process we have developed a checklist of important aspects and related evaluation hints that helps to speed up and improve the selection process. The checklist presented in this paper was designed to be useful for both researchers and practitioners. It combines pertinent information from relevant standards like ISO 9241-210 with results from current research literature and experience from several AR projects in industrial contexts. It can be applied both in collaboration with AR experts, where it helps to prepare relevant information for the collaboration and thus streamlines the process, or stand-alone, as a guideline for the evaluation of different options by a design team.

Managing Complexity: Towards Intelligent Error-Handling Assistance Trough Interactive Alarm Flood Reduction

The current trend of integrating machines and factories into cyber-physical systems (CPS) creates an enormous complexity for operators of such systems. Especially the search for the root cause of cascading failures becomes highly time-consuming. Within this paper, we address the question on how to help human users to better and faster understand root causes of such situations. We propose a concept of interactive alarm flood reduction and present the implementation of a first vertical prototype for such a system. We consider this prototype as a first artifact to be discussed by the research community and aim towards an incremental further development of the system in order to support humans in complex error situations.