Tilman Becker

SmartKom: adaptive and flexible multimodal access to multiple applications

The development of an intelligent user interface that supports multimodal access to multiple applications is a challenging task. In this paper we present a generic multimodal interface system where the user interacts with an anthropomorphic personalized interface agent using speech and natural gestures. The knowledge-based and uniform approach of SmartKom enables us to realize a comprehensive system that understands imprecise, ambiguous, or incomplete multimodal input and generates coordinated, cohesive, and coherent multimodal presentations for three scenarios, currently addressing more than 50 different functionalities of 14 applications. We demonstrate the main ideas in a walk through the main processing steps from modality fusion to modality fission.

Extrinsic summarization evaluation: A decision audit task

In this work we describe a large-scale extrinsic evaluation of automatic speech summarization technologies for meeting speech. The particular task is a decision audit, wherein a user must satisfy a complex information need, navigating several meetings in order to gain an understanding of how and why a given decision was made. We compare the usefulness of extractive and abstractive technologies in satisfying this information need, and assess the impact of automatic speech recognition (ASR) errors on user performance. We employ several evaluation methods for participant performance, including post-questionnaire data, human subjective and objective judgments, and a detailed analysis of participant browsing behavior. We find that while ASR errors affect user satisfaction on an information retrieval task, users can adapt their browsing behavior to complete the task satisfactorily. Results also indicate that users consider extractive summaries to be intuitive and useful tools for browsing multimodal meeting data. We discuss areas in which automatic summarization techniques can be improved in comparison with gold-standard meeting abstracts.

Large-scale software integration for spoken language and multimodal dialog systems

The development of large-scale dialog systems requires a flexible architecture model and adequate software support to cope with the challenge of presents a general framework for building integrated natural-language and multimodal dialog systems. Our approach relies on a distributed component model that enables flexible re-use and extension of existing software modules and is able to deal with a heterogeneous software environment. A practical result of our research is the development of a sophisticated integration platform, called MULTIPLATFORM, which is based on the proposed framework. This MULTIPLATFORM testbed has been used in various large and mid-size research projects to develop integrated system prototypes.

Combining Multiple Information Layers for the Automatic Generation of Indicative Meeting Abstracts

We describe a new application for NLG technology: the generation of indicative, abstractive summaries of multi-party meetings. Based on the freely available AMI corpus of 100 hours of recorded meetings, we are developing a summarizer that uses the rich annotations in the AMI corpus.

Extrinsic Summarization Evaluation: A Decision Audit Task

In this work we describe a large-scale extrinsic evaluation of automatic speech summarization technologies for meeting speech. The particular task is a decision audit, wherein a user must satisfy a complex information need, navigating several meetings in order to gain an understanding of how and why a given decision was made. We compare the usefulness of extractive and abstractive technologies in satisfying this information need, and assess the impact of automatic speech recognition (ASR) errors on user performance. We employ several evaluation methods for participant performance, including post-questionnaire data, human subjective and objective judgments, and an analysis of participant browsing behaviour.

Big Data Usage

Big data usage covers the business goals that need access to data, its analyses, and integration into business decision-making. This chapter gives an overview of the applications of big data, focusing on decision support through big data in different sectors. Big data usage is a wide field that is addressed in this chapter by viewing data usage from various perspectives, including the underlying technology stacks, trends in various sectors, the impact on business models, and requirements on human–computer interaction. The chapter explores data usage tools, query and scripting languages, execution engines, APIs, programming models, different technology stacks, and some of the trade-offs involved are discussed. The chapter presents general aspects of decision support, followed by a discussion of specific access to analysis results through visualization and new explorative interfaces. Emerging trends and future requirements are presented with special emphasis on Industry 4.0 and the emerging need for smart data and smart services.

A Unified Approach for Semantic-Based Multimodal Interaction

This article gives an overview of the basic dialog shell for multimodal interaction that was created as the foundation for the software components developed in the Core Technology Cluster of the THESEUS research program. It explains the basic setup of systems based on the dialog shell for specific use cases. The general processing paradigm for interaction in our framework using semantic interface elements as well as several dialog phenomena handled by the dialog shell are described. Finally, a selection of different demonstrator systems that were built for a range of applications is presented briefly. They offer access to quite varied use case scenarios featuring multimodal input and output, including gestures, multi-touch, spoken input and output, and interactions across mobile and stationary devices.

Big Data-Driven Innovation in Industrial Sectors

This chapter provides the conceptual background and overview of big data-driven innovation in society. Specifically, it examines the nature of data-driven innovation, exemplars of big data-driven innovations in sectors spanning healthcare, public sector, finance, media, energy, and transport. It discusses core enablers for these innovations highlighting factors and challenges associated with the adequate diffusion, uptake, and sustainability of big data-driven initiatives. Finally, it presents policy recommendations to guide the development of a big data innovation ecosystem.

New Horizons for a Data-Driven Economy: Roadmaps and Action Plans for Technology, Businesses, Policy, and Society

This chapter describes big data roadmaps for Europe in the areas of technology, business, policy, and society. The roadmaps outline the most urgent and challenging issues for big data in Europe. They are the result of over 2 years of extensive research and input from a wide range of stakeholders from the European big data ecosystem. The roadmaps will foster the creation of a more stable big data environment by enabling enterprises, business, entrepreneurs, SMEs, and society to gain from the benefits of big data in Europe. The chapter introduces the Big Data Value Association (BDVA) and the Big Data Value contractual Public Private Partnership (BDV cPPP) and describes the role played by the BIG project in their establishment. The BDVA and the BDV cPPP will provide the necessary framework for industrial leadership, investment, and commitment of both the private and public side to build a data-driven economy across Europe.

Cross-sectorial Requirements Analysis for Big Data Research

This chapter identifies the cross-sectorial requirements for big data research necessary to define a research roadmap. The aim of the roadmap is to maximize and sustain the impact of big data technologies and applications in different industrial sectors by identifying and driving opportunities in Europe. This chapter details the process used to consolidate the big data requirements from different sectors into a single roadmap. The results comprise a prioritized set of cross-sector requirements that were used to define the technology policy, business, and society roadmaps together with action recommendations. This chapter presents a summarized description of the cross-sectorial consolidated requirements. It discusses each of the high-level and sub-level requirements together with the associated challenges that need to be tackled. Finally, the chapter concludes with a prioritization of the cross-sectorial requirements based on their expected impacts.