Michael Krug

SmartComposition: A Component-Based Approach for Creating Multi-screen Mashups

The spread and usage of mobile devices, such as smartphones or tablets, increases continuously. While most of the applications developed for these devices can only be used on the device itself, mobile devices also offer a way to create a new kind of applications: multi-screen applications. These applications run distributedly on multiple screens, like a PC, tablet, smartphone or TV. The composition of all these screens creates a new user experience for single as well as for several users. While creating mashups is a common way for designing end user interfaces, they fail in supporting multiple screens. This paper presents a component-based approach for developing multi-screen mashups, named SmartComposition. The SmartComposition approach extends the OMELETTE reference architecture to deal with multiple screens. Furthermore, we enhance the OMDL for describing multi-screen mashups platform independently. We draw up several scenarios that illustrate the opportunities of multi-screen mashups. From these scenarios we derive requirements SmartComposition needs to comply with. A huge challenge we face is the synchronization between the screens. SmartComposition solves this through real-time communication via WebSockets or Peer-to-Peer communication. We present a first prototype and evaluate our approach by developing two different multi-screen mashups. Finally, next research steps are discussed and challenges for further research are defined.

The chrooma+ approach to enrich video content using HTML5

The Internet has become an important source for media content. Content types are not limited to text and pictures but also include video and audio. Currently audiovisual media is presented as it is. However, these media do not integrate the huge amount of related information, which is available on the Web. In this paper we present the Chrooma+ approach to improve the user experience of media consumption by enriching media content with additional information from various sources in the Web. Our approach focuses on the aggregation and combination of this related information with audiovisual media. This approach involves using new HTML5 technologies and with WebVTT a new annotation format to display relevant information at definite times. Some of the advantages of this approach are the usage of a rich annotation format and extensibility to include heterogeneous information sources.

Media Enrichment on Distributed Displays by Selective Information Presentation: A First Prototype

The Internet offers a wide range of information and media content. Currently, users who are watching a video and look for related information have to search for it by themselves. In our recent work we focused on how to enrich video content with additional related information. This paper presents an approach to extend media enrichment to support the presentation of additional content on multiple distributed displays. Our approach focuses on real time synchronization between a video on one display and the presentation of related information on the same or any other display with a web browser.

SmartComposition: Extending Web Applications to Multi-screen Mashups

The overall objective of UI mashups is to enable non-experts to create rich web applications. While current approaches focus on creating UI mashups running on a single screen, we propose SmartCompositionto enable local developers to create multi-screen mashups. We present our enhanced SmartComponents, which are based on the latest developments of the family of W3C standards called “Web Components”, as part of our SmartCompositionapproach. SmartComponents provide loosely coupling and support both single- and multi-device usage scenarios by extending Web Components with dedicated communication and synchronization features. We support multiple types of SmartComponents, not limiting them to user interface components. In contrast to other approaches, SmartComponents are independent, encapsulated, configurable and programmable, which ensures hassle-free reuse in any HTML5 web application. SmartCompositionprovides an event-based communication infrastructure which enables inter-component communication as well as message exchange across multiple screens utilizing a WebSocket-based synchronization service.

AttributeLinking: Exploiting Attributes for Inter-component Communication

In this paper, we propose exploiting attributes of client-side web components to provide inter-component communication by external configuration. With the standardization of WebComponents, the Web is finally getting a uniform way to define and use client-side components. We determined that DOM elements already provide a standard configuration interface: attributes. Using the WebComponents technologies for state-of-the-art user-interface components, attributes can also act as output interfaces. By providing an Attribute-Link component, new web applications can be composed directly in the markup without knowledge of JavaScript. With the integration of a multi-device supporting Messaging-Service, components can be even linked across multiple connected devices. This enables the development of distributed user interfaces.

KESeDa: Knowledge Extraction from Heterogeneous Semi-Structured Data Sources

A large part of the free knowledge existing on the Web is available as heterogeneous, semi-structured data, which is only weakly interlinked and in general does not include any semantic classification. Due to the enormous amount of information the necessary preparation of this data for integrating it in the Web of Data requires automated processes. The extraction of knowledge from structured as well as unstructured data has already been the topic of research. But especially for the semi-structured data format JSON, which is widely used as a data exchange format e.g., in social networks, extraction solutions are missing. Based on the findings we made by analyzing existing extraction methods, we present our KESeDa approach for extracting knowledge from heterogeneous, semi-structured data sources. We show how knowledge can be extracted by describing different analysis and processing steps. With the resulting semantically enriched data the potential of Linked Data can be utilized.

The SmartComposition Approach for Creating Environment-Aware Multi-screen Mashups

Mashups aim to enable non-experts to compose complex applications by combining various existing building blocks. The special case of user interface mashups is focusing on the composition of user interface components. In this paper, we present our SmartComposition approach as an UI mashup framework that supports local developers in creating environment-aware multi-screen mashups. We aim for simplicity and focus on the definition of mashup scenarios by only using HTML markup. Therefore, we facilitate Web Component technologies to build SmartComponents – the building blocks in our approach. For achieving environment-awareness, our approach integrates features of the Web of Things into mashups, such as controlling actors and accessing sensors. SmartComposition provides mashup composition by external communication configuration through markup. We additionally propose the distribution of mashup components and their communication across multiple screens using a messaging service utilizing WebSockets.

Easing Access for Novice Users in Multi-screen Mashups by Rule-Based Adaption

Novice users often need support to become familiar with a new mashup. The most common problem is that mashups offer a high grade of personalization, such as the user’s choice which widgets she wants to use. This problem becomes more difficult in multi-screen mashups, because the user has to decide additionally on which screen the widgets should run. In our recent work we focused on creating multi-screen mashups for enriching multimedia content. That is, a user can watch a video on one screen and also can consume additional content, like a Google Maps excerpt, on another one. This paper presents an approach for rule-based adaption of multi-screen mashups to ease the access for novice users. Therefore, we analyze the users’ interaction with the mashup and detect patterns. Based on these patterns we derive rules which will be applied to the mashups of novice users as well as experienced ones. Thus, widgets will be added and arranged automatically on the user’s several screens when the execution of a previously generated rule is triggered.