Job Zwiers

Towards a reactive virtual trainer

A Reactive Virtual Trainer (RVT) is an Intelligent Virtual Agent (IVA) capable of presenting physical exercises that are to be performed by a human, monitoring the user and providing feedback at different levels. Depending on the motivation and the application context, the exercises may be general ones of fitness to improve the users physical condition, special exercises to be performed from time to time during work to prevent for example RSI, or physiotherapy exercises with medical indications. In the paper we discuss the functional and technical requirements of a framework which can be used to author specific RVT applications. The focus is on the reactivity of the RVT, manifested in natural language comments on readjusting the tempo, pointing out mistakes or rescheduling the exercises. We outline the components we have implemented so far: our animation engine, the composition of exercises from basic motions and the module for analysis of tempo in acoustic input.

A tractable hybrid ddn–pomdp approach to affective dialogue modeling for probabilistic frame-based dialogue systems

We propose a novel approach to developing a tractable affective dialogue model for probabilistic frame-based dialogue systems. The affective dialogue model, based on Partially Observable Markov Decision Process (POMDP) and Dynamic Decision Network (DDN) techniques, is composed of two main parts: the slot-level dialogue manager and the global dialogue manager. It has two new features: (1) being able to deal with a large number of slots and (2) being able to take into account some aspects of the users affective state in deriving the adaptive dialogue strategies. Our implemented prototype dialogue manager can handle hundreds of slots, where each individual slot might have hundreds of values. Our approach is illustrated through a route navigation example in the crisis management domain. We conducted various experiments to evaluate our approach and to compare it with approximate POMDP techniques and handcrafted policies. The experimental results showed that the DDN–POMDP policy outperforms three handcrafted policies when the users action error is induced by stress as well as when the observation error increases. Further, performance of the one-step look-ahead DDN–POMDP policy after optimizing its internal reward is close to state-of-the-art approximate POMDP counterparts.

Virtual meeting rooms: from observation to simulation

Much working time is spent in meetings and, as a consequence, meetings have become the subject of multidisciplinary research. Virtual Meeting Rooms (VMRs) are 3D virtual replicas of meeting rooms, where various modalities such as speech, gaze, distance, gestures and facial expressions can be controlled. This allows VMRs to be used to improve remote meeting participation, to visualize multimedia data and as an instrument for research into social interaction in meetings. This paper describes how these three uses can be realized in a VMR. We describe the process from observation through annotation to simulation and a model that describes the relations between the annotated features of verbal and non-verbal conversational behavior. As an example of social perception research in the VMR, we describe an experiment to assess human observers’ accuracy for head orientation.

Presenting in Virtual Worlds: An Architecture for a 3D Anthropomorphic Presenter

Meeting and lecture room technology is a burgeoning field. Such technology can provide real-time support for physically present participants, for online remote participation, or for offline access to meetings or lectures. Capturing relevant information from meetings or lectures is necessary to provide this kind of support. Multimedia presentation of this captured information requires a lot of attention. In this article, we focus on models and associated algorithms that steer the virtual presenters presentation animations. In our approach, we generate the presentations from a script describing the synchronization of speech, gestures, and movements. The script has also a channel devoted to presentation sheets (slides) and sheet changes, which we assume are an essential part of the presentation. To present and explain information, this 3D humanoid presenter uses output channels such as speech and animation of posture, painting, and involuntary movements

From sequential layers to distributed processes: deriving a distributed minimum weight spanning tree algorithm

Analysis and design of distributed algorithms and protocols are difficult issues. An important cause for those difficulties is the fact that the logical structure of the solution is often invisible in the actual implementation. We introduce a framework that allows for a formal treatment of the design process, from an abstract initial design to an implementation tailored to specific architectures. A combination of algebraic and axiomatic techniques is used to verify correctness of the derivation steps. This is shown by deriving an implementation of a distributed minimum weight spanning tree algorithm in the style of [GHS].

Assertional data reification proofs : surveys and perspective

In this survey we discuss three methods for program development, which incorporate data reification: VDM, Reynolds’ method, and Back’s method and develop a modest predicate transformer based framework to relate them. At first we consider partial correctness only, and discuss Reynolds’ method and a partial correctness version of VDM. Later we also consider total correctness in order to cover (part of) Back’s refinement calculus and the full notion of specification and associated refinement methods in VDM.

Modular Completeness for Communication Closed Layers

The Communication Closed Layers law is shown to be modular complete for a model related to that of Mazurkiewicz. It is shown that in a modular style of program development the CCL rule cannot be derived from simpler ones. Within a non-modular set-up the CCL rule can be derived however from a simpler independence rule and an analog of the expansion rule for process algebras. Part of this work has been supported by Esprit/BRA Project 6021 (REACT).

Supporting Engagement and Floor Control in Hybrid Meetings

Remote participants in hybrid meetings often have problems to follow what is going on in the (physical) meeting room they are connected with. This paper describes a videoconferencing system for participation in hybrid meetings. The system has been developed as a research vehicle to see how technology based on automatic real-time recognition of conversational behavior in meetings can be used to improve engagement and floor control by remote participants. The system uses modules for online speech recognition, real-time visual focus of attention as well as a module that signals who is being addressed by the speaker. A built-in keyword spotter allows an automatic meeting assistant to call the remote participants attention when a topic of interest is raised, pointing at the transcription of the fragment to help him catch-up.

Layering Techniques for Development of Parallel Systems

A process language is presented which makes a clear distinction between temporal order and causal order. This allows for several algebraic laws that are particularly interesting for the design of concurrent systems. One of these is an algebraic formulation of the communication closed layers principle by [EF82]. These laws suffice to rewrite process terms that avoid specification of temporal ordering into a unique normal form. Other transformations allow for gradually imposing temporal ordering on an already functionally correct design. The combination of such laws enables a design strategy where architecture independent designs are transformed towards a form that matches a particular implementation architecture. We apply this style of design to various distributed algorithms, including an algorithm for the ”point-in-polygon” problem transformed to a form suitable for pipelined execution on a tree network, and the Floyd-Warshall algorithm for the all-points shortest path transformed to a form suitable for execution on a SIMD architecture.

Layering and Action Refinement for Timed Systems

A separate treatment of time, temporal order and causality is used to explain action refinement for timed systems within a process algebraic setting. Communication closed layers, serializability, the SIMD model of execution, and real-time local clocks are among the phenomena that can be described and explained within the formalism. The relation with L. Lamports model of Interprocess communication is considered.