Allan Meng Krebs

A framework for rapid development of multimodal interfaces

Despite the availability of multimodal devices, there are very few commercial multimodal applications available. One reason for this may be the lack of a framework to support development of multimodal applications in reasonable time and with limited resources. This paper describes a multimodal framework enabling rapid development of applications using a variety of modalities and methods for ambiguity resolution, featuring a novel approach to multimodal fusion. An example application is studied that was created using the framework.

The DISCIPLE system for collaboration over the heterogeneous Web

With the proliferation of mobile devices we witness an increasing demand for supporting collaboration among users working in the field and in the office. A key component for collaboration in this domain is sharing and manipulation of information using very different display capabilities on the diverse devices. We present a system based on a distributed repository of shared data objects and a client-server based infrastructure. The system is robust to intermittent connections, and a mixture of slow and fast links. To preserve bandwidth, application-specific data distribution agents decide what data to send to the clients. We also present a framework for building collaborative applications for clients with different display and processing capabilities. We describe example applications implemented both as Java applets to run in Web browsers and as Java spotlets to run on Palm OS based handheld computers. Using these applications we evaluated the framework and the results show that the framework is scaleable, offers good performance and has a high degree of code reusability.

Designing and examining PC to Palm collaboration

One trend in day-to-day computing involves moving seamlessly from large powerful workstations to small hand-held devices. A second trend is continuous collaboration with colleagues. Combining these trends requires solutions to both the problem of transferring large complex displays to smaller, less capable devices and of ensuring that a viable collaboration takes place even when the collaborators are using vastly different tools and viewing screen environments that differ significantly in their display richness. We briefly describe an architecture for managing displays across multiple platforms, which we call the Manifold framework. This architecture is incorporated into applications using our DISCIPLE collaboration system. We explore the use of Manifold by creating a 3D layout task that communicates with a 2D version of this task running on a Palm Pilot that is wirelessly connected to the Internet. In order to get measurable data on the collaboration problems and successes that users might encounter in this diverse communication tool arrangement, we ran two separate studies that captured the performance time, user errors and transcripts of the communication exchanges between the two users. We found that interface problems with each environment affected the task performance and that the different capabilities of the 3D and 2D environments created collaborative advantages rather than negatively affecting the collaboration.

Supporting Collaboration in Heterogeneous Environments

Heterogeneous sharing in synchronous collaboration is important with the proliferation of diverse computing environments, such as wearable computers and handheld devices. We present here a data-centric design for synchronous collaboration of users with heterogeneous computing platforms. Our approach allows clients with different capabilities to share different subsets of data in order to conserve communication bandwidth. We have built a robust middleware consisting of a distributed repository of shared data objects and a client-server-based infrastructure. Using the middleware, we have developed a framework for building collaborative applications for clients with different display and processing capabilities. We discuss the design and implementation of our middleware and framework and evaluate them by building four complex sample applications that demonstrate scalability, good performance, and high degree of code reusability.

Size Does Matter in Computer Collaboration: Heterogeneous Platform Effects on Human-Human Interaction

Because todays workforce is highly mobile, small wireless devices are being used to support mobile work collaboration. However, do computer platform differences affect such collaborations? This question is investigated through a controlled experiment that examines collaborative problem solving on different combinations of small and large computers. Experiment participants in the study work together on solving 2- and 3-dimensional variations of the popular Tetris™ game. Gender is used as a moderating variable to ascertain if prior observed effects on groups of males would be found among females. The findings indicate that platform differences affect communication and social behavior among both groups. Unexpectedly, collaboration amongst partners using small handhelds was the most cooperative and friendly because of the difficulty of solving the problem with the small device.

Collaboration in parallel worlds

We present a novel paradigm for human to human asymmetric collaboration. There is a need for people at geographically separate locations to seamlessly collaborate in real time as if they are physically co-located. In our system one user (novice) works in the real world and the other user (expert) works in a parallel virtual world. They are assisted in this task by an Intelligent Agent (IA) with considerable knowledge about the environment. Current tele-collaboration systems deal primarily with collaboration purely in the real or virtual worlds. The use of a combination of virtual and real worlds allows us to leverage the advantages from both the worlds.

Asymmetric collaboration through tele-presence

A heterogeneous distributed system that enables people in geographically separate locations to share a common workspace is presented. In particular, the applicability of such a system to the notion of asymmetric collaboration is illustrated by a chess scenario. In our system one user (novice) works in the real world and the other user (expert) works in a parallel virtual world. They are assisted in this task by an Intelligent Agent (IA) with considerable knowledge about the environment.