David Sun

Transparent adaptation of single-user applications for multi-user real-time collaboration

Single-user interactive computer applications are pervasive in our daily lives and work. Leveraging single-user applications for supporting multi-user collaboration has the potential to significantly increase the availability and improve the usability of collaborative applications. In this article, we report an innovative Transparent Adaptation (TA) approach and associated supporting techniques that can be used to convert existing and new single-user applications into collaborative ones, without changing the source code of the original application. The cornerstone of the TA approach is the operational transformation (OT) technique and the method of adapting the single-user application programming interface to the data and operation models of OT. This approach and supporting techniques were developed and tested in the process of transparently converting two commercial off-the-shelf single-user applications (Microsoft Word and PowerPoint) into real-time collaborative applications, called CoWord and CoPowerPoint, respectively. CoWord and CoPowerPoint not only retain the functionalities and “look-and-feel” of their single-user counterparts, but also provide advanced multi-user collaboration capabilities for supporting multiple interaction paradigms, ranging from concurrent and free interaction to sequential and synchronized interaction, and for supporting detailed workspace awareness, including multi-user telepointers and radar views. The TA approach and generic collaboration engine software component developed from this work are potentially applicable and reusable in adapting a wide range of single-user applications.

Leveraging single-user applications for multi-user collaboration: the coword approach

Single-user interactive computer applications are pervasive in our daily lives and work. Leveraging single-user applications for multi-user collaboration has the potential to significantly increase the availability and improve the usability of collaborative applications. In this paper, we report an innovative transparent adaptation approach for this purpose. The basic idea is to adapt the single-user application programming interface to the data and operational models of the underlying collaboration supporting technique, namely Operational Transformation. Distinctive features of this approach include: (1) Application transparency: it does not require access to the source code of the single-user application; (2) Unconstrained collaboration: it supports concurrent and free interaction and collaboration among multiple users; and (3) Reusable collaborative software components: collaborative software components developed with this approach can be reused in adapting a wide range of single-user applications. This approach has been applied to transparently convert MS Word into a real-time collaborative word processor, called CoWord, which supports multiple users to view and edit any objects in the same Word document at the same time over the Internet. The generality of this approach has been tested by re-applying it to convert MS PowerPoint into CoPowerPoint.

Operational transformation for collaborative word processing

Operational Transformation (OT) is a technique originally invented for supporting consistency maintenance in collaborative text editors. Word processors have much richer data types and more comprehensive operations than plain text editors. Among others, the capability of updating attributes of any types of object is an essential feature of all word processors. In this paper, we report an extension of OT for supporting a generic Update operation, in addition to Insert and Delete operations, for collaborative word processing. We focus on technical issues and solutions involved in transforming Updates for both consistency maintenance and group undo. A novel technique, called Multi-Version Single-Display (MVSD), has been devised to resolve conflict between concurrent Updates, and integrated into the framework of OT. This work has been motivated by and conducted in the CoWord project, which aims to convert MS Word into a real-time collaborative word processor without changing its source code. This OT extension is relevant not only to word processors but also to a range of interactive applications that can be modelled as editors.

Context-Based Operational Transformation in Distributed Collaborative Editing Systems

Operational Transformation (OT) is a consistency maintenance technique for collaborative editing systems-a special class of distributed applications for supporting human-computer-human interaction and collaboration over communication networks. The theory of causality has been the foundation of all prior OT systems, but it is inadequate to meet essential OT requirements in functionality and correctness. In this paper, we analyze the limitation of the causality theory, propose a novel theory of operation context as the new foundation for OT systems, and present a new OT algorithm-Context-based OT (COT)-which provides uniform and efficient solutions to both consistency maintenance and undo problems. The COT algorithm has been implemented and used for supporting a range of novel collaborative applications. The context theory and context vectors are potentially applicable to other distributed computing applications.

Operation context and context-based operational transformation

Operational Transformation (OT) is a technique for consistency maintenance and group undo, and is being applied to an increasing number of collaborative applications. The theoretical foundation for OT is crucial in determining its capability to solve existing and new problems, as well as the quality of those solutions. The theory of causality has been the foundation of all prior OT systems, but it is inadequate to capture essential correctness requirements. Past research had invented various patches to work around this problem, resulting in increasingly intricate and complicated OT algorithms. After having designed, implemented, and experimented with a series of OT algorithms, we reflected on what had been learned and set out to develop a new theoretical framework for better understanding and resolving OT problems, reducing its complexity, and supporting its continual evolution. In this paper, we report the main results of this effort: the theory of operation context and the COT (Context-based OT) algorithm. The COT algorithm is capable of supporting both do and undo of any operations at anytime, without requiring transformation functions to preserve Reversibility Property, Convergence Property 2, Inverse Properties 2 and 3. The COT algorithm is not only simpler and more efficient than prior OT control algorithms, but also simplifies the design of transformation functions. We have implemented the COT algorithm in a generic collaboration engine and used it for supporting a range of novel collaborative applications.

Object-associated telepointer for real-time collaborative document editing systems

Telepointers are a real-time groupware interface feature to indicate where other users are pointing. None of existing telepointer techniques, however, is capable of tracking the reference object and preserving the relative position in the face of dynamic content and view changes in real-time collaborative document editing systems. In this paper, we report a novel object-associated telepointer (OAT) technique with the following features in the face of dynamic content and view changes: (1) relocating telepointers to track the reference object; (2) preserving the position relative to the reference object; and (3) providing feedback of the telepointer relocation to the local user with a virtual local cursor. The key technique for supporting OAT is to extend the operational transformation technique with a generic Refer operation to support adjusting the reference object position. The OAT technique has been implemented in the CoWord system based on the transparent adaptation approach so that it can provide the OAT support to a range of realtime collaborative applications

Creative conflict resolution in realtime collaborative editing systems

Conflict is common in collaboration, and may have both negative and positive effects on collaborative work. Past research has focused on controlling negative aspects of conflict by preventing, eliminating or isolating conflicts, but done little on exploring positive aspects of conflict. In this paper, we contribute a novel creative conflict resolution (CCR) approach to address these issues in real-time collaborative editing systems. In addition to maintaining consistency, the CCR approach is able to create new results from conflicts, generate alternative solutions based on collective effects of conflict operations, and support users to choose suitable conflict solutions and conflict resolution policies according to their needs. The CCR approach provides not only a new way of resolving conflicts in real-time collaborative editing systems, but also a framework for supporting a range of existing conflict resolution strategies. Techniques and user interface issues related to the CCR approach and a prototype implementation are discussed in this paper.

A collaborative table editing technique based on transparent adaptation

Tables are an efficient means to organize information. Collaboration is important for table editing. In this paper, we report an innovative technique, called CoTable, for supporting collaborative table editing in both table-centric and word-centric complex documents. This collaborative table editing technique is based on the Transparent Adaptation approach and hence applicable to commercial off-the-shelf single-user editing applications. Key technical elements of the CoTable technique include: (1) techniques for adapting a variety of table-related data address models, accessible from the single-user Application Programming Interface (API), to that of the underlying Operational Transformation (OT) technique; and (2) techniques for translating user-level table editing operations into the primitive operations supported by OT. The CoTable technique has been implemented in the CoWord system, and CoWord-specific table processing issues and techniques are discussed in detail as well.

A high accuracy, low-latency, scalable microphone-array system for conversation analysis

Understanding and facilitating real-life social interaction is a high-impact goal for Ubicomp research. Microphone arrays offer the unique capability to provide continuous, calm capture of verbal interaction in large physical spaces, such as homes and (especially open-plan) offices. Most microphone array work has focused on arrays of custom sensors in small spaces, and a few recent works have tested small arrays of commodity sensors in single rooms. This paper describes the first working scalable and cost-effective array that offers high-precision localization of conversational speech, and hence enables ongoing studies of verbal interactions in large semi-structured spaces. This work represents significant improvements over prior work in three dimensions -- cost, scale and accuracy. It also achieves high throughput for real-time updates of tens of active sources using off-the-shelf components. We describe the system design, key localization algorithms, and a systematic performance evaluation. We then show how source location data can be usefully aggregated to reveal interesting patterns in group conversations, such as dominance and engagement.

An Integrated Session and Repository Management Approach for Real-Time Collaborative Editing Systems

Real-time collaborative editing systems may be applied in collaborative activities with a quite spontaneous and impromptu style. In such application activities, the session awareness information is particularly important. One key technical issue in achieving this flexibility is how collaboration sessions are managed in the system. However, existing approaches to session management are unable to provide sufficient session awareness information. In this paper, we propose a novel integrated repository and session management (IRSM) approach for supporting real-time collaborative editing. IRSM supports lightweight session management to significantly reduce the overhead of explicit session initialization; and detailed session awareness information through the integration of session and repository management mechanisms. In addition, IRSM is application-independent, so it can be used to support a wide range of collaborative editing systems. The feasibility and generality of this approach has been demonstrated in two real-time collaborative editing systems: CoWord and CoPowerPoint