Joseph M. Joy

DebugAdvisor: a recommender system for debugging

In large software development projects, when a programmer is assigned a bug to fix, she typically spends a lot of time searching (in an ad-hoc manner) for instances from the past where similar bugs have been debugged, analyzed and resolved. Systematic search tools that allow the programmer to express the context of the current bug, and search through diverse data repositories associated with large projects can greatly improve the productivity of debugging This paper presents the design, implementation and experience from such a search tool called DebugAdvisor. The context of a bug includes all the information a programmer has about the bug, including natural language text, textual rendering of core dumps, debugger output etc. Our key insight is to allow the programmer to collate this entire context as a query to search for related information. Thus, DebugAdvisor allows the programmer to search using a fat query, which could be kilobytes of structured and unstructured data describing the contextual information for the current bug. Information retrieval in the presence of fat queries and variegated data repositories, all of which contain a mix of structured and unstructured data is a challenging problem. We present novel ideas to solve this problem. We have deployed DebugAdvisor to over 100 users inside Microsoft. In addition to standard metrics such as precision and recall, we present extensive qualitative and quantitative feedback from our users.

Digital heritage

The India Digital Heritage Project is a collaborative initiative between the industry and academia, with the aim of using novel techniques to efficiently capture and present various aspects of Indias diverse heritage, while at the same time advancing the state-of-the art in related research areas. As part of the Digital Heritage Project, we have built a prototype virtual tour of a South Indian temple that, for the first time, integrates technologies such as Photosynth and HDView, opening up new ways to interactively explore visually complex sites. These technologies are combined with audio, video and guided walkthroughs, to provide a compelling end user experience. The accompanying video highlights the key scenarios of our prototype.

Crosslingual location search

Address geocoding, the process of finding the map location for a structured postal address, is a relatively well-studied problem. In this paper we consider the more general problem of crosslingual location search, where the queries are not limited to postal addresses, and the language and script used in the search query is different from the one in which the underlying data is stored. To the best of our knowledge, our system is the first crosslingual location search system that is able to geocode complex addresses. We use a statistical machine transliteration system to convert location names from the script of the query to that of the stored data. However, we show that it is not sufficient to simply feed the resulting transliterations into a monolingual geocoding system, as the ambiguity inherent in the conversion drastically expands the location search space and significantly lowers the quality of results. The strength of our approach lies in its integrated, end-to-end nature: we use abstraction and fuzzy search (in the text domain) to achieve maximum coverage despite transliteration ambiguities, while applying spatial constraints (in the geographic domain) to focus only on viable interpretations of the query. Our experiments with structured and unstructured queries in a set of diverse languages and scripts (Arabic, English, Hindi and Japanese) searching for locations in different regions of the world, show full crosslingual location search accuracy at levels comparable to that of commercial monolingual systems. We achieve these levels of performance using techniques that may be applied to crosslingual searches in any language/script, and over arbitrary spatial data.

Programming asynchronous layers with CLARITY

Asynchronous systems components are hard to write, hard to reason about, and (not coincidentally) hard to mechanically verify. In order to achieve high performance, asynchronous code is often written in an event-driven style that introduces non-sequential control flow and persistent heap data to track pending operations. As a result, existing sequential verification and static analysis tools are ineffective on event-driven code. We describe CLARITY, a programming language that enables analyzable design of asynchronous components. Clarity has three novel features: (1) Nonblocking function calls which allow event-driven code to be written in a sequential style. If a blocking statement is encountered during the execution of such a call, the call returns and the remainder of the operation is automatically queued for later execution. (2) Coords, a set of high-level coordination primitives, which encapsulate common interactions between asynchronous components and make high-level coordination protocols explicit. (3) Linearity annotations, which delegate coord protocol obligations to exactly one thread at each asynchronous function call, transforming a concurrent analysis problem into a sequential one. We demonstrate how these language features enable both a more intuitive expression of program logic and more effective program analysis---most checking is done using simple sequential analysis. We describe our experience in developing a network device driver with CLARITY. We are able to mechanically verify several properties of the CLARITY driver that are beyond the reach of current analysis technology applied to equivalent C code.

Robust location search from text queries

Robust, global, address geocoding is challenging because there is no single address format that applies to all geographies, and in any case, users may not restrict themselves to well-formed addresses. Particularly in online mapping systems, users frequently enter queries with missing or conflicting information, misspellings, address transpositions, and other such variations. We present a novel system which handles these difficulties by using a combination of textual similarity and spatial coherence to guide a depth-first search over the large space of possible interpretations of a text query. The system robustly matches text subsequences of a query with text attributes (i.e., any text labels associated with the entity) in a spatial-entity database. Each matched attribute is associated with the pre-computed spatial union of all entities that have that attribute. Candidate results are formed by incremental spatial intersections of these unions. Experimental results demonstrate that our system is capable of supporting regions with widely differing address formats, without region-specific customization or training. Furthermore, we show that our system significantly outperforms commercial systems for unstructured location queries and queries containing errors.

An interactive multimedia framework for digital heritage narratives

The cultural heritage of a region is conveyed by both tangible physical artifacts and intangible aspects in the form of stories, dance styles, rituals, etc. Hitherto, the task of creating digital representations for each of these aspects has been addressed in isolation, i.e. using specific media most suited to the artifact such as video, audio, three-dimensional (3D) models, scanning, etc. The challenge of bringing together these separate elements to create a coherent story, however, has remained unaddressed until recently. In this paper we present a unified digital framework that enables the integration of disparate representations of heritage elements into a holistic entity. Our approach results in a compelling and engaging narration that affords a unified user experience. Our solution supports both active (user-controlled explorations) and passive (watching pre-orchestrated narrations) user interactions. We demonstrate the capabilities of our framework through a qualitative user study based on two rich interactive narratives built using our framework: (1) history and folklore surrounding a temple in South India, and (2) a historical account of an educational institution also in South India.

The Hypercube as a Dynamically Reconfigurable Processor Mesh

We describe a technique for the efficient processing of large, multidimensional arrays on a MIMD hypercube. The technique allows the hypercube to be used as a processor mesh whose relative dimension sizes may be changeddynamically, while always keeping adjacent array elements on the same node or on physically adjacent nodes. The technique is based on a mapping scheme, calledpermuted Gray code mapping, which is a generalization of the binary reflected Gray code mapping. We also extend the technique to allowinterleavingof the array data over the nodes of the hypercube. This technique can be used to efficiently parallelize scan-line algorithms, including operations such as volume rotation and volume rendering.