Ruth Bergman

Web Interface Interpretation Using Graph Grammars

With the advent of the Internet, it is desirable to interpret and extract useful information from the Web. One major challenge in Web interface interpretation is to discover the semantic structure underlying a Web interface. Many heuristic approaches have been developed to discover and group semantically related interface objects. However, those approaches cannot solve the problem of nonuniformity satisfactorily and are not able to tag the semantic role of each object. Distinct from existing approaches, this paper develops a robust and formal approach to recovering interface semantics using graph grammars. Because of the distinct capability of spatial specifications in the abstract syntax, the spatial graph grammar (SGG) is selected to perform the semantic grouping and interpretation of segmented screen objects. Instead of analyzing HTML source codes, we apply an efficient image-processing technology to recognize atomic interface objects from the screenshot of an interface and produce a spatial graph, which records significant spatial relations among recognized objects. A spatial graph is more concise than its corresponding document object model structure and, thus, facilitates interface analysis and interpretation. Based on the spatial graph, the SGG parser recovers the hierarchical relations among interface objects.

Perceptual Segmentation: Combining Image Segmentation With Object Tagging

Human observers understand the content of an image intuitively. Based upon image content, they perform many image-related tasks, such as creating slide shows and photo albums, and organizing their image archives. For example, to select photos for an album, people assess image quality based upon the main objects in the image. They modify colors in an image based upon the color of important objects, such as sky, grass or skin. Serious photographers might modify each object separately. Photo applications, in contrast, use low-level descriptors to guide similar tasks. Typical descriptors, such as color histograms, noise level, JPEG artifacts and overall sharpness, can guide an imaging application and safeguard against blunders. However, there is a gap between the outcome of such operations and the same task performed by a person. We believe that the gap can be bridged by automatically understanding the content of the image. This paper presents algorithms for automatic tagging of perceptual objects in images, including sky, skin, and foliage, which constitutes an important step toward this goal.

Graph mining for discovering infrastructure patterns in configuration management databases

A configuration management database (CMDB) can be considered to be a large graph representing the IT infrastructure entities and their interrelationships. Mining such graphs is challenging because they are large, complex, and multi-attributed and have many repeated labels. These characteristics pose challenges for graph mining algorithms, due to the increased cost of subgraph isomorphism (for support counting) and graph isomorphism (for eliminating duplicate patterns). The notion of pattern frequency or support is also more challenging in a single graph, since it has to be defined in terms of the number of its (potentially, exponentially many) embeddings. We present CMDB-Miner, a novel two-step method for mining infrastructure patterns from CMDB graphs. It first samples the set of maximal frequent patterns and then clusters them to extract the representative infrastructure patterns. We demonstrate the effectiveness of CMDB-Miner on real-world CMDB graphs, as well as synthetic graphs.

Comprehensive solutions for automatic removal of dust and scratches from images

Dust, scratches, or hair on originals (prints, slides, or negatives) distinctly appear as light or dark artifacts on a scan. These unsightly artifacts have become a major consumer concern. There are several scenarios for removal of dust and scratch artifacts. One scenario is during acquisition, e.g., while scanning photographic media. Another is artifact removal from a digital image in an image editor. For each scenario, a different solution is suitable, with different performance requirements and differing levels of user interaction. This work describes a comprehensive set of algorithms for automatically removing dust and scratches from images. Our algorithms solve a wide range of use scenarios. A dust and scratch removal solution has two steps: a detection step and a reconstruction step. Very good detection of dust and scratches is possible using side information, such as provided by dedicated hardware. Without hardware assistance, dust and scratch removal algorithms generally resort to blurring, thereby losing image detail. We present algorithmic alternatives for dust and scratch detection. In addition, we present reconstruction algorithms that preserve image detail better than previously available alternatives. These algorithms consistently produce visually pleasing images in extensive testing.

Detection of textured areas in natural images using an indicator based on component counts

An algorithm is presented for the detection of textured areas in natural images. Texture detection has potential application to image enhancement, tone correction, defect detection, content classification, and image segmentation. For example, texture detection may be useful for object detection when combined with color models and other descriptors. Sky, e.g., is generally smooth, and foliage is textured. The texture detector presented here is based on the intuition that texture in a natural image is comprised of many components. The measure we develop examines the structure of local regions of the image. This structural approach enables us to detect both structured and unstructured textures at many scales. Furthermore, it distinguishes between edges and texture, and also between texture and noise. Automatic detection results are shown to match human classification of corresponding image areas.

Multi-source uncertain entity resolution

In this work we present a multi-source uncertain entity resolution model and show its implementation in a use case of Yad Vashem, the central repository of Holocaust-era information. The Yad Vashem dataset is unique with respect to classic entity resolution, by virtue of being both massively multi-source and by requiring multi-level entity resolution. With todays abundance of information sources, this project motivates the use of multi-source resolution on a big-data scale. We instantiate the proposed model using the MFIBlocks entity resolution algorithm and a machine learning approach, based upon decision trees to transform soft clusters into ranked clustering of records, representing possible entities. An extensive empirical evaluation demonstrates the unique properties of this dataset that make it a good candidate for multi-source entity resolution. We conclude with proposing avenues for future research in this realm. HighlightsUncertain Entity Resolution allows creating multiple narratives from complementary sources of data.The approach was demonstrated during a unique project performed on the Yad Vashem Names database.Algorithms implementing the approach were empirically evaluated on a tagged subset on various configurations and versus equivalent algorithms.The accurate and insightful results are being integrated into Yad Vashem systems and user applications.

Infrastructure Pattern Discovery in Configuration Management Databases via Large Sparse Graph Mining

A configuration management database (CMDB) can be considered to be a large graph representing the IT infrastructure entities and their inter-relationships. Mining such graphs is challenging because they are large, complex, and multi-attributed, and have many repeated labels. These characteristics pose challenges for graph mining algorithms, due to the increased cost of sub graph isomorphism (for support counting), and graph isomorphism (for eliminating duplicate patterns). The notion of pattern frequency or support is also more challenging in a single graph, since it has to be defined in terms of the number of its (potentially, exponentially many) embeddings. We present CMDB-Miner, a novel two-step method for mining infrastructure patterns from CMDB graphs. It first samples the set of maximal frequent patterns, and then clusters them to extract the representative infrastructure patterns. We demonstrate the effectiveness of CMDB-Miner on real-world CMDB graphs.

Automatic Policy Rule Extraction for Configuration Management

We propose a new IT automation technology for configuration management: automatic baseline policy extraction out of the Configuration Management Data Base (CMDB). Whereas authoring a configuration policy rule manually is time consuming and unlikely to realize the actual state of the configurations in the overall organization, this new approach summarizes the de-facto configurations from the data. IT staff, instead of authoring the policy rule, is required to simply validate and possibly enhance the automatically extracted policy. Our technology applies data-mining to organizations configuration assets in the CMDB, and automatically identifies repeating structures of compound configurations. Based on these repeating structures, we build policy rules for compound configuration items. The heart of our technique is a new distance measure we introduce between the configuration assets, whose computation is reduced to a minimum-cost flow problem.

Multi-Source Uncertain Entity Resolution at Yad Vashem: Transforming Holocaust Victim Reports into People

In this work we describe an entity resolution project performed at Yad Vashem, the central repository of Holocaust-era information. The Yad Vashem dataset is unique with respect to classic entity resolution, by virtue of being both massively multi-source and by requiring multi-level entity resolution. With todays abundance of information sources, this project sets an example for multi-source resolution on a big-data scale. We discuss a set of requirements that led us to choose the MFIBlocks entity resolution algorithm in achieving the goals of the application. We also provide a machine learning approach, based upon decision trees to transform soft clusters into ranked clustering of records, representing possible entities. An extensive empirical evaluation demonstrates the unique properties of this dataset, highlighting the shortcomings of current methods and proposing avenues for future research in this realm.

A visual tool for rapid integration of enterprise software applications

Integrating software applications is a challenging, but often very necessary, activity for businesses to perform. Even when applications are designed to fit together, creating an integrated solution often requires a significant effort in terms of configuration, fine tuning or resolving deployment conflicts. This is often the case when the original applications have been designed in isolation. This paper presents a visual method allowing an application designer to quickly integrate two products, taking the output of a sequence of steps on the first product and using that as input of a sequence of steps on the second product. The tool achieves this by: (1) copying UI components from the underlying applications user interface; (2) capturing user interaction using recording technology, rather than by relying on the underlying data sources; and (3) exposing the important business transactions that the existing application enables as macros which can then be used to integrate products together.