Sagi Schein

Web Interface Interpretation Using Graph Grammars

With the advent of the Internet, it is desirable to interpret and extract useful information from the Web. One major challenge in Web interface interpretation is to discover the semantic structure underlying a Web interface. Many heuristic approaches have been developed to discover and group semantically related interface objects. However, those approaches cannot solve the problem of nonuniformity satisfactorily and are not able to tag the semantic role of each object. Distinct from existing approaches, this paper develops a robust and formal approach to recovering interface semantics using graph grammars. Because of the distinct capability of spatial specifications in the abstract syntax, the spatial graph grammar (SGG) is selected to perform the semantic grouping and interpretation of segmented screen objects. Instead of analyzing HTML source codes, we apply an efficient image-processing technology to recognize atomic interface objects from the screenshot of an interface and produce a spatial graph, which records significant spatial relations among recognized objects. A spatial graph is more concise than its corresponding document object model structure and, thus, facilitates interface analysis and interpretation. Based on the spatial graph, the SGG parser recovers the hierarchical relations among interface objects.

Automatic visual inspection and defect detection on variable data prints

We present a system for automatic, on-line visual inspection and print defect detection for variable data printing (VDP). This system can be used to automatically stop the printing process and alert the operator to problems. We lay out the components required for constructing a vision-based inspection system and show that our approach is novel for the high-speed detection of defects on variable data. When implemented in a high-speed digital printing press, the system allows a single skilled operator to monitor and maintain several presses, reducing the number of operators required to run a shop floor of presses as well as reduce wasted consumables when a defect goes undetected.

Visibly Pushdown Languages for a GUI Parsing Application with Probabilistic Lexer

Automatic understanding of GUI (Graphic User Interfaces) is vitally important for applications such as quality assurance, user monitoring, speech activated devices, automatic generation of GUI for application accessibility, and GUI design. Likewise, automatic understanding of visually structured documents (e.g. PDF files) is vitally important for data mining purposes. Current GUI parsers share two major shortcomings: First, instead of representing the user experience, they are tightly coupled to the underlying object model of the GUI. Second, from a linguistic point of view, they are either too restrictive to describe enough GUIs, or too permissive, in which case, the language structure itself becomes very fragile. We designed and implemented a new GUI parsing language which avoids these problems. It is easy to maintain, robust to changes in the input, and finally - as a computer program - decidable and fast to parse.

Real-time video enhancement for high quality videoconferencing

In this paper we present a novel method for high quality real-time video enhancement; it improves the sharpness and the contrast of video streams, and simultaneously suppresses noise. The method is comprised of three main modules: (1) noise analysis, (2) spatial processing, based on a new multi-scale pseudo-bilateral filter, and (3) temporal processing that includes robust motion detection and recursive temporal noise filtering. To achieve video frame rates for HD signals used in high-end telepresence systems such as the HP Halo room, we employ the computational capacity of modern graphics cards (GPUs), and distribute the tasks of analysis and of processing between CPU and GPU. The proposed scheme allows achieving video quality which is comparable with high-end camera systems, while using much lower cost cameras and reducing channel bandwidth requirements.

PDF document restoration and optimization during image enhancement

We present a document processing method that addresses some of the practical challenges in image enhancement for digital photo album in PDF documents. With the advent of digital offset presses, consumer photo books are becoming increasingly popular, and most such workflows convert the consumers photos and layout into PDF documents. In order to produce appealing photo albums from consumer photographs, some form of automatic enhancement is usually required, and this enhancement is often done late in the workflow just before printing, and therefore it is done on the PDF file. If each and every PDF generation tool simply inserted a single complete image each time an image appeared in the document, then the process of opening a PDF document, iterating through the document, extracting, enhancing, and replacing images, and then saving the enhanced document would be relatively easy. Unfortunately, PDF generation tools often violate that assumption in two ways. Firstly, large images are often written as a set of small images in strips or tiles, which visually appear to be a single image. Secondly, an image in a PDF document may be reused in the document on different position and pages; directly enhancing images without the consideration of the reuse model could result in great increase in the document size and poor system performance. Therefore, image reconstruction and document optimization were performed in our PDF photo album enhancement solution.