Yinfei Pan

A Parallel Approach to XML Parsing

A language for semi-structured documents, XML has emerged as the core of the Web services architecture, and is playing crucial roles in messaging systems, databases, and document processing. However, the processing of XML documents has a reputation for poor performance, and a number of optimizations have been developed to address this performance problem from different perspectives, none of which have been entirely satisfactory. In this paper, we present a seemingly quixotic, but novel approach: parallel XML parsing. Parallel XML parsing leverages the growing prevalence of multicore architectures in all sectors of the computer market, and yields significant performance improvements. This paper presents our design and implementation of parallel XML parsing. Our design consists of an initial preparsing phase to determine the structure of the XML document, followed by a full, parallel parse. The results of the preparsing phase are used to help partition the XML document for data parallel processing. Our parallel parsing phase is a modification of the libxml2 in Veillard, D. (2004) XML parser, which shows that our approach applies to real-world, production quality parsers. Our empirical study shows our parallel XML parsing algorithm can improved the XML parsing performance significantly and scales well

A Static Load-Balancing Scheme for Parallel XML Parsing on Multicore CPUs

A number of techniques to improve the parsing performance of XML have been developed. Generally, however, these techniques have limited impact on the construction of a DOM tree, which can be a significant bottleneck. Meanwhile, the trend in hardware technology is toward an increasing number of cores per CPU. As we have shown in previous work, these cores can be used to parse XML in parallel, resulting in significant speedups. In this paper, we introduce a new static partitioning and load-balancing mechanism. By using a static, global approach, we reduce synchronization and load-balancing overhead, thus improving performance over dynamic schemes for a large class of XML documents. Our approach leverages libxm12 without modification, which reduces development effort and shows that our approach is applicable to real-world, production parsers. Our scheme works well with Suns Niagara class of CMT architectures, and shows that multiple hardware threads can be effectively used for XML parsing.

Parallel XML Parsing Using Meta-DFAs

By leveraging the growing prevalence of multicore CPUs, parallel XML parsing(PXP) can significantly improve the performance of XML, enhancing its suitability for scientific data which is often dominated by floating-point numbers. One approach is to divide the XML document into equal-sized chunks, and parse each chunk in parallel. XML parsing is inherently sequential, however, because the state of an XML parser when reading a given character depends potentially on all preceding characters. In previous work, we addressed this by using a fast preparsing scan to build an outline of the document which we called the skeleton. The skeleton is then used to guide the parallel full parse. The preparse is a sequential phase that limits scalability, however, and so in this paper, we show how the preparse itself can be parallelized using a mechanism we call a meta-DFA. For each state q of the original preparser the meta-DFA incorporates a complete copy of the preparser state machine as a sub-DFA which starts in state q. The meta-DFA thus runs multiple instances of the preparser simultaneously when parsing a chunk, with each possible preparser state at the beginning of a chunk represented by an instance. By pursuing all possibilities simultaneously, the meta-DFA allows each chunk to be preparsed independently in parallel. The parallel full parse following the preparse is performed using libxml2, and outputs DOM trees that are fully compatible with existing applications that use libxml2. Our implementation scales well on a 30 CPU Sun E6500 machine.

Simultaneous transducers for data-parallel XML parsing

Though XML has gained significant acceptance in a number of application domains, XML parsing can still be a vexing performance bottleneck. With the growing prevalence of multicore CPUs, parallel XML parsing could be one option for addressing this bottleneck. Achieving data parallelism by dividing the XML document into chunks and then independently processing all chunks in parallel is difficult, however, because the state of an XML parser at the first character of a chunk depends potentially on the characters in all preceding chunks. In previous work, we have used a sequential preparser implementing a preparsing pass to determine the document structure, followed by a parallel full parse. The preparsing is sequential, however, and thus limits speedup. In this work, we parallelize the preparsing pass itself by using a simultaneous finite transducer (SFT), which implicitly maintains multiple preparser results. Each result corresponds to starting the preparser in a different state at the beginning of the chunk. This addresses the challenge of determining the correct initial state at beginning of a chunk by simply considering all possible initial states simultaneously. Since the SFT is finite, the simultaneity can be implemented efficiently simply by enumerating the states, which limits the overhead. To demonstrate effectiveness, we use an SFT to build a parallel XML parsing implementation on an unmodified version of libxml2, and obtained good scalability on both a 30 CPU Sun E6500 machine running Solaris and a Linux machine with two Intel Xeon L5320 CPUs for a total of 8 physical cores.

Hybrid Parallelism for XML SAX Parsing

XML has been widely adopted across a wide spectrum of applications. Its parsing efficiency, however, remains a concern, and can be a bottleneck. At the same time, with the trend towards multicore CPUs, parallelization to improve performance has become increasingly relevant. In previous work, we have investigated parallelizing DOM-style parsing and gained significant speedup. For streaming XML applications, however, SAX-style parsing is often required. In this paper, we present a technique and implementation of a parallel XML SAX parser. To handle inherent data dependencies in XML while still allowing reasonable scalability, we use a 4-stage software pipeline with a combination of strictly sequential stages and stages that can be further data-parallelized within the stage. We thus utilize a hybrid between pipelined parallelism and data parallelism. To demonstrate effectiveness, we test this approach on a Linux machine with two Intel Xeon L5320 CPUs for a total of 8 physical cores, and obtain good speedup up to about 8 CPUs.

Speculative p-DFAs for parallel XML parsing

XML has seen wide acceptance in a number of application domains, and contributed to the success of wide-scale grid and scientific computing environments. Performance, however, is still an issue, and limits adoption under some situations where it might otherwise be able to provide significant interoperability, flexibility, and extensibility. As CPUs increasingly have multiple cores, parallel XML parsing can help to address this concern. This paper explores the use of speculation to improve the performance of parallel XML parsing. Building on previous work, we use an initial preparsing stage to build a sketch of the document which we called the skeleton. This skeleton contains enough information so that we can then proceed to do the full parse in parallel using unmodified libxml2. The preparsing itself is parallelized using product machines which we call p-DFAs. During execution, unlikely possibilities are discarded in favor of more likely ones. Statistics are gathered to decide which possibilities are not likely. The results show good performance and scalability on both a 30 CPU Sun E6500 machine running Solaris and a Linux machine with two Intel Xeon L5320 CPUs for a total of 8 physical cores.

A High Performance Schema-Specific XML Parser

Performance of XML parsers with validation are usually suffer. This is because such parsers need first parsing and undertanding XML schemas, and thus are limited by the very complexity of XML schemas. Schema-specific approach, however, may adjust such problem. In this paper, we introduce a high performance SAX like validating XML parser using a schema-specific approach. In this approach, a schema compiler first transforms the schema into an intermediate representation, called generalized automata, which abstracts the computations required to parse XML documents as well as validate them against a schema. The generalized automaton is then translated to a schema specific parser, which is capable of parsing and validating XML documents with namespaces through a schema specific modified SAX API. Our performance evaluation shows good results when compared with other validating parsers.

A Parallel XPath Engine Based on Concurrent NFA Execution

The importance of XPath in XML filtering systems has led to a significant body of research on improving the processing performance of XPath queries. Most of the work, however, has been in the context of a single processing core. Given the prevalence of multicore processors, we believe that a parallel approach can provide significant benefits for a number of application scenarios. In this paper we thus investigate the use of multiple threads to concurrently process XPath queries on a shared incoming XML document. Using an approach that builds on YFilter, we divide the NFA into several smaller ones for concurrent processing. We implement and test two strategies for load balancing: a static approach and a dynamic approach. We test our approach on an eight-core machine, and show that it provides reasonable speedup up to eight cores.

Parsing XML using parallel traversal of streaming trees

XML has been widely adopted across a wide spectrum of applications.Its parsing efficiency, however, remains a concern, and can be a bottleneck.With the current trend towards multicore CPUs, parallelization to improve performanceis increasingly relevant. In many applications, the XML is streamedfrom the network, and thus the complete XML document is never in memory atany single moment in time. Parallel parsing of such a stream can be equated toparallel depth-first traversal of a streaming tree. Existing research on parallel treetraversal has assumed the entire tree was available in-memory, and thus cannotbe directly applied. In this paper we investigate parallel, SAX-style parsing ofXML via a parallel, depth-first traversal of the streaming document. We showgood scalability up to about 6 cores on a Linux platform.