Jianxin Luo

CVS: fast cardinality estimation for large-scale data streams over sliding windows

Abstract Estimating the cardinality of data streams over a sliding window is an important problem in many applications, such as network traffic monitoring, web access log analysis and database. The problem becomes more difficult in large-scale data streams when time and space complexity is taken into account. In this paper, we present a novel randomized data structure to address the problem. The significant contributions are as follows: (1) A space-efficient counter vector sketch (CVS) are proposed, which extends the well-known bitmap sketch to sliding window settings. (2) Based on the CVS, a random update mechanism is introduced, whereby a small fixed number of entries are randomly chosen from CVS in a step and then updated. This means that the update procedure just costs constant time. (3) Furthermore, estimating cardinality by CVS just needs one-pass scan of the data. (4) Finally, a theoretical analysis is given to show the accuracy of CVS-based estimators. Our comprehensive experiments confirm that the CVS-based schema attains high accuracy, and demonstate its time efficiency in comparison with the Timestamp Vector (TSV) and the auxiliary indexing method.

Exclusive grouped spatial hashing

Abstract A novel multidimensional hashing scheme, named the Exclusive Grouped Spatial Hashing (EGSH), which compresses repetitive spatial data into several compact tables while retaining efficient random access, is presented. EGSH represents a multi-level hashing without any losses. Moreover, EGSH compresses a group of repetitive elements into the same entry of the hash tables, while it uses a coverage table to mark the corresponding hash tables of the compressed data. Although, prior hashing work is related to hash collisions mitigation, here a full use of these collisions is obtained and therefore the spatial data compression rate is improved. The performance of exclusive grouped spatial hashing is presented in 2D and 3D graphic examples.

3D Point Cloud Generation Using Incremental Structure-from-Motion

Structure from motion is a computer vision technique of estimating camera motions and generating 3D models of the object from a sequence of multiple view images. In this paper, we describe the pipeline of structure from motion strategy especially the incremental approach and also introduce some prevalent open source software implemented structure from motion algorithms briefly. The results of our experiment depending on two different datasets show 3D point cloud and camera motions obtained from a 3D reconstruction program developed by applying bundle adjustment from scratch with the use of OpenCV library.

Fast counting the cardinality of flows for big traffic over sliding windows

Counting the cardinality of flows for massive high-speed traffic over sliding windows is still a challenging work under time and space constrains, but plays a key role in many network applications, such as traffic management and routing optimization in software defined network. In this paper, we propose a novel data structure (called LRU-Sketch) to address the problem. The significant contributions are as follows. 1) The proposed data structure adapts a well-known probabilistic sketch to sliding window model; 2) By using the least-recently used (LRU) replacement policy, we design a highly time-efficient algorithm for timely forgetting stale information, which takes constant (O(1)) time per time slot; 3) Moreover, a further memory-reducing schema is given at a cost of very little loss of accuracy; 4) Comprehensive experiments, performed on two real IP trace files, confirm that the proposed schema attains high accuracy and high time efficiency.

View-Dependent Projective Atlases.

This paper introduces View-dependent Projective Atlases (VPAs), a new algorithm for rendering height field and dynamic vector primitives in a uniform way. VPAs create projective region based on the intersections of view frustum with height field bounding boxes. This projective region is split with a dynamical and view-dependent algorithm to calculate the viewports of atlases. As an intermediate data structure, atlases are compatible with existed rasterizing based method or ray-casting method to render height field. Dynamic vector primitives can be rendered in a texture-based like way. Experiment results demonstrate that both height field and dynamic vector primitives can be rendered with high quality and efficiency.