Alexander Hall

HyperLogLog in practice: algorithmic engineering of a state of the art cardinality estimation algorithm

Cardinality estimation has a wide range of applications and is of particular importance in database systems. Various algorithms have been proposed in the past, and the HyperLogLog algorithm is one of them. In this paper, we present a series of improvements to this algorithm that reduce its memory requirements and significantly increase its accuracy for an important range of cardinalities. We have implemented our proposed algorithm for a system at Google and evaluated it empirically, comparing it to the original HyperLogLog algorithm. Like HyperLogLog, our improved algorithm parallelizes perfectly and computes the cardinality estimate in a single pass.

Processing a trillion cells per mouse click

Column-oriented database systems have been a real game changer for the industry in recent years. Highly tuned and performant systems have evolved that provide users with the possibility of answering ad hoc queries over large datasets in an interactive manner. In this paper we present the column-oriented datastore developed as one of the central components of PowerDrill. It combines the advantages of columnar data layout with other known techniques (such as using composite range partitions) and extensive algorithmic engineering on key data structures. The main goal of the latter being to reduce the main memory footprint and to increase the efficiency in processing typical user queries. In this combination we achieve large speed-ups. These enable a highly interactive Web UI where it is common that a single mouse click leads to processing a trillion values in the underlying dataset.

Energy efficient application mapping to NoC processing elements operating at multiple voltage levels

An efficient technique for mapping application tasks to heterogeneous processing elements (PEs) on a Network-on-Chip (NoC) platform, operating at multiple voltage levels, is presented in this paper. The goal of the mapping is to minimize energy consumption subject to the performance constraints. Such a mapping involves solving several subproblems. Most of the research effort in this area often address these subproblems in a sequential fashion or a subset of them. We take a unified approach to the problem without compromising the solution time and provide techniques for optimal and heuristic solutions. We prove that the voltage assignment component of the problem itself is NP-hard and is inapproximable within any constant factor. Our optimal solution utilizes a Mixed Integer Linear Program (MILP) formulation of the problem. The heuristic utilizes MILP relaxation and randomized rounding. Experimental results based on E3S benchmark applications and a few real applications show that our heuristic produces near-optimal solution in a fraction of time needed to find the optimal.

Length-bounded cuts and flows

For a given number <i>L</i>, an <i>L</i>-length-bounded edge-cut (node-cut, respectively) in a graph <i>G</i> with source <i>s</i> and sink <i>t</i> is a set <i>C</i> of edges (nodes, respectively) such that no <i>s</i>-<i>t</i>-path of length at most <i>L</i> remains in the graph after removing the edges (nodes, respectively) in <i>C</i>. An <i>L</i>-length-bounded flow is a flow that can be decomposed into flow paths of length at most <i>L</i>. In contrast to classical flow theory, we describe instances for which the minimum <i>L</i>-length-bounded edge-cut (node-cut, respectively) is Θ(<i>n</i>^2/3)-times (Θ(&sqrt;<i>n</i>)-times, respectively) larger than the maximum <i>L</i>-length-bounded flow, where <i>n</i> denotes the number of nodes; this is the worst case. We show that the minimum length-bounded cut problem is <i>NP</i>-hard to approximate within a factor of 1.1377 for <i>L</i>≥ 5 in the case of node-cuts and for <i>L</i>≥ 4 in the case of edge-cuts. We also describe algorithms with approximation ratio <i>O</i>(min{<i>L</i>,<i>n/L</i>}) ⊆ <i>O</i>&sqrt;<i>n</i> in the node case and <i>O</i>(min {<i>L</i>,<i>n</i>²/<i>L</i>²,&sqrt;<i>m</i>} ⊆ <i>O</i>^2/3 in the edge case, where <i>m</i> denotes the number of edges. Concerning <i>L</i>-length-bounded flows, we show that in graphs with unit-capacities and general edge lengths it is <i>NP</i>-complete to decide whether there is a fractional length-bounded flow of a given value. We analyze the structure of optimal solutions and present further complexity results.

Cuts and disjoint paths in the valley-free path model

In the valley-free path model, a path in a given directed graph is valid if it consists of a sequence of forward edges followed by a sequence of backward edges. This model is motivated by routing policies of autonomous systems in the Internet. We give a 2-approximation algorithm for the problem of computing a maximum number of edge- or vertex-disjoint valid paths between two given vertices s and t, and we show that no better approximation ratio is possible unless P = NP. Furthermore, we give a 2-approximation algorithm for the problem of computing a minimum vertex cut that separates s and t with respect to all valid paths and prove that the problem is APXhard. The corresponding problem for edge cuts is shown to be polynomial-time solvable. For the multiway variant of the cut problem, we give a 4-approximation algorithm. We present additional results for acyclic graphs.

A combinatorial approach to multi-domain sketch recognition

In this paper we propose a combinatorial model for sketch recognition. Two fundamental problems, the evaluation of individual symbols and the interpretation of a complete sketch scene possibly containing several symbols, are expressed as combinatorial optimization problems. We settle the computational complexity of the combinatorial problems and present a branch and bound algorithm for computing optimal symbol confidences. To handle sketch scenes in practice we propose a modest restriction of drawing freedom and present an algorithm which only needs to compute a polynomial number of symbol confidences.

Incentive-Compatible Interdomain Routing with Linear Utilities

We revisit the problem of incentive-compatible interdomain routing, examining the quite realistic special case in which the utilities of autonomous systems (ASes) are linear functions of the traffic in the incident links and the traffic leaving each AS. We show that incentive-compatibility toward maximizing total welfare is achievable efficiently, and in the uncapacitated case, by an algorithm that can be easily implemented by the border gateway protocol (BGP), the standard protocol for interdomain routing.

The Maximum Energy-Constrained Dynamic Flow Problem

We study a natural class of flow problems that occur in the context of wireless networks; the objective is to maximize the flow from a set of sources to one sink node within a given time limit, while satisfying a number of constraints. These restrictions include capacities and transit times for edges; in addition, every node has a bound on the amount of transmission it can perform, due to limited battery energy it carries. We show that this Maximum energy-constrained dynamic flow problem(ECDF) is difficult in various ways: it is NP-hard for arbitrary transit times; a solution using flow paths can have exponential-size growth; a solution using edge flow values may not exist; and finding an integral solution is NP-hard. On the positive side, we show that the problem can be solved polynomially for uniform transit times for a limited time limit; we give an FPTAS for finding a fractional flow; and, most notably, there is a distributed FPTAS that can be run directly on the network.

Approximate discovery of random graphs

In the layered-graph query model of network discovery, a query at a node v of an undirected graph G discovers all edges and non-edges whose endpoints have different distance from v. We study the number of queries at randomly selected nodes that are needed for approximate network discovery in Erdos-Renyi random graphs Gn,p. We show that a constant number of queries is sufficient if p is a constant, while Ω(nα) queries are needed if p = ne/n, for arbitrarily small choices of e = 3/(6 ċ i + 5) with i ∈ N. Note that α > 0 is a constant depending only on e. Our proof of the latter result yields also a somewhat surprising result on pairwise distances in random graphs which may be of independent interest: We show that for a random graph Gn,p with p = ne/n, for arbitrarily small choices of e > 0 as above, in any constant cardinality subset of the nodes the pairwise distances are all identical with high probability.