Jelani Nelson

An optimal algorithm for the distinct elements problem

We give the first optimal algorithm for estimating the number of distinct elements in a data stream, closing a long line of theoretical research on this problem begun by Flajolet and Martin in their seminal paper in FOCS 1983. This problem has applications to query optimization, Internet routing, network topology, and data mining. For a stream of indices in {1,...,n}, our algorithm computes a (1 ± ε)-approximation using an optimal O(1/ε-2 + log(n)) bits of space with 2/3 success probability, where 0<ε<1 is given. This probability can be amplified by independent repetition. Furthermore, our algorithm processes each stream update in O(1) worst-case time, and can report an estimate at any point midstream in O(1) worst-case time, thus settling both the space and time complexities simultaneously. We also give an algorithm to estimate the Hamming norm of a stream, a generalization of the number of distinct elements, which is useful in data cleaning, packet tracing, and database auditing. Our algorithm uses nearly optimal space, and has optimal O(1) update and reporting times.

OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings

An oblivious subspace embedding (OSE) given some parameters ε, d is a distribution D over matrices Π ∈ R^m×n such that for any linear subspace W ⊆ Rⁿ with dim(W) = d, P_Π~D(∀x ∈ W ||Πx||₂ ∈ (1 ± ε)||x||₂) > 2/3. We show that a certain class of distributions, Oblivious Sparse Norm-Approximating Projections (OSNAPs), provides OSEs with m = O(d^1+γ/ε²), and where every matrix Π in the support of the OSE has only s = O_γ(1/ε) non-zero entries per column, for γ > 0 any desired constant. Plugging OSNAPs into known algorithms for approximate least squares regression, ℓ_p regression, low rank approximation, and approximating leverage scores implies faster algorithms for all these problems. Our main result is essentially a Bai-Yin type theorem in random matrix theory and is likely to be of independent interest: we show that for any fixed U ∈ R^n×d with orthonormal columns and random sparse Π, all singular values of ΠU lie in [1 - ε, 1 + ε] with good probability. This can be seen as a generalization of the sparse Johnson-Lindenstrauss lemma, which was concerned with d = 1. Our methods also recover a slightly sharper version of a main result of [Clarkson-Woodruff, STOC 2013], with a much simpler proof. That is, we show that OSNAPs give an OSE with m = O(d²/ε²), s = 1.

Sparser Johnson-Lindenstrauss Transforms

We give two different and simple constructions for dimensionality reduction in <i>ℓ</i>₂ via linear mappings that are sparse: only an <i>O</i>(<i>ϵ</i>)-fraction of entries in each column of our embedding matrices are non-zero to achieve distortion 1 + <i>ϵ</i> with high probability, while still achieving the asymptotically optimal number of rows. These are the first constructions to provide subconstant sparsity for all values of parameters, improving upon previous works of Achlioptas [2003] and Dasgupta et al. [2010]. Such distributions can be used to speed up applications where <i>ℓ</i>₂ dimensionality reduction is used.

Cache-oblivious streaming B-trees

A <b><i>streaming B-tree</i></b> is a dictionary that efficiently implements insertions and range queries. We present two cache-oblivious streaming B-trees, the <b><i>shuttle tree</i></b>, and the <b><i>cache-oblivious lookahead array (COLA)</i></b>. For block-transfer size <i>B</i> and on <i>N</i> elements, the shuttle tree implements searches in optimal <i>O</i>(log _<i>B</i>+1<i>N</i>) transfers, range queries of <i>L</i> successive elements in optimal <i>O</i>(log _<i>B</i>+1<i>N</i> +<i>L/B</i>) transfers, and insertions in <i>O</i>((log _<i>B</i>+1<i>N</i>)/<i>B</i>^{Θ(1/(log log <i>B</i>)²)}+(log²<i>N</i>)/<i>B</i>) transfers, which is an asymptotic speedup over traditional B-trees if <i>B</i> ≥ (log <i>N</i>)^{1+<i>c</i> log log log² <i>N</i>} for any constant <i>c</i> >1. A COLA implements searches in <i>O</i>(log <i>N</i>) transfers, range queries in O(log <i>N</i> + <i>L/B</i>) transfers, and insertions in amortized <i>O</i>((log <i>N</i>)/<i>B</i>) transfers, matching the bounds for a (cache-aware) buffered repository tree. A partially deamortized COLA matches these bounds but reduces the worst-case insertion cost to <i>O</i>(log <i>N</i>) if memory size <i>M</i> = Ω(log <i>N</i>). We also present a cache-aware version of the COLA, the <b><i>lookahead array</i></b>, which achieves the same bounds as Brodal and Fagerbergs (cache-aware) B^ε-tree. We compare our COLA implementation to a traditional B-tree. Our COLA implementation runs 790 times faster for random inser-tions, 3.1 times slower for insertions of sorted data, and 3.5 times slower for searches.

Sketching and Streaming Entropy via Approximation Theory

We give near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain suboptimal space bounds in the general model, and near-optimal bounds in the insertion-only model without sketching. Our high-level approach is simple: we give algorithms to estimate Tsallis entropy, and use them to extrapolate an estimate of Shannon entropy. The accuracy of our estimates is proven using approximation theory arguments and extremal properties of Chebyshev polynomials. Our work also yields the best-known and near-optimal additive approximations for entropy, and hence also for conditional entropy and mutual information.

Bounded Independence Fools Degree-2 Threshold Functions

For an

Fast moment estimation in data streams in optimal space

n

Toward a Unified Theory of Sparse Dimensionality Reduction in Euclidean Space

-variate degree–

Fast Manhattan sketches in data streams

2

Almost optimal explicit Johnson-Lindenstrauss families

real polynomial