Anna V. Little

Estimation of intrinsic dimensionality of samples from noisy low-dimensional manifolds in high dimensions with multiscale SVD

The problem of estimating the intrinsic dimensionality of certain point clouds is of interest in many applications in statistics and analysis of high-dimensional data sets. Our setting is the following: the points are sampled from a manifold M of dimension k, embedded in ℝD, with k ≪ D, and corrupted by D-dimensional noise. When M is a linear manifold (hyperplane), one may analyse this situation by SVD, hoping the noise would perturb the rank k covariance matrix. When M is a nonlinear manifold, SVD performed globally may dramatically overestimate the intrinsic dimensionality. We discuss a multiscale version SVD that is useful in estimating the intrinsic dimensionality of nonlinear manifolds.

Some Recent Advances in Multiscale Geometric Analysis of Point Clouds

We discuss recent work based on multiscale geometric analyis for the study of large data sets that lie in high-dimensional spaces but have low-dimensional structure. We present three applications: the first one to the estimation of intrinsic dimension of sampled manifolds, the second one to the construction of multiscale dictionaries, called Geometric Wavelets, for the analysis of point clouds, and the third one to the inference of point clouds modeled as unions of multiple planes of varying dimensions.

Multi-Resolution Geometric Analysis for Data in High Dimensions

Large data sets arise in a wide variety of applications and are often modeled as samples from a probability distribution in high-dimensional space. It is sometimes assumed that the support of such probability distribution is well approximated by a set of low intrinsic dimension, perhaps even a low-dimensional smooth manifold. Samples are often corrupted by high-dimensional noise. We are interested in developing tools for studying the geometry of such high-dimensional data sets. In particular, we present here a multiscale transform that maps high-dimensional data as above to a set of multiscale coefficients that are compressible/sparse under suitable assumptions on the data. We think of this as a geometric counterpart to multi-resolution analysis in wavelet theory: whereas wavelets map a signal (typically low dimensional, such as a one-dimensional time series or a two-dimensional image) to a set of multiscale coefficients, the geometric wavelets discussed here map points in a high-dimensional point cloud to a multiscale set of coefficients. The geometric multi-resolution analysis (GMRA) we construct depends on the support of the probability distribution, and in this sense it fits with the paradigm of dictionary learning or data-adaptive representations, albeit the type of representation we construct is in fact mildly nonlinear, as opposed to standard linear representations. Finally, we apply the transform to a set of synthetic and real-world data sets.

A Multiscale Spectral Method for Learning Number of Clusters

We propose a novel multiscale, spectral algorithm for estimating the number of clusters in a data set. Our algorithm computes the eigenvalues of the graph Laplacian iteratively for a large range of values of the scale parameter, and estimates the number of clusters from the maximal eigengap. Thus variation of the scale parameter, which usually confuses the clustering problem, is used to infer the number of clusters in a robust and efficient way. Commute distances are used to transform the distance matrix into a block-diagonal form, allowing the algorithm to succeed on irregularly shaped clusters, and the algorithm is applied to test data sets (both simulated and real-world) for method validation.