Jeremy Kepner

Detecting Clusters of Galaxies in the Sloan Digital Sky Survey I : Monte Carlo Comparison of Cluster Detection Algorithms

We present a comparison of three cluster-finding algorithms from imaging data using Monte Carlo simulations of clusters embedded in a 25 deg2 region of Sloan Digital Sky Survey (SDSS) imaging data: the matched filter (MF; Postman et al., published in 1996), the adaptive matched filter (AMF; Kepner et al., published in 1999), and a color-magnitude filtered Voronoi tessellation technique (VTT). Among the two matched filters, we find that the MF is more efficient in detecting faint clusters, whereas the AMF evaluates the redshifts and richnesses more accurately, therefore suggesting a hybrid method (HMF) that combines the two. The HMF outperforms the VTT when using a background that is uniform, but it is more sensitive to the presence of a nonuniform galaxy background than is the VTT; this is due to the assumption of a uniform background in the HMF model. We thus find that for the detection thresholds we determine to be appropriate for the SDSS data, the performance of both algorithms are similar; we present the selection function for each method evaluated with these thresholds as a function of redshift and richness. For simulated clusters generated with a Schechter luminosity function (M = -21.5 and α = -1.1), both algorithms are complete for Abell richness 1 clusters up to z ~ 0.4 for a sample magnitude limited to r = 21. While the cluster parameter evaluation shows a mild correlation with the local background density, the detection efficiency is not significantly affected by the background fluctuations, unlike previous shallower surveys.

The Delayed Formation of Dwarf Galaxies

One of the largest uncertainties in understanding the effect of a background UV field on galaxy formation is the intensity and evolution of the radiation field with redshift. This work attempts to shed light on this issue by computing the quasi-hydrostatic equilibrium states of gas in spherically symmetric dark matter halos (roughly corresponding to dwarf galaxies) as a function of the amplitude of the background UV field. We integrate the full equations of radiative transfer, heating, cooling, and nonequilibrium chemistry for nine species: H, H+, H-, H2, H+2, He, He+, He++, and e-. As the amplitude of the UV background is decreased, the gas in the core of the dwarf goes through three stages characterized by the predominance of ionized (H+), neutral (H), and molecular (H2) hydrogen. Characterizing the gas state of a dwarf galaxy with the radiation field allows us to estimate its behavior for a variety of models of the background UV flux. Our results indicate that a typical radiation field can easily delay the collapse of gas in halos corresponding to 1 σ cold dark matter perturbations with circular velocities of less than 30 km s-1.

Dynamic distributed dimensional data model (D4M) database and computation system

A crucial element of large web companies is their ability to collect and analyze massive amounts of data. Tuple store databases are a key enabling technology employed by many of these companies (e.g., Google Big Table and Amazon Dynamo). Tuple stores are highly scalable and run on commodity clusters, but lack interfaces to support efficient development of mathematically based analytics. D4M (Dynamic Distributed Dimensional Data Model) has been developed to provide a mathematically rich interface to tuple stores (and structured query language “SQL” databases). D4M allows linear algebra to be readily applied to databases. Using D4M, it is possible to create composable analytics with significantly less effort than using traditional approaches. This work describes the D4M technology and its application and performance.

pMatlab Parallel Matlab Library

MATLAB® has emerged as one of the languages most commonly used by scientists and engineers for technical computing, with approximately one million users worldwide. The primary benefits of MATLAB are reduced code development time via high levels of abstractions (e.g. first class multi-dimensional arrays and thousands of built in functions), interpretive, interactive programming, and powerful mathematical graphics. The compute intensive nature of technical computing means that many MATLAB users have codes that can significantly benefit from the increased performance offered by parallel computing. pMatlab provides this capability by implementing parallel global array semantics using standard operator overloading techniques. The core data structure in pMatlab is a distributed numerical array whose distribution onto multiple processors is specified with a “map” construct. Communication operations between distributed arrays are abstracted away from the user and pMatlab transparently supports redistribution between any block-cyclic-overlapped distributions up to four dimensions. pMatlab is built on top of the MatlabMPI communication library and runs on any combination of heterogeneous systems that support MATLAB, which includes Windows, Linux, MacOS X, and SunOS. This paper describes the overall design and architecture of the pMatlab implementation. Performance is validated by implementing the HPC Challenge benchmark suite and comparing pMatlab performance with the equivalent C+MPI codes. These results indicate that pMatlab can often achieve comparable performance to C+MPI, usually at one tenth the code size. Finally, we present implementation data collected from a sample of real pMatlab applications drawn from the approximately one hundred users at MIT Lincoln Laboratory. These data indicate that users are typically able to go from a serial code to an efficient pMatlab code in about 3 hours while changing less than 1% of their code.

The BigDAWG Polystore System

This paper presents a new view of federated databases to address the growing need for managing information that spans multiple data models. This trend is fueled by the proliferation of storage engines and query languages based on the observation that â no one size fits allâ . To address this shift, we propose a polystore architecture; it is designed to unify querying over multiple data models. We consider the challenges and opportunities associated with polystores. Open questions in this space revolve around query optimization and the assignment of objects to storage engines. We introduce our approach to these topics and discuss our prototype in the context of the Intel Science and Technology Center for Big Data

The evolutionary status of the stellar population in the rho Ophiuchi cloud core

This contribution reports the results of an infrared imaging survey aimed at characterizing the stellar populations associated with the three densest star-forming cores in the Ophiuchus molecular cloud complex. The survey has sufficient sensitivity at J, H, and K (at 5 sigma limits of 16.5, 15.4, and 14.2) to provide a complete census of embedded young stellar objects (YSOs) with masses greater than the hydrogen-burning limit, provided that their ages are less than 3 Myr and that they are obscured by no more than approximately 18 mag of visual extinction. Our data suggest (1) a large fraction (greater than 70%) of the sources located within the cores are still surrounded by circumstellar disks and/or envelopes; and (2) the shape of the initial mass function for masses, M less than 1 solar mass, appears to be consistent with that derived from the solar neighborhood. We also report the results of a deeper imaging survey of centimeter continuum sources (14 sources) in these star-forming cores and in the larger Ophiuchus complex (eight sources). A large fraction (11/14) of the radio sources associated with the cores appear to have infrared excesses diagnostic of circumstellar accretion disks and/or infalling circumstellar envelopes. In these cases, the centimeter continuum radiation most likely diagnoses the ionized component of energetic winds or jets which characterizes YSOs during the disk accretion phase. By contrast, of the eight radio sources located outside dense cores, only two show infrared excesses.

HPC Productivity: An Overarching View

The Defense Advanced Research Projects Agency (DARPA) High Productivity Computing Systems (HPCS) program is focused on providing a new generation of economically viable high productivity computing systems for national security and for the industrial user community. The value of a high performance computing (HPC) system to a user includes many factors, such as execution time on a particular problem, software development time, direct hardware costs, and indirect administrative and maintenance costs. This special issue, which focuses on HPC productivity, brings together, for the first time, a series of novel papers written by several distinguished authors who share their views on this topic. The topic of productivity in HPC is very new and the authors have been encouraged to speculate. The goal of this first paper is to present an overarching context and framework for the other papers and to define some common ideas that have emerged in considering the problem of HPC productivity. In addition, this paper defines several characteristic HPC workflows that are useful for understanding how users exploit HPC systems, and discusses the role of activity and purpose benchmarks in establishing an empirical basis for HPC productivity.

Standards for graph algorithm primitives

It is our view that the state of the art in constructing a large collection of graph algorithms in terms of linear algebraic operations is mature enough to support the emergence of a standard set of primitive building blocks. This paper is a position paper defining the problem and announcing our intention to launch an open effort to define this standard.

Parallel VSIPL++: An Open Standard Software Library for High-Performance Parallel Signal Processing

Real-time signal processing consumes the majority of the worlds computing power. Increasingly, programmable parallel processors are used to address a wide variety of signal processing applications (e.g., scientific, video, wireless, medical, communication, encoding, radar, sonar, and imaging). In programmable systems, the major challenge is no longer hardware but software. Specifically, the key technical hurdle lies in allowing the user to write programs at high level, while still achieving performance and preserving the portability of the code across parallel computing hardware platforms. The Parallel Vector, Signal, and Image Processing Library (Parallel VSIPL++) addresses this hurdle by providing high-level C++ array constructs, a simple mechanism for mapping data and functions onto parallel hardware, and a community-defined portable interface. This paper presents an overview of the Parallel VSIPL++ standard as well as a deeper description of the technical foundations and expected performance of the library. Parallel VSIPL++ supports adaptive optimization at many levels. The C++ arrays are designed to support automatic hardware specialization by the compiler. The computation objects (e.g., fast Fourier transforms) are built with explicit setup and run stages to allow for runtime optimization. Parallel arrays and functions in Parallel VSIPL++ also support explicit setup and run stages, which are used to accelerate communication operations. The parallel mapping mechanism provides an external interface that allows optimal mappings to be generated offline and read into the system at runtime. Finally, the standard has been developed in collaboration with high performance embedded computing vendors and is compatible with their proprietary approaches to achieving performance.

D4M 2.0 schema: A general purpose high performance schema for the Accumulo database

Non-traditional, relaxed consistency, triple store databases are the backbone of many web companies (e.g., Google Big Table, Amazon Dynamo, and Facebook Cassandra). The Apache Accumulo database is a high performance open source relaxed consistency database that is widely used for government applications. Obtaining the full benefits of Accumulo requires using novel schemas. The Dynamic Distributed Dimensional Data Model (D4M)[http://www.mit.edu/~kepner/D4M] provides a uniform mathematical framework based on associative arrays that encompasses both traditional (i.e., SQL) and non-traditional databases. For non-traditional databases D4M naturally leads to a general purpose schema that can be used to fully index and rapidly query every unique string in a dataset. The D4M 2.0 Schema has been applied with little or no customization to cyber, bioinformatics, scientific citation, free text, and social media data. The D4M 2.0 Schema is simple, requires minimal parsing, and achieves the highest published Accumulo ingest rates. The benefits of the D4M 2.0 Schema are independent of the D4M interface. Any interface to Accumulo can achieve these benefits by using the D4M 2.0 Schema.