Martial Michel

The NIST Smart Space and Meeting Room projects: signals, acquisition annotation, and metrics

Pervasive computing devices, sensors, and networks, provide infrastructure for context aware smart meeting rooms that sense ongoing human activities and respond to them. This requires advances in areas including networking, distributed computing, sensor data acquisition, signal processing, speech recognition, human identification, and natural language processing. Open interoperability and metrology standards for the sensor and recognition technologies can aid R&D programs in making these advances. The NIST (National Institute of Standards and Technology) Smart Space and Meeting Room projects are developing tools for data formats, transport, distributed processing, and metadata. We are using them to create annotated multi modal research corpora and measurement algorithms for smart meeting rooms, which we are making available to the research and development community.

The CLEAR 2007 Evaluation

This paper is a summary of the 2007 CLEAR Evaluation on the Classification of Events, Activities, and Relationships which took place in early 2007 and culminated with a two-day workshop held in May 2007. CLEAR is an international effort to evaluate systems for the perception of people, their activities, and interactions. In its second year, CLEAR has developed a following from the computer vision and speech communities, spawning a more multimodal perspective of research evaluation. This paper describes the evaluation tasks, including metrics and databases used, and discusses the results achieved. The CLEAR 2007 tasks comprise person, face, and vehicle tracking, head pose estimation, as well as acoustic scene analysis. These include subtasks performed in the visual, acoustic and audio-visual domains for meeting room and surveillance data.

AutoMap and AutoLink: Tools for Communicating Complex and Dynamic Data-Structures Using MPI

This article describes two software tools, AutoMap and AutoLink, that facilitate the use of data-structures in MPI. AutoMap is a program that parses a file of user-defined data-structures and generates new MPI types out of basic and previously defined MPI data-types. Our software tool automatically handles specialized error checking related to memory mapping. AutoLink is an MPI library that allows the transfer of complex, dynamically linked, and possibly heterogeneous structures through MPI. AutoLink uses files generated by AutoMap to automatically define the needed MPI data-types. We describe each of these tools, and give an example of their use. Finally we discuss the internals of AutoLink design, and focus on the performance rationale behind them.

The TRECVid 2008 Event Detection evaluation

This paper is a summary of the 2008 TRECVid Event Detection evaluation track. TRECVid is a laboratory-style evaluation that aims to model real world situations or significant component tasks. The event detection evaluation was organized to address detection of a set of specific events that would be of potential interest to an operator in the surveillance domain. This paper describes the video data, evaluation tasks, evaluation metrics, and results of the event detection evaluation.

Synchronizing multimodal data streams acquired using commodity hardware

We have developed tools and techniques that allow video frame level synchronization of multiple free-running commodity video cameras,microphones,and computer nodes using non-realtime operating systems.The techniques rely on physical audiovisual synchronization pulses,statistical procedures to correlate and interpolate the multiple timestamp streams,and software tools for review to produce smoothed and drift-corrected timestamp streams in our multimodal corpora. In this article we present those techniques and tools. Our project is open source and we are seeking collaborative developers for future work.

A generalized approach for transferring data-types with arbitrary communication libraries

We present a generalized algorithm for implementing a communications library for dynamic data structures created with heterogeneous composed data types such as multiple C structs, and where the data-types may be nested and may contain pointers. This algorithm is divided into an absolute part that is the same for all instantiations, and a relative part that is specific to the communications mechanism used, such as PVM or MPI. We describe the algorithm in terms of our AutoMap/AutoLink implementation in C/MPI. First, we talk of the MPI case and of the AutoMap and AutoLink solutions (with ideas from version 3.0). Then we discuss what is to be followed in order to generalize the data-type transfer concepts presented. With this addition to AutoMap/AutoLink we can extend the functions provided from the current send and receive functions (blocking and non blocking) available for any data-types, to any kind of transfer function; from broadcast to reduce (as long as the reduce called process is message aware). This will also simplify the extension of this work to data-types load balancing.

The NIST data science initiative

We examine foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new Data Science Initiative and evaluation series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The evaluations will facilitate research efforts, collaboration, leverage shared infrastructure, and effectively address cross-cutting challenges faced by diverse data science communities. The evaluations will have multiple research tracks championed by members of the data science community, and will enable rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will measure several different data science technologies in a wide range of fields, starting with a pre-pilot. In addition to developing data science evaluation methods and metrics, it will address computing infrastructure, standards for an interoperability framework, and domain-specific examples.

AVSS Multiple Camera Person Tracking Challenge Evaluation Overview

Technologies that track a specific person as they traverse a network of surveillance cameras can be used as the basis for a multitude of video surveillance applications including mass transit monitoring, large venue security, building security, and the like. In order to continue supporting the development robust people tracking technologies, the first AVSS Multiple Camera Person Tracking (MCPT) Challenge Evaluation was established to provide data and evaluation resources for researchers to build Single Person Tracking (SPT) technologies. This special session will focus on the AVSS MCPT Challenge Evaluation which will include a description of the evaluation task, the i-LIDS Multiple-camera tracking scenario data set used for the evaluation, and presentations by the challenge evaluation participants describing their systems.

A genetic programming ecosystem

Algorithms are needed in every aspect of parallel computing. Genetic Programming is an evolutionary technique for automating the design of algorithms through iterative steps of mutation and crossover operations on an initial population of randomly generated computer programs. This paper describes a novel parallel genetic programming (GP) system inspired by the symbiogenesis model of evolution, wherein new organisms are generated through the absorption of different life-forms in addition to the usual mutation and crossover operations. Different organisms are expressed in this GP system through multiple program representations. Two program representations considered in this paper are the procedural representation (PR) and the tree representation (TR). Populations of these representations evolve separately. Individuals in each population migrate to the other and participate in evolution via representation change algorithms. Parallelism is achieved through use of the AutoMap/AutoLink MPI library. The differences in the locality properties of the representations serve as a source of new ideas for creating the final algorithm.

A new data science research program: evaluation, metrology, standards, and community outreach

This article examines foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new data science research program (DSRP) and associated data science evaluation (DSE) series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The DSRP is designed to facilitate and accelerate research progress in the field of data science and consists of four components: evaluation and metrology, standards, compute infrastructure, and community outreach. A key part of the evaluation and measurement component is the DSE. The DSE series aims to address logistical and evaluation design challenges while providing rigorous measurement methods and an emphasis on generalizability rather than domain- and application-specific approaches. Toward that end, each year the DSE will consist of multiple research tracks and will encourage the application of tasks that span these tracks. The evaluations are intended to facilitate research efforts and collaboration, leverage shared infrastructure, and effectively address crosscutting challenges faced by diverse data science communities. Multiple research tracks will be championed by members of the data science community with the goal of enabling rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will permit us to measure several different data science technologies in a wide range of fields and will address computing infrastructure, standards for an interoperability framework, and domain-specific examples. This article also summarizes lessons learned from the data science evaluation series pre-pilot that was held in fall of 2015.