Norman Haas

Smart video surveillance: exploring the concept of multiscale spatiotemporal tracking

Situation awareness is the key to security. Awareness requires information that spans multiple scales of space and time. Smart video surveillance systems are capable of enhancing situational awareness across multiple scales of space and time. However, at the present time, the component technologies are evolving in isolation. To provide comprehensive, nonintrusive situation awareness, it is imperative to address the challenge of multiscale, spatiotemporal tracking. This article explores the concepts of multiscale spatiotemporal tracking through the use of real-time video analysis, active cameras, multiple object models, and long-term pattern analysis to provide comprehensive situation awareness.

Personalized news through content augmentation and profiling

This paper is concerned with the topic of personalized news assembly at the set-top box, based on augmented video. This is video complemented with additional information that is somehow relevant to the semantic video content. We touch upon the technique that is used for video augmentation, which is video subject detection followed by information searches on the subject. The focus of this paper is on subject detection implemented using traditional text analysis tools; video segmentation is based on these results and visual processing. We describe the architecture of such a system and the benefits to the consumer. Further we discuss a preliminary system that shows the viability of the concept.

Rail Component Detection, Optimization, and Assessment for Automatic Rail Track Inspection

In this paper, we present a real-time automatic vision-based rail inspection system, which performs inspections at 16 km/h with a frame rate of 20 fps. The system robustly detects important rail components such as ties, tie plates, and anchors, with high accuracy and efficiency. To achieve this goal, we first develop a set of image and video analytics and then propose a novel global optimization framework to combine evidence from multiple cameras, Global Positioning System, and distance measurement instrument to further improve the detection performance. Moreover, as the anchor is an important type of rail fastener, we have thus advanced the effort to detect anchor exceptions, which includes assessing the anchor conditions at the tie level and identifying anchor pattern exceptions at the compliance level. Quantitative analysis performed on a large video data set captured with different track and lighting conditions, as well as on a real-time field test, has demonstrated very encouraging performance on both rail component detection and anchor exception detection. Specifically, an average of 94.67% precision and 93% recall rate has been achieved for detecting all three rail components, and a 100% detection rate is achieved for compliance-level anchor exception with three false positives per hour. To our best knowledge, our system is the first to address and solve both component and exception detection problems in this rail inspection area.

S3-R1: the IBM smart surveillance system-release 1

Summary form only given. One of the key components of tele-presence systems is automatic awareness of the remote environment. This very same capability of automatic situation awareness is currently being developed and deployed in the context of the next generation smart surveillance systems. Smart surveillance systems use a number of automatic video analysis techniques like object detection, tracking and classification in conjunction with database and Web application servers to provide users with the capability of distributed smart surveillance. The IBM smart surveillance system is one of the few advanced surveillance systems which provides not only the capability to automatically monitor a scene but also the capability to manage the surveillance data, perform event based retrieval, receive real time event alerts thru standard web infrastructure and extract long term statistical patterns of activity.

Enhanced rail component detection and consolidation for rail track inspection

For safety purposes, railroad tracks need to be inspected on a regular basis for physical defects or design noncompliances. Such track defects and non-compliances, if not detected in a timely manner, may eventually lead to grave consequences such as train derailments. In this paper, we present a real-time automatic vision-based rail inspection system, with main focus on anchors - an important rail component type, and anchor-related rail defects, or exceptions. Our system robustly detects important rail components including ties, tie plates, anchors with high accuracy and efficiency. Detected objects are then consolidated across video frames and across camera views to map to physical rail objects, by combining the video data streams from all camera views with GPS information and speed information from the distance measuring instrument (DMI). After these rail components are detected and consolidated, further data integration and analysis is followed to detect sequence-level track defects, or exceptions. Quantitative analysis performed on a real online field test conducted on different track conditions demonstrates that our system achieves very promising performance in terms of rail component detection, anchor condition assessment, and compliance-level exception detection. We also show that our system outperforms another advanced rail inspection system in anchor detection.

A real-time prototype for small-vocabulary audio-visual ASR

We present a prototype for the automatic recognition of audio-visual speech, developed to augment the IBM ViaVoice/spl trade/ speech recognition system. Frontal face, full frame video is captured through a USB 2.0 interface by means of an inexpensive PC camera, and processed to obtain appearance-based visual features. Subsequently, these are combined with audio features, synchronously extracted from the acoustic signal, using a simple discriminant feature fusion technique. On the average, the required computations utilize approximately 67% of a Pentium/spl trade/ 4, 1.8 GHz processor, leaving the remaining resources available to hidden Markov model based speech recognition. Real-time performance is there- fore achieved for small-vocabulary tasks, such as connected-digit recognition. In the paper, we discuss the prototype architecture based on the ViaVoice engine, the basic algorithms employed, and their necessary modifications to ensure real-time performance and causality of the visual front end processing. We benchmark the resulting system performance on stored videos against prior research experiments, and we report a close match between the two.

Component-based track inspection using machine-vision technology

In this paper, we present our latest research engagement with a railroad company to apply machine vision technologies to automate the inspection and condition monitoring of railroad tracks. Specifically, we have proposed a complete architecture including imaging setup for capturing multiple video streams, important rail component detection such as tie plate, spike, anchor and joint bar bolt, defect identification such as raised spikes, defect severity analysis and temporal condition analysis, and long-term predictive assessment. This paper will particularly present various video analytics that we have developed to detect rail components, which form the building block of the entire framework. Our preliminary performance study has achieved an average of 98.2% detection rate, 1.57% false positive rate and 1.78% false negative rate on the component detection. Finally, with the lack of sufficient representative data and annotations to evaluate system performance on exception detection at both sequence and compliance levels, we proposed a mathematical modeling approach to calculate the probabilities of detecting such exceptions. Such analysis shows that there is still big room for us to improve our approaches in order to achieve desired false positive rate and miss detection rate at the sequence level.

Fingerprint Quality Assessment

For a particular biometric to be effective, it should be universal: Every individual in the target population should possess the biometrics, and every acquisition from each individual should provide useful information for personal identity verification or recognition. In other words, everybody should have the biometrics and it should be easy to sample or acquire. In practice, adverse signal acquisition conditions and inconsistent presentations of the signal often result in unusable or nearly unusable biometrics signals (biometrics samples). This is confounded by the problem that the underlying individual biometrics signal can vary over time due, for example, to aging. Hence, poor quality of the actual machine sample of a biometrics constitutes the single most cause of poor accuracy performance of a biometrics system. Therefore, it is important to quantify the quality of the signal, either for seeking a better representation of the signal or for subjecting the poor signal to alternative methods of processin g (e.g., enhancement [9]). In this chapter,1 we explore a definition of the quality of fingerprint impressions and present detailed algorithms to measure image quality. The proposed quality measure has been developed with the use of human annotated images, and tested on a large number of fingerprints of different modes of fingerprint acquisition methods.

Visual item verification for fraud prevention in retail self-checkout

Many modern retail stores have self-checkout stations where customers can ring up their own orders without the assistance of any store personnel. To promote customer honesty these systems often weigh each item as it is placed in the bag to confirm that it has the expected mass for the product scanned. In our system we augment this basic check with an assessment of the items visual appearance to further ensure that the correct code has been entered.

Retail video analytics: an overview and survey

Today retail video analytics has gone beyond the traditional domain of security and loss prevention by providing retailers insightful business intelligence such as store traffic statistics and queue data. Such information allows for enhanced customer experience, optimized store performance, reduced operational costs, and ultimately higher profitability. This paper gives an overview of various camera-based applications in retail as well as the state-ofthe- art computer vision techniques behind them. It also presents some of the promising technical directions for exploration in retail video analytics.