Kedar Anil Patwardhan

Monitoring, recognizing and discovering social networks

This work addresses the important problem of the discovery and analysis of social networks from surveillance video. A computer vision approach to this problem is made possible by the proliferation of video data obtained from camera networks, particularly state-of-the-art Pan-Tilt-Zoom (PTZ) and tracking camera systems that have the capability to acquire high-resolution face images as well as tracks of people under challenging conditions. We perform “opportunistic” face recognition on captured images and compute motion similarities between tracks of people on the ground plane. To deal with the unknown correspondences between faces and tracks, we present a novel graph-cut based algorithm to solve this association problem. It enables the robust estimation of a social network that captures the interactions between individuals in spite of large amounts of noise in the datasets. We also introduce an algorithm that we call “modularity-cut”, which is an Eigen-analysis based approach for discovering community and leadership structure in the estimated social network. Our approach is illustrated with promising results from a fully integrated multi-camera system under challenging conditions over long period of time.

Automated catheter detection in volumetric ultrasound

Ultrasound guided catheter insertion is a common procedure in current clinical practice, but it requires a skilled ultrasound practitioner to correctly acquire the images so that the catheter and tip are well visualized. Automated detection of the catheter location in ultrasound images can help in procedural guidance as well as surveillance of catheter position post placement, especially when the ultrasound imaging is being performed by a non-sonographer. Accurate and fast localization of the catheter is a very challenging task because of the poor observability of the catheter in ultrasound images. In this paper, we present a novel algorithm for fast catheter detection in 3D ultrasound images. We begin by generating a catheter-likelihood map using a physical model of the catheter. For fast detection, we perform a greedy optimization, where the likelihood map is projected onto a single image plane using a variation on the maximum-intensity-projection approach. The highest likelihood curves on this plane then help us to determine the curved planes on which the catheter may be located. The potential catheter locations are then detected on these curved planes. We finally apply some post-processing to enable robust detection of the catheter. We illustrate the performance of our proposed method through quantitative comparisons with expert annotations as well as qualitative results on 26 images obtained from 7 subjects (of which 15 are with and 11 are without a catheter).

Intelligent Video for Protecting Crowded Sports Venues

Intelligent video in urban settings can be challenging due the presence of crowds, clutter, poor camera placement and continuously changing light conditions. The surveillance of sports venues is particularly difficult, because thousands of people can enter or exit a venue in short periods of time. This paper presents a case study of successfully monitoring a sports venue using a multi-camera multi-target tracking system. The system performed site-wide tracking throughout a network of calibrated cameras and was able to accurately track thousands of people in real-time under challenging conditions. The extracted tracking information was used to detect a range of real-time events such as crowd formation, left luggage, and loitering. In addition all video,track and event information was indexed and stored to allow operators to perform playback and forensic search. This paper will present an overview of the deployed system and discuss the challenges that were encountered during the deployment.

Multi-camera person tracking in crowded environments

Reliably tracking people throughout a camera network is an important capability in areas such as law enforcement, homeland protection, and healthcare. In this paper we will provide an overview of GE Global Researchs tracking system and evaluate it against a subset of the PETS 2009 dataset. The tasks of counting, density estimation, multiperson tracking as well as the tracking of selected individuals will be addressed. Qualitative and quantitative performance results will be reported.

Collaborative Control of Active Cameras in Large-Scale Surveillance

A system that controls a set of pan-tilt-zoom (PTZ) cameras for acquiring closeup views of subjects in a surveillance site is presented. The PTZ control is based on the output of a multi-camera, multi-target tracking system operating on a set of fixed cameras, and the main goal is to acquire views of subjects for biometrics purposes, such as face recognition and nonfacial identification. For this purpose, this chapter introduces an algorithm to address the generic problem of collaboratively controlling a limited number of PTZ cameras to capture an observed number of subjects in an optimal fashion. Optimality is achieved by maximizing the probability of successfully completing the addressed biometrics task, which is determined by an objective function parameterized on expected capture conditions, including distance at which a subject is imaged, angle of capture, and several others. Such an objective function serves to effectively balance the number of captures per subject and their quality. Qualitative and quantitative experimental results are provided to demonstrate the performance of the system, which operates in real time under real-world conditions on four PTZ and four static CCTV cameras, all of which are processed and controlled via a single workstation.

Automated bone and joint-region segmentation in volumetric ultrasound

Ultrasound (US) is being increasingly used in the assessment of joints for disorders such as rheumatoid arthritis (RA). Pathologies related to RA in joints, manifest themselves commonly as changes in the bone (e.g. erosions) and the region enclosed by the joint-capsule (e.g. synovitis). Automated tools for detecting and segmenting such structures would help significantly towards objective and quantitative assessment of RA in joints. In this paper we present a novel method for automatic extraction of the 3D bone surface and segmentation of the joint-capsule region from volumetric (3D) scans of commonly affected joints such as the metacar-pophalangeal (MCP) and metatarsophalangeal (MTP) joints. We improve and extend the cost function proposed in [1] to 3D such that it is more robust to loss of acoustic intensity due to the surface normals facing away from the incident acoustic beam. The extracted bone coupled with a simple anatomical model of the MCP joint provides a coarse localization of the joint-capsule region. A probabilistic speckle model is then used to iteratively refine the capsule segmentation. We illustrate the performance of proposed methods through quantitative comparisons with expert annotations as well as qualitative results on over 30 scans obtained from 11 subjects.

Toward Quantitative Assessment of Rheumatoid Arthritis Using Volumetric Ultrasound

Goal: Rheumatoid arthritis (RA) is characterized by inflammation within the joint space as well as erosion or destruction of the bone surface. We believe that volumetric (3-D) ultrasound imaging of the joints in conjunction with automated image-analysis tools for segmenting and quantifying the regions of interest can lead to improved RA assessment. Methods: In this paper, we describe our proposed algorithms for segmenting 1) the 3-D bone surface and 2) the 3-D joint capsule region. We improve and extend previous 2-D bone extraction methods to 3-D and make our algorithm more robust to the intensity loss due to surface normals facing away from incident acoustic beams. The extracted bone surfaces coupled with a joint-specific anatomical model are used to initialize a coarse localization of the joint capsule region. The joint capsule segmentation is refined iteratively utilizing a probabilistic speckle model. Results: We apply our methods on 51 volumes from 8 subjects, and validate segmentation results with expert annotations. We also provide the quantitative comparison of our bone detection with magnetic resonance imaging. These automated methods have achieved average sensitivity/precision rates of 94%/93% for bone surface detection, and 87%/83% for joint capsule segmentation. Segmentations of normal and inflamed joints are compared to demonstrate the potential of using proposed tools to assess RA pathology at the joint level. Conclusion: The proposed image-analysis methods showed encouraging results as compared to expert annotations. Significance: These computer-assisted tools can be used to help visualize 3-D anatomy in joints and help develop quantitative measurements toward RA assessment.

Volumetric ultrasound and computer-assisted analysis at the point-of-care: A musculoskeletal exemplar

In this paper we motivate the hypothesis that the use of volumetric ultrasound imaging and automated image analysis tools would improve clinical workflows as well as outcomes at the point-of-care. To make our case, this paper presents results from a rheumatoid arthritis (RA) study [1] where several image analysis techniques have been applied to volumetric ultrasound, highlighting anatomy of interest to better understand disease progression. Pathologies related to RA in joints, manifest themselves commonly as changes in the bone (e.g. erosions) and the region enclosed by the joint-capsule (e.g. synovitis). Automated tools for detecting and segmenting such structures would help significantly towards objective and quantitative assessment of RA in joints. Extracted bone coupled with a simple anatomical model of the joint provides a coarse localization of the joint-capsule region. A probabilistic speckle model is then used to iteratively refine the capsule segmentation. We illustrate the performance of proposed algorithms through quantitative comparisons with expert annotations as well as qualitative results on over 30 scans obtained from 11 subjects.

4D vessel segmentation and tracking in Ultrasound

In this paper we describe a fast method to segment and track a vessel of interest in 4D (i.e. 3D + time) Ultrasound images. An initial 2D seed is used to initialize a single spatial Kalman-Filter tracker which tracks the vessel center-line in 3D. The 3D vessel is then segmented using a fast area weighted active contour method. This segmented 3D vessel is then tracked across multiple time-points using a set of temporal Kalman- Filters which track the movement of the center of the vessel in each 2D slice. The vessel boundaries are estimated by growing an area weighted active contour outward from the centerline. Based on qualitative as well as quantitative performance measures, the proposed method shows promising tracking results on numerous phantom as well as patient datasets.

Video Analytics for Force Protection

For troop and military installation protection, modern computer vision methods must be harnessed to enable a comprehensive approach to contextual awareness. In this chapter we present a collection of intelligent video technologies currently under development at the General Electric Global Research Center, which can be applied to this challenging problem. These technologies include: aerial analysis for object detection and tracking, site-wide tracking from networks of fixed video cameras, person detection from moving platforms, biometrics at a distance and facial analysis for the purposes of inferring intent. We hypothesize that a robust approach to troop protection will require the synthesis of all of these technologies into a comprehensive system of systems.