Asim Khan

A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems

Pedestrian detection is a key problem in computer vision and is currently addressed with increasingly complex solutions involving compute-intensive features and classification schemes. In this scope, histogram of oriented gradients (HOG) in conjunction with linear support vector machine (SVM) classifier is considered to be the single most discriminative feature that has been adopted as a stand-alone detector as well as a key instrument in advance systems involving hybrid features and cascaded detectors. In this paper, we propose a pedestrian detection framework that is computationally less expensive as well as more accurate than HOG-linear SVM. The proposed scheme exploits the discriminating power of the locally significant gradients in building orientation histograms without involving complex floating point operations while computing the feature. The integer-only feature allows the use of powerful histogram inter-section kernel SVM classifier in a fast lookup-table-based implementation. Resultantly, the proposed framework achieves at least 3% more accurate detection results than HOG on standard data sets while being 1.8 and 2.6 times faster on conventional desktop PC and embedded ARM platforms, respectively, for a single scale pedestrian detection on VGA resolution video. In addition, hardware implementation on Altera Cyclone IV field-programmable gate array results in more than 40% savings in logic resources compared with its HOG-linear SVM competitor. Hence, the proposed feature and classification setup is shown to be a better candidate as the single most discriminative pedestrian detector than the currently accepted HOG-linear SVM.

Hardware architecture and optimization of sliding window based pedestrian detection on FPGA for high resolution images by varying local features

Pedestrian detection has lately attracted considerable interest from researchers due to many practical applications. However, the low accuracy and high complexity of pedestrian detection has still not enabled its use in successful commercial applications. In this paper, we present insights into the complexity-accuracy relationship of pedestrian detection. We consider the Histogram of Oriented Gradients (HOG) scheme with linear Support Vector Machine (LinSVM) as a benchmark. We describe parallel implementations of various blocks of the pedestrian detection system which are designed for full-HD (1920×1080) resolution. Features are improved by optimal selection of cell size and histogram bins which have been shown to significantly affect the accuracy and complexity of pedestrian detection. It is seen that with a careful choice of these parameters a frame rate of 39.2 fps is achieved with a negligible loss in accuracy which is 16.3x and 3.8x higher than state of the art GPU and FPGA implementations respectively. Moreover 97.14% and 10.2% reduction in energy consumption is observed to process one frame. Finally, features are further enhanced by removing petty gradients in histograms which result in loss of accuracy. This increases the frame rate to 42.7 fps (18x and 4.1x higher) and lowers the energy consumption by 97.34% and 16.4% while improving the accuracy by 2% as compared to state of the art GPU and FPGA implementations respectively.

EBSCam: Background Subtraction for Ubiquitous Computing

Background subtraction (BS) is a crucial machine vision scheme for detecting moving objects in a scene. With the advent of smart cameras, the embedded implementation of BS finds ever-increasing applications. This paper presents a new BS scheme called efficient BS for smart cameras (EBSCam). EBSCam thresholds the change in the estimated background model, which suppresses variance of the estimates, resulting in competitive performance compared with standard BS schemes. The percentage of wrong classification of EBSCam is lower than those of the Gaussian mixture model (GMM) (10.97%) and the pixel-based adaptive segmenter (PBAS) (4.66%) algorithms in FPGA implementations. Moreover, the memory bandwidth requirement of EBSCam is 6.66%, 41.36%, and 90.48% lower than the state-of-the-art FPGA implementation of GMM, ViBe, and PBAS algorithms, respectively. EBSCam achieves a significant speed up compared with the FPGA implementations of GMM (by 43.3%), ViBe (by 118.6%), and PBAS (by 144.8%) schemes. Similarly, the energy consumption of EBSCam is 80.56% and 99.9% less compared with GMM and PBAS, respectively. In summary, the advantages of EBSCam in accuracy, speed, and energy consumption combined together make it especially suitable for embedded applications.

Squeezing maximizing performance out of 3D cache-stacked multicore architectures

3D integration is one of the most promising options to fulfill the demands of high performance and large cache by integrating multiple processor cores and 3D stacked cache. There are however temperature problems in 3D integration. This paper presents a method for performance maximization of a 3D cache-stacked multicore system keeping the temperature under a given limit while by assigning the clock frequencies and number of cache banks to each core according to the requirement. We have done experiments on multiple benchmark programs and have found a peak 32% and an average 29.8% improvement in performance as compared to the base case which assigns the same frequency and the same number of banks to each core.

Dual frame rate motion detection for memory- and energy-constrained surveillance systems

CCTV-based surveillance systems gaining widespread popularity still waste computational power, transmission bandwidth and storage space. This paper tries to respond to this necessity by proposing a motion-based video recording scheme with dual frame rate motion detection. Statistical models for memory and energy consumption of the overall system are described. Root-mean-square error of the models for memory consumption is 0.54 and 0.78 for systems with single frame rate and dual frame rate motion detection, respectively. For a typical surveillance video, the proposed dual frame rate system stores 1.22, 4.94, 7.81 less frames per second on at 10fps, 1fps, 0.5fps respectively, compared to the single frame rate motion detection system. We have suggested a criterion for using dual frame rate motion detection in surveillance based on a mathematical model.

Exploiting maximum throughput in 3D multicore architectures with stacked NUCA cache

Demands for high performance are growing rapidly and multiple processor cores and huge caches are required to meet these requirements. 3D integration provides us a very bright option to encounter this by integrating numerous cores and cache layers in a single chip. Temperature however becomes a problem in 3D integration due to increased power density. A methodology to exploit maximum performance while keeping the temperature under a given limit has been proposed in this paper. We have solved for the optimum clock frequencies, cache capacity and the placement of cache banks for each core to get the maximum throughput. Experiments are done on multiple benchmark programs and a peak 53% and an average 49% improvement in performance as compared to the base case which assigns same frequency and number of banks to each core is found.

Depth Estimation Using Single Camera with Dual Apertures

Depth sensing is an active area of research in imaging technology. Here, we use a dual-aperture system to infer depth from a single image based on the principle of depth from defocus (DFD). Dual-aperture camera includes a small all-pass aperture (which allows all light through the aperture) and a larger RGB-pass aperture (which allows visible light only). IR image captured through the smaller aperture is sharper than the RGB image captured through the large aperture. Since the difference of blurriness between two images is dependent on the actual distance, using a dual-aperture camera provides an opportunity to estimate depth of a scene. Measuring the absolute blur size is difficult, since it is affected by illuminant’s spectral distribution, noise, specular highlight, vignetting, etc. By using a dual-aperture camera, however, the relative blurriness can be measured in a robust way. In this article,, a detailed description of extracting depth using a dual-aperture camera is provided which includes procedures for fixing each of artifacts that degrade the depth quality based on DFD. Experimental results confirm the improved depth extraction by employing the aforementioned schemes.

Memory efficient hardware accelerator for kernel support vector machine based pedestrian detection

Pedestrian detection being a vital as well as complex problem poses a unique challenge from accuracy and complexity point of view. On-chip memory requirement is one of the key issues for sliding window based detectors. In this paper a memory efficient hardware architecture is proposed which estimates the weights from a partially stored model at runtime. It uses a simple and robust feature with histogram intersection classifier. The implementation results show 80% reduction in logic resources and 46% reduction in memory without sacrificing accuracy as compared to the state of the art hardware implementations.

Depth refinement on sparse-depth images using visual perception cues

Numerous depth extraction schemes cannot extract depth on textureless regions, thus generating sparse depth maps. In this paper, we propose using perception cues to improve the sparse depth map. We consider the local neighborhood as well the global surface properties of objects. We use this information to complement depth extraction schemes. The method is not scene or class specific. With quantitative evaluation, the proposed method is shown to perform better compared to previous depth refinement methods. The error in terms of standard deviation of depth has been reduced down by 60%. The computational overhead of the proposed method is also very low, making it a suitable candidate for depth refinement.

A Hardware Accelerator for Real Time Sliding Window Based Pedestrian Detection on High Resolution Images

Pedestrian detection has lately attracted considerable interest from researchers due to many practical applications. However, the low accuracy and high complexity of pedestrian detection has still not enabled its use in successful commercial applications. In this chapter, we present insights into the complexity-accuracy relationship of pedestrian detection. We consider the Histogram of Oriented Gradients (HOG) scheme with linear Support Vector Machine (LinSVM) as a benchmark. We describe parallel implementations of various blocks of the pedestrian detection system which are designed for full-HD (1920 × 1080) resolution. Features are improved by optimal selection of cell size and histogram bins which have been shown to significantly affect the accuracy and complexity of pedestrian detection. It is seen that with a careful choice of these parameters a frame rate of 39.2 fps is achieved with a negligible loss in accuracy which is 16.3x and 3.8x higher than state of the art GPU and FPGA implementations respectively. Moreover 97.14 % and 10.2 % reduction in energy consumption is observed to process one frame. Finally, features are further enhanced by removing petty gradients in histograms which result in loss of accuracy. This increases the frame rate to 42.7 fps (18x and 4.1x higher) and lowers the energy consumption by 97.34 % and 16.4 % while improving the accuracy by 2 % as compared to state of the art GPU and FPGA implementations respectively.