Byungin Moon

A Rectification Hardware Architecture for an Adaptive Multiple-Baseline Stereo Vision System

In this paper, we propose a new rectification hardware architecture that is suitable for an adaptive multiple-baseline stereo vision system. The adaptive multiple-baseline stereo vision system can obtain precise distance information compared with the fixed-baseline stereo vision system. To reduce the computation overhead of the stereo matching, the rectification is an essential step in the stereo vision system. However, the conventional rectification hardware architectures are designed for the fixed-baseline stereo vision system. Also, previously proposed the lookup table (LUT) based rectification hardware architectures are not suitable for the adaptive multiple-baseline stereo vision system. This is because these require large memory resources as the number of baseline increases. The experimental results show that the proposed hardware architecture does not need memory resources and also only few additional hardware resources are required as the number of the baseline increases as compared with the LUT-based rectification hardware.

Design and analysis of 3D IC-based low power stereo matching processors

This paper presents comprehensive design and analysis results of 3D IC-based low-power stereo matching processors. Our design efforts range from architecture design and verification to RTL-to-GDSII design and sign-off analysis based on GlobalFoundries 130-nm PDK. We conduct comprehensive studies on the area, performance, and power benefits of our 3D IC designs over 2D IC designs. Our 2-tier 3D IC designs attain 43% area, 14% wire length, and 13% power saving over 2D IC designs. We also study a pipeline-based partitioning method shown to be effective at minimizing power consumption and the total number of TSVs while balancing the size of each tier.

An accurate and cost-effective stereo matching algorithm and processor for real-time embedded multimedia systems

Stereo matching is a vision technique for finding three-dimensional (3D) distance information in various multimedia applications by calculating pixel disparities between the matching points of a stereo image pair captured from a stereo camera. The most important considerations in stereo matching are highly accurate results and real-time performance. Thus, this paper proposes an accurate stereo matching algorithm that uses both a census transform algorithm and the sum of absolute differences algorithm in a complementary manner and its real-time hardware architecture. In addition, the proposed algorithm uses a vertical census transform with cost aggregation (VCTCA) to reduce hardware costs while maintaining high matching accuracy. We model the proposed algorithm using C language and verify it in several environments. Using a hardware description language, we implement the proposed hardware architecture and verify it on a field-programmable gate array-based platform to confirm the cost and performance of the hardware. The experimental results show that the proposed algorithm using the VCTCA produces accurate 3D distance information in real environments and reduces the hardware complexity. Thus, the algorithm and its hardware architecture are suitable for real-time embedded multimedia systems.

A census-based stereo matching algorithm with multiple sparse windows

Stereo matching is one of the most active research areas in intelligent vehicle technology. In order to apply the stereo matching to intelligent vehicles, it must generate high-accuracy three-dimensional information in real time. For real-time stereo matching, this paper proposes a sparse multi-window method which not only gives robustness to noise but also reduces hardware cost with high matching accuracy. This is achieved by reusing the result of overlapped operations between adjacent windows. The experimental results show that the proposed method can reduce the hardware complexity of stereo matching processors with higher accuracy compared with the conventional window method.

Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors

Recently, to obtain three-dimensional depth information from a set of stereo images, stereo matching processors are widely used in intelligent robots, autonomous vehicles, and the Internet of things environment, all of which require real-time processing capability with minimal hardware resources. In this paper, we propose a modified adaptive support weight scheme with rectangular ring-type window configurations that minimize hardware resources while maintaining matching accuracy. In addition, to reduce the computational overhead of window-based local stereo matching algorithms, we present a robust disparity search range estimation scheme based on stretched depth histograms. To evaluate the performance of the proposed schemes, we implemented them using C language and performed experiments. In addition, to show the feasibility of the hardware implementation of the proposed schemes, we also describe them using Verilog hardware description language and implemented them using a field-programmable gate array-based platform. Experimental results show that compared to conventional method, the proposed schemes reduced up to 57% of hardware resources and 33% of computational overhead, respectively.

A hardware architecture based on the NCC algorithm for fast disparity estimation in 3D shape measurement systems

This paper proposes an efficient hardware architecture to estimate disparities between 2D images for generating 3D depth images in a stereo vision system. Stereo matching methods are classified into global and local methods. The local matching method uses the cost functions based on pixel windows such as SAD(sum of absolute difference), SSD(sum of squared difference) and NCC(normalized cross correlation). The NCC-based cost function is less susceptible to differences in noise and lighting condition between left and right images than the subtraction-based functions such as SAD and SSD, and for this reason, the NCC is preferred to the other functions. However, software-based implementations are not adequate for the NCC-based real-time stereo matching, due to its numerous complex operations. Therefore, we propose a fast pipelined hardware architecture suitable for real-time operations of the NCC function. By adopting a block-based box-filtering scheme to perform NCC operations in parallel, the proposed architecture improves processing speed compared with the previous researches. In this architecture, it takes almost the same number of cycles to process all the pixels, irrespective of the window size. Also, the simulation results show that its disparity estimation has low error rate.

The Impact of 3D Stacking and Technology Scaling on the Power and Area of Stereo Matching Processors

Recently, stereo matching processors have been adopted in real-time embedded systems such as intelligent robots and autonomous vehicles, which require minimal hardware resources and low power consumption. Meanwhile, thanks to the through-silicon via (TSV), three-dimensional (3D) stacking technology has emerged as a practical solution to achieving the desired requirements of a high-performance circuit. In this paper, we present the benefits of 3D stacking and process technology scaling on stereo matching processors. We implemented 2-tier 3D-stacked stereo matching processors with GlobalFoundries 130-nm and Nangate 45-nm process design kits and compare them with their two-dimensional (2D) counterparts to identify comprehensive design benefits. In addition, we examine the findings from various analyses to identify the power benefits of 3D-stacked integrated circuit (IC) and device technology advancements. From experiments, we observe that the proposed 3D-stacked ICs, compared to their 2D IC counterparts, obtain 43% area, 13% power, and 14% wire length reductions. In addition, we present a logic partitioning method suitable for a pipeline-based hardware architecture that minimizes the use of TSVs.

A study of partitioned DIMM tree management for multimedia server systems

In-memory computing systems have been attracting considerable attention as a method for servicing high-quality multimedia contents. In-memory computing was intended to store entire data sets in the main memory of a computer to eliminate the need to access slow mechanical hard discs and increase the ability to process complex and large volumes of data. Prior studies have proposed a dual inline memory module (DIMM) tree architecture (DTA) as a new structure for implementing the in-memory computing system. The DTA can apply a partitioned DIMM tree policy to efficiently manage memory. However, the partitioned DIMM tree has several drawbacks, including hardware overhead resulting from additional fields in both the translation lookaside buffer (TLB) and the page table and the demand for an additional fast partition area for the fast partition page table (FPPT). To overcome these drawbacks, this paper proposes an advanced TLB management policy for the partitioned DIMM tree, DIMM tree TLB and two new partitioned DIMM tree management policies, fast-FPPT and slow-FPPT. We model the proposed policies using C language and verify them by special workloads in experiments employing a large-capacity memory system. The experimental results show how the proposed policies influence system performance and confirm that they overcome problems in the existing DTA. The simulations revealed a similarity between the performance of systems using the proposed policies and that of the existing DTA model. However, as the proposed policies demand a considerably lower hardware cost than the existing DTA model, the proposed policies are more practical.

A modified census transform using the representative intensity values

In the census transform, hamming weight calculation is an important process because the hamming weight is the criterion for finding the similarity between stereo images. If the intensity value of a center pixel is changed by different illumination of two cameras, erroneous hamming weight calculation can happen. To compensate for such vulnerability to different illumination, this paper suggests a modified census transform method using representative values. In the proposed method, the intensity value of the center pixel in the window is replaced by the representative values, which are the midpoint and mean intensity values in the window. Experimental results show that the census transform with the proposed hamming weight calculation is less dependent on the different illumination of two cameras and has higher matching accuracy compared with the previous recent work on the census transform.

An Efficient Interworking Architecture of a Network Processor for Layer 7 Packet Processing

This paper presents a new interworking architecture for a network processor (NP) that is able to process packets from OSI layer 2 (L2) to layer 7 (L7) by combining a conventional NP with a general-purpose processor (GP). In general, most commercially available NPs could not afford to support a variety of network services. This is mainly because the conventional NPs are not able to process L7 packets. Thus, one of the most important requirements for the state-of-the-art NP is the ability to process packets of L2 to L7. To process L7 packets efficiently through both the conventional NP and GP, the proposed interworking architecture uses a deep packet inspector (DPI) and it controls the packet processing flow depending on the OSI layers of packets. Experimental results show that the proposed interworking architecture is not only able to process packets of L2 to L7 but also increase the throughput and load balance of the packet processing in the NP without large hardware overhead when compared with the conventional interworking architecture.