Yen-Chieh Lai

Hardware-efficient belief propagation

Belief propagation (BP) is an effective algorithm for solving energy minimization problems in computer vision. However, it requires enormous memory, bandwidth, and computation because messages are iteratively passed between nodes in the Markov random field (MRF). In this paper, we propose two methods to address this problem. The first method is a message passing scheme called tile-based belief propagation. The key idea of this method is that a message can be well approximated from other faraway ones. We split the MRF into many tiles and perform BP within each one. To preserve the global optimality, we store the outgoing boundary messages of a tile and use them when performing BP in the neighboring tiles. The tile-based BP only requires 1-5% memory and 0.2-1% bandwidth of the ordinary BP. The second method is an O(L) message construction algorithm for the robust functions commonly used for describing the smoothness terms in the energy function. We find that many variables in constructing a message are repetitive; thus these variables can be calculated once and reused many times. The proposed algorithms are suitable for parallel implementations. We design a low-power VLSI circuit for disparity estimation that can construct 440 M messages per second and generate high quality disparity maps in near real-time. We also implement the proposed algorithms on a GPU, which can calculate messages 4 times faster than the sequential O(L) method.

Hardware-Efficient Belief Propagation

Loopy belief propagation (BP) is an effective solution for assigning labels to the nodes of a graphical model such as the Markov random field (MRF), but it requires high memory, bandwidth, and computational costs. Furthermore, the iterative, pixel-wise, and sequential operations of BP make it difficult to parallelize the computation. In this paper, we propose two techniques to address these issues. The first technique is a new message passing scheme named tile-based BP that reduces the memory and bandwidth to a fraction of the ordinary BP algorithms without performance degradation by splitting the MRF into many tiles and only storing the messages across the neighboring tiles. The tile-wise processing also enables data reuse and pipeline, resulting in efficient hardware implementation. The second technique is an O(L) fast message construction algorithm that exploits the properties of robust functions for parallelization. We apply these two techniques to a very large-scale integration circuit for stereo matching that generates high-resolution disparity maps in near real-time. We also implement the proposed schemes on graphics processing unit (GPU) which is four-time faster than standard BP on GPU.

Full length article: Network capacity of cognitive radio relay network

After successful dynamic spectrum access, cognitive radio (CR) must be able to relay the message/packets to the destination node by utilizing existing primary system(s) (PS) and/or cooperative/cognitive radio nodes in the cognitive radio network. In this paper, we pioneer the exploration of the fundamental behaviors of interference between CRs and PS in such a relay network via network coding. Interference on PSs network capacity is shown to be unavoidable and unbounded in the one-hop relay network. Extending to the tandem structure, interference is unbounded but avoidable by appropriate constraints. In cooperative relay network, interference is bounded and avoidable. Moreover, parallel cooperative relay network can accommodate more CR transmission pairs. Such an analysis can be generalized to arbitrary networks. We derive that interference is avoidable when at least one route from CRs source to the sink bypasses the bottlenecks of PS. Then under the constraint of no interference to PS, we derive CRs maximum network capacity in such a network. Link allocation to achieve the maximum network capacity can be formulated and solved as a linear programming problem. Consequently, given any network topology, we can determine whether CRs interference is avoidable, and maximize CRs network capacity without interfering PSs network capacity. Simulation results on randomly generated network topologies show that CRs network capacity achieves on average 1.3 times of PSs network capacity with interference avoidance constraint, and demonstrates spectrum efficiency at networking throughput and high availability.

Architecture design of stereo matching using belief propagation

We propose a new architecture for stereo matching using belief propagation. The architecture combines our fast, fully-parallel processing element (PE) and memory-efficient tile-based BP (TBP) algorithm. On the architectural level, we develop several novel techniques, including a three stage pipeline, a message forwarding scheme, and a boundary message reuse scheme, which greatly reduce the required bandwidth and power consumption without sacrificing performance. The simulation shows that the architecture can generate HDTV720p results at 30 fps when operating at 227MHz. The high-quality depth maps enable real-time depth image based rendering and many other important applications in the 3D TV industry.

Analysis of belief propagation for hardware realization

Belief propagation has become a popular technique for solving computer vision problems, such as stereo estimation and image denoising. However, it requires large memory and bandwidth, and hence naive hardware implementation is prohibitive. In this paper, we first analyze the memory and bandwidth requirements of the technique from the hardware perspective. Then, we propose a tile-based belief propagation algorithm that works with existing data reuse schemes and achieves bandwidth reduction by a factor of 10 to 400. We apply the proposed algorithm to stereo estimation and show that its performance is comparable to the original algorithm.

Fast belief propagation process element for high-quality stereo estimation

Belief propagation is a popular global optimization technique for many computer vision problems. However, it requires extensive computation due to the iterative message passing operations. In this paper, we present a new process element (PE) for efficient message construction. The efficiency is gained by exploiting the unique characteristics of the generalized Potts model (truncated linear mode) of the smoothness term in the Markov random field. For stereo estimation with L disparity values, the algorithm successfully reduces the computation from O(L2) to O(L) and retains the high throughput and low latency. Compared with the direct message construction PE, our method achieves 87.14% computation saving and a 94.38% PE area reduction.

Brain-Inspired Framework for Fusion of Multiple Depth Cues

2-D-to-3-D conversion is an important step for obtaining 3-D videos, as a variety of monocular depth cues have been explored to generate 3-D videos from 2-D videos. As in a human brain, a fusion of these monocular depth cues can regenerate 3-D data from 2-D data. By mimicking how our brains generate depth perception, we propose a reliability-based fusion of multiple depth cues for an automatic 2-D-to-3-D video conversion. A series of comparisons between the proposed framework and the previous methods is also presented. It shows that significant improvement is achieved in both subjective and objective experimental results. From the subjective viewpoint, the brain-inspired framework outperforms earlier conversion methods by preserving more reliable depth cues. Moreover, an enhancement of 0.70-3.14 dB and 0.0059-0.1517 in the perceptual quality of the videos is realized in terms of the objective-modified peak signal-to-noise ratio and disparity distortion model, respectively.

Efficient message reduction algorithm for stereo matching using belief propagation

Belief propagation (BP) is a popular global optimization technique in computer vision. However, it requires huge bandwidth and memory in hardware implementation because it iteratively processes messages between the neighboring nodes. In this paper, we propose an efficient message reduction algorithm that greatly reduces the bandwidth and memory consumption. Compared with the original message passing operation, we successfully reduce the memory and bandwidth with similar quality. For stereo matching of a VGA input where the disparity range is 64, the proposed algorithm can achieve 93.75% message memory reduction with about only 0.2–2.2% bad pixels quality degradation. The proposed algorithm greatly reduces the memory requirement and is suitable both for hardware and software realization.

3D image correction by Hilbert Huang decomposition

This paper presented a novel correction for 3D image signals by Hilbert Huang decomposition. Hilbert Huang decomposition is applied for dividing edges of the textures and edges of the objects. Depth map is corrected by regrouping pixels with edges of the objects. The proposed correction outperforms conventional methods especially at the boundaries of the objects.

Perceptual multi-cues 2D-to-3D conversion system

The concept of perceptual integration of multiple depth cues is proposed for converting 2D videos to 3D videos. Several monocular depth cues have been explored to reconstruct 3D videos. We propose reliability-based promotion and fusion of depth cues for an automatic reconstruction of 3D videos in this demo by mimicking how human visual system does.