Xing Mei

On building an accurate stereo matching system on graphics hardware

This paper presents a GPU-based stereo matching system with good performance in both accuracy and speed. The matching cost volume is initialized with an AD-Census measure, aggregated in dynamic cross-based regions, and updated in a scanline optimization framework to produce the disparity results. Various errors in the disparity results are effectively handled in a multi-step refinement process. Each stage of the system is designed with parallelism considerations such that the computations can be accelerated with CUDA implementations. Experimental results demonstrate the accuracy and the efficiency of the system: currently it is the top performer in the Middlebury benchmark, and the results are achieved on GPU within 0.1 seconds. We also provide extra examples on stereo video sequences and discuss the limitations of the system.

Segment-Tree Based Cost Aggregation for Stereo Matching

Segment-tree (ST) based cost aggregation algorithm for stereo matching successfully integrates the information of segmentation with non-local cost aggregation framework. The tree structure which is generated by the segmentation strategy directly determines the final results for this kind of algorithms. However, the original strategy performs unreasonable due to its coarse performance and ignores to meet the disparity consistency assumption. To improve these weaknesses we propose a novel segmentation algorithm for constructing a more faithful ST with enhanced segmentation advantage according to a robust initial over-segmentation. Then we implement non-local cost aggregation framework on this new ST structure and obtain improved disparity maps. Performance evaluations on all 31 Middlebury stereo pairs show that the proposed algorithm outperforms than other five state-of-the-art aggregated based algorithms and also keeps time efficiency.

Stereo Matching with Reliable Disparity Propagation

In this paper, we propose a novel propagation-based stereo matching algorithm. Starting from an initial disparity map, our algorithm selects highly reliable pixels and propagates their disparities along the scan line to produce dense disparity results. The key idea is to construct a line segment region for each pixel with local color and connectivity constraints. The pixel wise line segments are efficiently used to compute initial disparities, select reliable pixels and determine proper propagation regions. Streaking artifacts are effectively removed in a refinement process. Experimental results demonstrate the performance of the proposed method: it ranks 5th in the Middlebury benchmark, and the results can be computed within a few seconds.

Fast Hydraulic Erosion Simulation and Visualization on GPU

Natural mountains and valleys are gradually eroded by rainfall and river flows. Physically-based modeling of this complex phenomenon is a major concern in producing realistic synthesized terrains. However, despite some recent improvements, existing algorithms are still computationally expensive, leading to a time-consuming process fairly impractical for terrain designers and 3D artists. In this paper, we present a new method to model the hydraulic erosion phenomenon which runs at interactive rates on todays computers. The method is based on the velocity field of the running water, which is created with an efficient shallow-water fluid model. The velocity field is used to calculate the erosion and deposition process, and the sediment transportation process. The method has been carefully designed to be implemented totally on GPU, and thus takes full advantage of the parallelism of current graphics hardware. Results from experiments demonstrate that the proposed method is effective and efficient. It can create realistic erosion effects by rainfall and river flows, and produce fast simulation results for terrains with large sizes.

Content-Based Colour Transfer

This paper presents a novel content‐based method for transferring the colour patterns between images. Unlike previous methods that rely on image colour statistics, our method puts an emphasis on high‐level scene content analysis. We first automatically extract the foreground subject areas and background scene layout from the scene. The semantic correspondences of the regions between source and target images are established. In the second step, the source image is re‐coloured in a novel optimization framework, which incorporates the extracted content information and the spatial distributions of the target colour styles. A new progressive transfer scheme is proposed to integrate the advantages of both global and local transfer algorithms, as well as avoid the over‐segmentation artefact in the result. Experiments show that with a better understanding of the scene contents, our method well preserves the spatial layout, the colour distribution and the visual coherence in the transfer process. As an interesting extension, our method can also be used to re‐colour video clips with spatially‐varied colour effects.

Real-time local stereo via edge-aware disparity propagation

This letter presents a novel method for real-time local stereo matching. Different from previous methods which have spent many efforts on cost aggregation, the proposed method re-solves the stereo problem by propagating disparities in the cost domain. It is started by pre-detecting the disparity priors, on which a new cost volume is built for disparity assignment. Then the reliable disparities are propagated via filtering on this cost volume. Specially, a new O ( 1 ) geodesic filter is proposed and demonstrated effective for the task of edge-aware disparity propagation. As can be expected, the proposed framework is highly efficient, due to leaving double aggregation on left-right views, as well as costly post-processing steps, out of account. Moreover, by properly designing a quadric cost function, our method could be extended to good sub-pixel accuracy with a simple quadratic polynomial interpolation. Quantitative evaluation shows that it outperforms all the other local methods both in terms of accuracy and speed on Middlebury benchmark. It ranks 8th out of over 150 submissions if sub-pixel precision is considered, and the average runtime is only 9ms on a NVIDIA GeForce GTX 580GPU.

Simple Empty-Space Removal for Interactive Volume Rendering

Interactive volume rendering methods such as texture-based slicing techniques and ray casting have been well developed in recent years. The rendering performance is generally restricted by the volume size, the fill-rate, and the texture fetch speed of the graphics hardware. For most 3D data sets, a fraction of the volume is empty, which will reduce the rendering performance without specific optimization. In this paper, we present a simple kd-tree-based space partitioning scheme to efficiently remove the empty spaces from the volume data sets at the preprocessing stage. The splitting rule of the scheme is based on a simple yet effective cost function evaluated through a fast approximation of the bounding volume of the nonempty regions. The scheme culls a large number of empty voxels and encloses the remaining data with a small number of axis-aligned bounding boxes, which are then used for interactive rendering. The number of boxes is controlled by halting criteria. In addition to its simplicity, our scheme requires little preprocessing time and improves the rendering performance significantly.

Simulation and Visualisation of Functional Landscapes: Effects of the Water Resource Competition between Plants

Vegetation ecosystem simulation and visualisation are challenging topics involving multidisciplinary aspects. In this paper, we present a new generic frame for the simulation of natural phenomena through manageable and interacting models. It focuses on the functional growth of large vegetal ecosystems, showing coherence for scales ranging from the individual plant to communities and with a particular attention to the effects of water resource competition between plants. The proposed approach is based on a model of plant growth in interaction with the environmental conditions. These are deduced from the climatic data (light, temperature, rainfall) and a model of soil hydrological budget. A set of layers is used to store the water resources and to build the interfaces between the environmental data and landscape components: temperature, rain, light, altitude, lakes, plant positions, biomass, cycles, etc. At the plant level, the simulation is performed for each individual by a structural-functional growth model, interacting with the plant’s environment. Temperature is spatialised, changing according to altitude, and thus locally controls plant growth speed. The competition for water is based on a soil hydrological model taking into account rainfalls, water runoff, absorption, diffusion, percolation in soil. So far, the incoming light radiation is not studied in detail and is supposed constant. However, competition for light between plants is directly taken into account in the plant growth model. In our implementation, we propose a simple architecture for such a simulator and a simulation scheme to synchronise the water resource updating (on a temporal basis) and the plant growth cycles (determined by the sum of daily temperatures). The visualisation techniques are based on sets of layers, allowing both morphological and functional landscape views and providing interesting tools for ecosystem management. The implementation of the proposed frame leads to encouraging results that are presented and illustrate simple academic cases.

Image Retargeting by Texture-Aware Synthesis

Real-world images usually contain vivid contents and rich textural details, which will complicate the manipulation on them. In this paper, we design a new framework based on exampled-based texture synthesis to enhance content-aware image retargeting. By detecting the textural regions in an image, the textural image content can be synthesized rather than simply distorted or cropped. This method enables the manipulation of textural & non-textural regions with different strategies since they have different natures. We propose to retarget the textural regions by example-based synthesis and non-textural regions by fast multi-operator. To achieve practical retargeting applications for general images, we develop an automatic and fast texture detection method that can detect multiple disjoint textural regions. We adjust the saliency of the image according to the features of the textural regions. To validate the proposed method, comparisons with state-of-the-art image retargeting techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.

Real-Time Marker Level Set on GPU

Level set methods have been extensively used to track the dynamical interfaces between different materials for physically based simulation, geometry modeling, oceanic modeling and other scientific and engineering applications. Due to the inherent Eulerian characteristics, interface evolution based on level set usually suffers from numerical diffusion, sharp feature missing and mass loss. Although some effective methods such as particle level set (PLS) and marker level set (MLS) have been proposed to tackle these difficulties, the complicated correction process and the high computational cost pose severe limitations for real-time applications. In this paper we provide an efficient parallel implementation of the marker level set method on latest graphics hardware. Each step of the MLS method is fully mapped on GPU with an innovative combination of different computation techniques. Relying on GPLs parallelism and flexible programmability, the method provides real-time performance for large size 2D examples and moderate 3D examples, which is significantly faster than previous CPU-based methods.