Archive | 2019
Closed-Form Solution to Disambiguate Defocus Blur in Single-Perspective Images
Abstract
Depth-from-defocus techniques suffer an ambiguity problem where depth planes on opposite sides of the focal plane have identical defocus. We solve the ambiguity by relying on the wavelength-dependent relationship between defocus and depth. We conduct a robustness analysis and validation on consumer lenses. © 2019 The Author(s) OCIS codes: 080.1010, 120.5710. 1. Mathematical Framework and Solution Our proposed solution is developed in a simple lens framework. Blur is characterized by the radius of a circle of confusion, representing the image of a point light source, and is wavelength dependent. For a wavelength channel C, the blur radius is rC(d)= L |1− x/ fC + x/d|, where L is the simple lens aperture radius, x is the distance between the center of the lens and the sensor plane, fC is the lens’ focal length for channel C and d is the depth of the source point relative to the lens. For a given channel C and sensor position, the focal plane is at depth d0 C, at which the blur radius is minimal: rC(d C)→ 0. This depth is given by d0 C = x(x/ fC−1). Varying the depth d, under a realistic scenario where x > fC, the blur radius is a convex function of d, with a minimum at the focal plane. This creates an ambiguity that was solved recently using coded-aperture lenses [1] or calibrated and memorized for modified lenses over shallow pre-defined depth ranges [2]. Here, we leverage the differential blur across spectral channels. We consider two channels A and B of different wavelength and set w.l.o.g. fB > fA. Noticing that the blur radius is wavelength-dependent, we study the measure ∆B,A(d), rB(d)− rA(d) given by ∆B,A(d) = \uf8f4\uf8f2\uf8f4\uf8f3 α , L(x/ fA− x/ fB) d ≤ d0 A 2L(1+ x/d)−L(x/ fA + x/ fB) d ∈ [d0 A,d B] −α = L(x/ fB− x/ fA) d ≥ d0 B. (1) For d smaller than d0 A or larger than d 0 B, ∆B,A(d) is constant and only depends on the camera parameters (L and x) and the focal lengths of the two spectral channels A and B. In practice, fA and fB are close between RGB channels, making the depths d0 A and d 0 B close, which in turn decreases α . Also, complex lenses are designed to correct color chromatic aberration and minimize the shift between color focal planes. Sec. 3 shows that the shift is nevertheless detectable even with complex lenses, and that with near-infrared (NIR) the corresponding α value becomes noticeably larger, allowing for a more robust solution (Fig. 1). The ∆ measure can be estimated directly from an input image. For a given image patch I(d), captured at wavelength λC, we observe a blurred version Ib(d,λC) = I(d) ∗PSFeq(d,λC,u,v) = I(d) ∗Hde f (d,λC,u,v) ∗H0(u,v), where d is the depth of the object in pixel coordinates (u,v). Hde f represents defocus blur in channel C, and H0 all wavelength-invariant blur (e.g., motion blur). The variance of PSFeq is σ2 eq(d,λC,u,v) = σ2 de f (d,λC,u,v) + σ2 0 (u,v). When the blur radius rC(d) is estimated, σ 2 eq(d,λC,u,v) is approximated instead of the desired σ2 de f (d,λC,u,v), modifying the estimated value of rC(d) by an offset shift [3]. As σ 2 0 (u,v) is invariant with respect to wavelength, the measure ∆B,A(d) defined as σ2 de f (d,λB,u,v)−σ de f (d,λA,u,v), can be estimated by the pixelwise subtraction σ2 eq(d,λB,u,v)−σ eq(d,λA,u,v) that cancels the undesirable offset. The sign of this estimated ∆B,A(d) is sufficient to resolve the blur ambiguity, and obtain a bijective blur-depth mapping. 2. Error Bounds Analysis for the Measure ∆ Channels capture radiation across a wavelength range determined by the sensor’s color filter array. Looking at the intensity present in a channel C, the captured radiation may have any wavelength λC±δC where λC is the central wavelength and δC is a wavelength shift bound to the filter limits. In the extreme case, all radiation captured in the channel C has wavelength λC +δ max C instead of λC, where δ max C is the maximum deviation. δ max C corresponds to a shift γmax C in focal length. We also denote by eC the algorithmic blur estimation error, and obtain the predicted blur 100