Spatio-Colour Asplünd 's Metric and Logarithmic Image Processing for Colour Images (LIPC)
SSpatio-colour Aspl¨und’s metric and LogarithmicImage Processing for Colour Images (LIPC)
Guillaume Noyel and Michel Jourlin , International Prevention Research Institute, 95 cours Lafayette, 69006 Lyon, France Lab. H. Curien, UMR CNRS 5516, 18 rue Pr. B. Lauras, 42000 St-Etienne, France
Abstract.
Aspl¨und’s metric, which is useful for pattern matching, con-sists in a double-sided probing, i.e. the over-graph and the sub-graph ofa function are probed jointly. This paper extends the Aspl¨und’s met-ric we previously defined for colour and multivariate images using amarginal approach (i.e. component by component) to the first spatio-colour Aspl¨und’s metric based on the vectorial colour LIP model (LIPC).LIPC is a non-linear model with operations between colour images whichare consistent with the human visual system. The defined colour metricis insensitive to lighting variations and a variant which is robust to noiseis used for colour pattern matching.
Keywords:
Aspl¨und’s metric, spatio-colour metric, colour LogarithmicImage Processing, double-sided probing, colour pattern recognition
The Aspl¨und’s metric initially defined for binary shapes [1,4] has been extendedto grey-scale images by Jourlin et al. [6,7] and to colour and multivariate imagesin the LIP framework by Noyel et al. [13]. It consists in probing a function bytwo homothetic template functions, i.e. the probes which are computed by theLIP multiplication.The Logarithmic Image Processing (LIP) model initially defined for greylevel images by Jourlin et al. [8,9] is perfectly suited for images acquired bytransmitted light (i.e. when the observed object is located between the sourceand the sensor) and by reflected light because of its consistency with the HumanVision [3]. The necessity to analyse together the channels of the colour images(i.e. by a vectorial analysis) has led to the introduction of the Logarithmic ImageProcessing for Colour images (LIPC) by Jourlin et al. [5].The LIP Aspl¨und’s metric was defined in [13] in a marginal way (i.e. channelby channel). In this paper, our contribution is to extend this metric by using thespatio-colour properties [11,12] of the colour LIPC framework.After some prerequisites about the colour LIPC model and about the marginalLIP Aspl¨und’s metric, we will define a spatio-colour Aspl¨und’s metric in theLIPC framework. Then we will perform spatio-colour pattern matching which isrobust to noise. Examples will illustrate the definitions. a r X i v : . [ c s . C V ] F e b Guillaume Noyel et al.
A colour image f , defined on a domain D ⊂ R N , with values in T = [0 , M [ , M ∈ R , is written: f : (cid:26) D → T = [0 , M [ x → f ( x ) = ( f R ( x ) , f G ( x ) , f B ( x )) (1) f R , f G , f B are the red, green and blue channels (i.e. components) of f , f ( x ) isa vector-pixel and x is the spatial coordinate of the vector-pixel. The real value M is equal to 2 = 256 for 8 bits images. Given P the number of pixels, thematrix F of E → T , E = 3 × P , associated to the image f is written: F = f R ( x ) f R ( x ) ... f R ( x P ) f G ( x ) f G ( x ) ... f G ( x P ) f B ( x ) f B ( x ) ... f B ( x P ) (2)To make the comments easier, the word “image” designates both the matrix F and the image f . The image space for 24-bits images F is written I .A colour image is a particular case of a multivariate image defined as f λ : D → T L , where L ∈ N is the number of channels [11,12].As for the grey-level LIP, the colour LIPC framework is based on colourtransmittance [5]. It is valid for transmitted and reflected images [3]. It modelsthe human perceptual system approach by taking into account: i ) the sensitivityof the human eye in the visible domain characterised by colour matching func-tions of Stiles and Burch (1959) [16] and ii ) the spectral distribution of lightwith the D65 illuminant [14].In the LIPC framework, the transmittance of the sum of two images T F (cid:52) + c G is equal to the product of their transmittances T F and T G : T F (cid:52) + c G = T F ∗ T G . The symbol of the LIPC addition is (cid:52) + c and ∗ represents the element-wisemultiplication [5]. The addition of two images F , G ∈ I is: F (cid:52) + c G = ´K − ´U ( ´U − ´KF ∗ ´U − ´KG ) . (3) ´K and ´U are real matrices of size 3 × . From the LIPC addition, a multiplication by a scalar α ∈ R has been defined: α (cid:52) × c F = ´K − ´U ( ´U − ´KF ) α . (5) With colour matching functions of Stiles and Burch (1959) and D65 illuminant [5],matrices ´K and ´U equal to: ´U = . . . . . . . . . ´K = . . . . . . . . . (4)patio-colour Aspl¨und’s metric with LIPC model 3 The space ( I , (cid:52) + c , (cid:52) × c ) is the positive cone of a vector space with robust math-ematical properties.Physical interpretation [5]: the LIPC addition corresponds to the superposi-tion of two semi-transparent layers. A LIPC multiplication by a scalar α ∈ ]0 , α ∈ ]1 , + ∞ [ darkens theresult by superimposing α times the image on itself. In [13], an Aspl¨und’s metric between colour images was defined with the LIPmodel by using a marginal approach (i.e. channel by channel) [11,12] .
Definition 1.
The Aspl¨und’s metric (with LIP multiplication) between two colourimages f and g on a region Z ⊂ D is d (cid:52) × As,Z ( f , g ) = ln( λ/µ ) (6) with λ = inf { k, ∀ x ∈ Z, k (cid:52) × g R ( x ) ≥ f R ( x ) , k (cid:52) × g G ( x ) ≥ f G ( x ) , k (cid:52) × g B ( x ) ≥ f B ( x ) } and µ = sup { k, ∀ x ∈ Z, k (cid:52) × g R ( x ) ≤ f R ( x ) , k (cid:52) × g G ( x ) ≤ f G ( x ) , k (cid:52) × g B ( x ) ≤ f B ( x ) } . In particular, by the property of the distance d (cid:52) × As,Z ( f , g ) = d (cid:52) × As,Z ( g , f ). Given two colours C = ( r , g , b ), C = ( r , g , b ) ∈ T , as we are only lookingfor lower and upper bounds, a marginal order [2] is used: C ≥ C ⇔ { r ≥ r and g ≥ g and b ≥ b } . Definition 2.
Given two colours C , C ∈ T , their Aspl¨und’s distance (withLIPC multiplication) is equal to: d (cid:52) × c As ( C , C ) = ln( µ/λ ) (7) λ = inf k { k (cid:52) × c C ≥ C } and µ = sup k { k (cid:52) × c C ≤ C } . Strictly speaking, d (cid:52) × c As is a metric if the colours C n are replaced by theirequivalence classes ˜ C n = (cid:8) C ∈ T / ∃ α ∈ R + , α (cid:52) × c C = C n } .Comment: in eq. 7 contrary to the Aspl¨und’s distance (with LIP multiplica-tion) defined in [13] (eq. 6), we have λ ≤ µ because, by definition of the colourLIPC model the scales are inverted as compared to the grey LIP model [5].Colour metrics (with LIPC multiplication) between two colour images f and g may be defined as the sum ( d metric) or the supremum ( d ∞ ) of d (cid:52) × c As ( C , C )on the region of interest Z ⊂ D of cardinal Zd (cid:52) × c ,Z ( f , g ) = Z (cid:80) x ∈ Z d (cid:52) × c As ( f ( x ) , g ( x )) d (cid:52) × c ∞ ,Z ( f , g ) = sup x ∈ Z d (cid:52) × c As ( f ( x ) , g ( x )) (8)The Aspl¨und’s metric can be extended to colour functions. Guillaume Noyel et al.
Definition 3.
The colour Aspl¨und’s metric (with LIPC multiplication) betweentwo colour images f and g on a region Z ⊂ D is d (cid:52) × c As,Z ( f , g ) = ln( µ/λ ) (9) λ = inf k {∀ x ∈ Z, k (cid:52) × c g ( x ) ≥ f ( x ) } and µ = sup k {∀ x ∈ Z, k (cid:52) × c g ( x ) ≤ f ( x ) } . In fig. 1, the Aspl¨und’s metric has been computed between the colour probe g and the colour function f on their definition domain D . redgreenblueIntensity x redgreenblueIntensity x fg lower boundupper bound x Intensity (a) Colour function f (b) Colour probe g (c) Lower ( µ ) and upper ( λ ) bounds Fig. 1.
Computation of the Aspl¨und’s distance between two colour functions d (cid:52) × c As,D ( f , g ) = 0 .
43. Each colour channel is represented by a line of the same colour.
Comment: the lower (resp. upper) bound µ (cid:52) × c g (resp. λ (cid:52) × c g ) may notbe equal to any point of the function f but strictly less (or greater) than thefunction. Indeed, one can demonstrate that the following assertion is verified:“given C ∈ T , ∀ C ∈ T , (cid:54) ∃ λ ∈ R + /λ (cid:52) × c C = C ”.The metric d (cid:52) × c As,Z can be adapted to local processing with a colour templateimage (i.e. the probe) t defined on a spatial support D t ⊂ D . For each point x ∈ D , the distance d (cid:52) × c As,D t ( f | D t ( x ) , t ) is computed on the neighbourhood D t ( x )centred in x where f | D t ( x ) is the restriction of f to D t ( x ). Definition 4.
Given a colour image f defined on D with values in T , (cid:0) T (cid:1) D ,a colour probe t defined on D t with values in T , (cid:0) T (cid:1) D t , and D t ( x ) the neigh-bourhood D t centred in x ∈ D , the map of Aspl¨und’s distances (with (cid:52) × c ) is: As (cid:52) × c t f : (cid:40) (cid:0) T (cid:1) D × (cid:0) T (cid:1) D t → ( R + ) D ( f , t ) → As (cid:52) × c t f ( x ) = d (cid:52) × c As,D t ( f | D t ( x ) , t ) (10)In figure 2, the map of Aspl¨und’s distances is computed between a colourfunction and a colour probe. The minima of the map corresponds to the locationof a pattern which is similar to the probe.Aspl¨und’s distance is sensitive to noise because the probe lays on regional ex-trema that may be caused by noise (Figure 1). In [7,13], definitions of Aspl¨und’sdistance with a tolerance on the extrema have been introduced. In this paper,we extend this definition for colour images with LIPC model. patio-colour Aspl¨und’s metric with LIPC model 5 redgreenblueIntensity x redgreenblueIntensity x Intensity x (a) Colour function f (b) Colour probe t (c) Map of Aspl¨und’sdistance As (cid:52) × c t fFig. 2. (c) Map of the Aspl¨und’s distances As (cid:52) × c t f between a colour function and aprobe. (a) and (b) Each colour channel is represented by a line of the same colour. To reduce the sensitivity of Aspl¨und’s distance to the noise, the “Measuremetric” or “M-metric” has been defined in the context of “Measure Theory”.The image being digitized, the number of pixels of D is finite and the “measure”of a subset of D is linked to the cardinal of this subset, e.g. the percentage P ofits elements with respect to D . We are looking for a subset D (cid:48) of D , such that f | D (cid:48) and g | D (cid:48) are neighbours for Aspl¨und’s metric and the complementary set D \ D (cid:48) of D (cid:48) into D is of small size when compared to D . This last condition iswritten as: P ( D \ D (cid:48) ) = D \ D (cid:48) ) D ≤ p , where p is an acceptable percentage and D is the number of elements in D .Given (cid:15) a small positive real number, the neighbourhood of function f is N P,d As ,(cid:15),p ( f ) = (cid:26) g \ ∃ D (cid:48) ⊂ D, d (cid:52) × c As,D (cid:48) ( f | D (cid:48) , g | D (cid:48) ) < (cid:15) and D \ D (cid:48) ) D ≤ p (cid:27) (11)The closest points of the probe to the function are discarded as in [5,13]. Definition 5.
Given two constant vector-pixels c µ , c λ ∈ T , a percentage p ofpoints to be discarded. The colour Aspl¨und’s metric (with LIPC multiplication)with tolerance between two colour images f and g on a region Z ⊂ D is d (cid:52) × c As,Z,p ( f , g ) = ln( µ (cid:48) /λ (cid:48) ) (12) λ (cid:48) = inf k {∀ x ∈ Z, k (cid:52) × c g ( x ) ≥ f ( x ) − c λ } and µ (cid:48) = sup k {∀ x ∈ Z, k (cid:52) × c g ( x ) ≤ f ( x ) + c µ } . c µ and c λ are increased such as a percentage p of points is discarded. In figure 3, a tolerance of p = 20% is used to discard two points. TheAspl¨und’s distance decreases from 0 .
43 to 0 . (cid:52) × c ) can now be defined. Definition 6.
Given a colour image f of (cid:0) T (cid:1) D , a colour probe t of (cid:0) T (cid:1) D t and a tolerance p ∈ [0 , , the map of Aspl¨und’s distances with a tolerance is: As (cid:52) × c t ,p f : (cid:40) (cid:0) T (cid:1) D × (cid:0) T (cid:1) D t → ( R + ) D ( f , t ) → As (cid:52) × c t ,p f ( x ) = d (cid:52) × c As,D t ,p ( f | D t ( x ) , t ) (13) D t ( x ) is the neighbourhood D t centred in x ∈ D . Guillaume Noyel et al. redgreenblueIntensity x redgreenblue Intensity x f λ * g µ * g λ ' * g µ ' * g Intensity x (a) Colour function f (b) Colour probe g (c) Lower and upperbounds, p = 20% Fig. 3.
Colour Aspl¨und’s distance with a tolerance of p = 20%. ( µ , λ ) are the scalarsmultiplying the probe without tolerance. ( µ (cid:48) , λ (cid:48) ) are the scalars multiplying the probewith tolerance. d (cid:52) × c As,D ( f , g ) = 0 .
43 and d (cid:52) × c As,D,p =20% ( f , g ) = 0 . (a) Image f and probe t (b) Map As (cid:52) × c t f (c) Map As (cid:52) × c t ,p =98% ˜ f (d) Noisy image ˜ f (e) Map As (cid:52) × c t ˜ f (f) Correlation map Fig. 4.
Maps of Aspl¨und’s distances without tolerance As (cid:52) × c t ˜ f and with As (cid:52) × c t ,p ˜ f . ˜ f imagewith a white noise ( σ = 2 .
6, spatial density 1%). (f) Correlation map.
In figure 4, we look for the bricks of a wall, similar to a colour probe. A bluebrick has been added to the wall. In the image without noise f , the regionalminima of the map As (cid:52) × c t f (dark points in fig. 4b) correspond to the centre of patio-colour Aspl¨und’s metric with LIPC model 7 the bricks similar to the probe (according to the Aspl¨und’s distance). The whiterectangle corresponds to the maxima of the distances between the blue brickand the probe. Therefore, the distance is sensitive to colour (i.e. the hue). In theimage with noise ˜ f , the map without tolerance As (cid:52) × c t ˜ f is more sensitive to noise(fig. 4e) than the map with tolerance As (cid:52) × c t ,p ˜ f (fig 4c). Indeed, the minima arepreserved into the map with tolerance (fig. 4c) compared to the map without(fig. 4e). The minima can be extracted using mathematical morphology [10,15].Importantly, all the maps of Aspl¨und’s distances are insensitive to the verticallighting drift. Moreover, a correlation map is useless to find the location of thebricks (fig. 4f).In figure 5, two images of the same scene, a bright image f and a dark image˜ f , are acquired with two different exposure times. The probe t is extracted inthe bright image and used to compute the map of Aspl¨und’s distance As (cid:52) × c t ˜ f inthe darker image. By finding the minima of the map, all the balls are detectedand their contours are added to the image in figure 5 (b). One can notice thatthe Aspl¨und’s distance is very robust to the lighting variations. (a) Initial image f (b) Dark image ˜ f (c) Map As (cid:52) × c t ˜ f and probe t Balls detected
Fig. 5.
Detection of coloured balls on a dark image ˜ f with a probe t extracted in thebright image f . (a) The border of the probe t is coloured in white. A new spatio-colour Aspl¨und’s distance based on colour LIPC model has beendefined. It is a true colour (i.e. vectorial) metric based on a colour model con-sistent with the human visual system. It is also consistent with the previousproperties given in [7,13]. An extension of this metric robust to noise has beenpresented and illustrated on pattern recognition examples. This double-sided
Guillaume Noyel et al. probing distance is efficient for colour pattern matching and performs betterthan traditional correlation methods. In future work, we will evaluate in detailsthe properties of this colour distance on practical applications (e.g. in medical,remote sensing or industrial images). We will compare it to the marginal colourAspl¨und’s distance and we will study the links between Aspl¨und’s probing andmathematical morphology.