Geometric Moment Invariants to Motion Blur
aa r X i v : . [ c s . C G ] J a n Geometric Moment Invariants to Motion Blur
Hongxiang Hao. a,b, ∗ , Hanlin Mo. a,b , Hua Li. a,b a Key Laboratory of Intelligent Information Processing, Institute ofComputing Technology, Chinese Academy of Sciences, Beijing, China b University of Chinese Academy of Sciences, Beijing, China
Abstract
In this paper, we focus on removing interference of motion blur by the deriva-tion of motion blur invariants.Unlike earlier work, we don’t restore any blurredimage. Based on geometric moment and mathematical model of motion blur,we prove that geometric moments of blurred image and original image are lin-early related. Depending on this property, we can analyse whether an existingmoment-based feature is invariant to motion blur. Surprisingly, we find some ge-ometric moment invariants are invariants to not only spatial transform but alsomotion blur. Meanwhile, we test invariance and robustness of these invariantsusing synthetic and real blur image datasets. And the results show these in-variants outperform some widely used blur moment invariants and non-momentimage features in image retrieval, classification and template matching.
Keywords: i nvariance, geometric moment invariant, rotational motion blur,linear motion blur, feature extraction, image retrieval
1. Introduction
Image analysis and recognition is one of the most fundamental goals in manyfields including computer vision. Researchers always expect high-quality images,but many factors can cause degradation and interference during the process ofimaging. Among these factors, blur is the most common, including atmospheric ∗ Corresponding author
Email address: [email protected] (Hongxiang Hao.)
Preprint submitted to Journal of L A TEX Templates January 26, 2021 nterference, out of focus, relative motion between camera and scene, and otherways of degradation. Actually, blur is an irreversible process with the abnormaldiffusion and superposition of image pixels. In general, image degradation canbe described by the following mathematical model[1]: g ( x ′ , y ′ ) = f ∗ h ( x, y ) + n ( x, y ) (1)where h stands for the point-spread function (PSF) of the imaging system.Motion blur is a common degradation. It can be modeled as the integra-tion over time of image density function. According to form of relative motionbetween the object and the imaging system during exposure time, motion blurcan be divided into 3 categories, such as linear motion blur, rotational motionblur and radial motion blur, which is more complex.To remove interference of motion blur , most existing work focuses on therealization of ”reversibility” and try to restore blurred image, named as “DE-BLUR”. Although, many researchers have obtained excellent results and widelyused them into many fields, they still faces many challenges. For example, com-plicated algorithm, heavy computation,probability of introducing new noisesand so on.In this paper, we adapt another approach of removing influences causedby motion blur, as known as extracting features that are invariant to motionblur. We focus on achieving “deblur-free” and conducting an ideal system ofrecognition. So our work mainly consists of three parts : First, we analyze themathematical model of motion blur, combine it with the concept of geomet-ric moments ; Then, we derive that geometric moments of blurred image andoriginal image are linearly relative and screen out several moment invariants de-pending on the derivation ; Finally, we test invariance and robustness of theseinvariants using synthetic and real blur image datasets.2 . Basic concepts and theories In this section, we introduce some basic definitions and classical conclusionsfor our work.
Based on the model (1), we can analyze the PSF of motion blur and deriveanother model with integration over time instead of convolution. We find themodel with integration can describe motion blur more directly and solve so muchmathematical problems caused by the complex convolution.So in our work, thedegradation model can be defined as[2]: g ( x ( g ) , y ( g ) ) = 1 T Z T f ( x ( t ) , y ( t ) ) dt (2)Where T stands for the exposure time, g stands for the blurred image andf stands for the image after corresponding spatial transform. In image den-sity functions g ,( x ( g ) , y ( g ) ) stands for point coordinates of blurred image; and( x ( t ) , y ( t ) ) in f stands for point coordinates of image during spatial transform.According to the model (2), we can regard linear motion blur as the diffusionand superposition of a series image translation transform.Supposing a and bstands for linear velocity of translation in x axis and in y axis direction. So therelations of point coordinates among original image, moving image and blurredimage can be derived as: x ( t ) = x + aty ( t ) = y + bt x ( g ) = x + aTy ( g ) = y + bT x ( g ) = x ( t ) − at + aTy ( g ) = y ( t ) − bt + bT (3)When we focus on rotational motion blu, we can see it as the diffusion andsuperposition of a series image rotation transform.Supposing ω stands for angu-lar velocity of rotation, α stands for deflection angle before and after rotationblur. So we can build the relations of point coordinates among original image,3oving image and blurred image as: x ( g ) y ( g ) = cos α − sin α sin α cos α · xy x ( t ) y ( t ) = cos φ ( t ) − sin φ ( t ) sin φ ( t ) cos φ ( t ) · xy x ( t ) y ( t ) = cos θ ( t ) − sin θ ( t ) sin θ ( t ) cos θ ( t ) · x ( g ) y ( g ) (4)where φ ( t ) = ωt and θ ( t ) = φ ( t ) − α = ωt − α . Geometric moments are regarded as the simplest image moments.For animage f ( x, y ), its geometric moments of (p+q)th order can be defined as: m ( f ) pq = Z Z x p y q f ( x, y ) dxdy (5)where p and q are non-negative integers. Based on (5), we can define centroidof the image as: x c = m ( f )10 m ( f )00 y c = m ( f )01 m ( f )00 (6)When we move the center of the image to centroid and re-establish thecoordinate system, the influence of translation transform can be easily removed.So we define geometric central moments of (p+q)th order as: u ( f ) pq = Z Z ( x − x c ) p ( y − y c ) q f ( x, y ) dxdy (7)It’s obvious that first central moments of any image equals zero accordingto definition (7). In 1962, Hu first proposed the concept of image geometric moments[3]. Heemployed the results of the theory of algebratic invariants and derived 7 ge-ometric moment invariants to similarity transform (including translation,scale4nd rotation) as known as Hu moment invariants. ϕ = η + η ; ϕ = ( η − η ) + 4 η ; ϕ = ( η − η ) + (3 η − η ) ; ϕ = ( η + η ) + ( η + η ) ; ϕ = ( η − η )( η + η ) h ( η + η ) − η + η ) i + (3 η − η ) · ( η + η ) h η + η ) − ( η + η ) i ; ϕ = ( η − η ) h ( η + η ) − ( η + η ) i + 4 η ( η − η )( η + η ) · h η + η ) − ( η + η ) i ; ϕ = (3 η − η )( η + η ) h ( η + η ) − η + η ) i + ( η − η ) · ( η + η ) h η + η ) − ( η + η ) i ; (8)These 7 invariants have become much popular image descriptors and havefound numerous applications, namely in shape analysis[4][5], object recognition[6][7],and speech analysis[8].After being widely used for nearly forty years, Flusser found this system isnot independent[9]. They showed that 7th invariant can be expressed as thefunction of the others: ( ϕ = ϕ · ( ϕ − ( ϕ (9)which indicates that 7th invariant can be excluded from the set without any lossof discrimination power So we only discuss 1st - 6th invariants in this paper.5 . Geometric moment invariants to linear motion blur Based on equation (3) and definition(2) (7), we can derive centroid of theblurred image as:¯ x ( g ) = RR (cid:13) x ( g ) · g ( x ( g ) , y ( g ) ) dx ( g ) dy ( g ) RR (cid:13) g ( x ( g ) , y ( g ) ) dx ( g ) dy ( g ) = RR (cid:13) n(cid:0) x ( t ) − at + aT (cid:1) · h T R T f ( x ( t ) , y ( t ) ) dt io dx ( t ) dy ( t ) RR (cid:13) f ( x ( t ) , y ( t ) ) dx ( t ) dy ( t ) = T R T (cid:8)(cid:2)RR (cid:13) (cid:2)(cid:0) x ( t ) − at + aT (cid:1) · f ( x ( t ) , y ( t ) ) (cid:3) dx ( t ) dy ( t ) (cid:3)(cid:9) dtm ( t )00 = T R T h m ( t )10 + ( − at + aT ) · m ( t )00 i dtm ( t )00 = ¯ x ( t ) + 12 aT ¯ y ( g ) = RR (cid:13) y ( g ) · g ( x ( g ) , y ( g ) ) dx ( g ) dy ( g ) RR (cid:13) g ( x ( g ) , y ( g ) ) dx ( g ) dy ( g ) = RR (cid:13) n(cid:0) y ( t ) − bt + bT (cid:1) · h T R T f ( x ( t ) , y ( t ) ) dt io dx ( t ) dy ( t ) RR (cid:13) f ( x ( t ) , y ( t ) ) dx ( t ) dy ( t ) = T R T (cid:8)(cid:2)RR (cid:13) (cid:2)(cid:0) y ( t ) − bt + bT (cid:1) · f ( x ( t ) , y ( t ) ) (cid:3) dx ( t ) dy ( t ) (cid:3)(cid:9) dtm ( t )00 = T R T h m ( t )01 + ( − bt + bT ) · m ( t )00 i dtm ( t )00 = ¯ y ( t ) + 12 bT (10)6hen we can define (p+q)th-order geometric central moments of blurredimage as: u ( g ) pq = ZZ (cid:13) (cid:0) x ( g ) − ¯ x ( g ) (cid:1) p (cid:0) y ( g ) − ¯ y ( g ) (cid:1) q g ( x ( g ) , y ( g ) ) dx ( g ) dy ( g ) = ZZ (cid:13) ( T "Z T f ( x ( t ) , y ( t ) ) dt · (cid:18) x ( t ) − ¯ x ( t ) − at + 12 aT (cid:19) p · (cid:18) y ( t ) − ¯ y ( t ) − bt + 12 bT (cid:19) q (cid:27) dx ( t ) dy ( t ) = 1 T Z T (cid:26) f ( x ( t ) , y ( t ) ) · ZZ (cid:13) (cid:20)(cid:18) x ( t ) − ¯ x ( t ) − at + 12 aT (cid:19) p · (cid:18) y ( t ) − ¯ y ( t ) − bt + 12 bT (cid:19) q (cid:21) dx ( t ) dy ( t ) (cid:27) dt = 1 T Z T (ZZ (cid:13) " f ( x ( t ) , y ( t ) ) · p X i =0 C ip · (cid:0) x ( t ) − ¯ x ( t ) (cid:1) i · (cid:18) − at + 12 aT (cid:19) p − i ! · q X j =0 C jq · (cid:0) y ( t ) − ¯ y ( t ) (cid:1) j · (cid:18) − bt + 12 bT (cid:19) q − j ! dx ( t ) dy ( t ) dt = 1 T Z T i ≤ p,j ≤ q X i = j =0 (cid:18) u ( f ) ij · C ip · C jq · a p − i · b q − j · ( T − t ) p + q − i − j (cid:19) dt = i ≤ p,j ≤ q X i = j =0 u ( f ) ij · C ip · C jq · a p − i · b q − j · T Z T (cid:20) ( T − t ) p + q − i − j (cid:21) dt ! = i ≤ p,j ≤ q X i = j =0 u ( f ) ij · C ip · C jq · a p − i · b q − j · (cid:18) T (cid:19) p + q − i − j · − p + q − i − j · ( p + q − i − j + 1) ! (11)According the simplified result in the following definition (12), we can learnthat geometric central moments of linear-motion-blurred image and originalimage are linearly relative. u ( g ) pq = i ≤ p,j ≤ q X i = j =0 (cid:16) u ( f ) ij · H ( p, q, i, j ) (cid:17) H ( p, q, i, j ) = C ip · C jq · a p − i · b q − j · S ( p, q, i, j ) S ( p, q, i, j ) = when ( p + q − i − j ) is odd ( T /2) p + q − i − j p + q − i − j +1 when ( p + q − i − j ) is even (12)7ased on the definition, we can obtain different moments of any order.Inthis paper, we give central moments from 1st to 4th order:1 st : u ( g )10 = 0 u ( g )01 = 02 nd : u ( g )20 = u ( f )20 + u ( f )00 · · (cid:18) aT (cid:19) u ( g )11 = u ( f )11 + u ( f )00 · · aT · bT u ( g )02 = u ( f )02 + u ( f )00 · · (cid:18) bT (cid:19) rd : u ( g )30 = u ( f )30 u ( g )21 = u ( f )21 u ( g )12 = u ( f )12 u ( g )03 = u ( f )03 th : u ( g )40 = u ( f )40 + u ( f )20 · · (cid:18) aT (cid:19) + u ( f )00 · · (cid:18) aT (cid:19) u ( g )31 = u ( f )31 + u ( f )20 · aT · bT u ( f )11 · (cid:18) aT (cid:19) + u ( f )00 · · (cid:18) aT (cid:19) · bT u ( g )22 = u ( f )22 + u ( f )20 · · (cid:18) bT (cid:19) + u ( f )02 · · (cid:18) aT (cid:19) + u ( f )11 · · aT · bT u ( f )00 · · (cid:18) aT (cid:19) · (cid:18) bT (cid:19) u ( g )13 = u ( f )13 + u ( f )02 · aT · bT u ( f )11 · (cid:18) bT (cid:19) + u ( f )00 · · aT · (cid:18) bT (cid:19) u ( g )04 = u ( f )04 + u ( f )02 · · (cid:18) bT (cid:19) + u ( f )00 · · (cid:18) bT (cid:19) (13)According to the results shown in equation (13), we learn that four 3rd-ordergeometric central moments are natural invariants to linear motion blur. So anyinvariant that consists of only 3rd-order geometric central moments also hasinvariance to linear motion blur. 8 . Geometric moment invariants to rotational motion blur Based on equation (4) and definition(2)(7), we can define (p+q)th-ordergeometric central moments of blurred image as: m ( g ) pq = ZZ (cid:13) (cid:20)Z x p ( g ) y q ( g ) g ( x ( g ) , y ( g ) ) (cid:21) dx ( g ) dy ( g ) = ZZ (cid:13) ( x p ( g ) y q ( g ) · " T Z T f ( x ( t ) , y ( t ) ) dt dx ( g ) dy ( g ) = 1 T Z T (cid:26)ZZ (cid:13) h x p ( g ) y q ( g ) f ( x ( t ) , y ( t ) ) i dx ( g ) dy ( g ) (cid:27) dt = 1 T Z T (cid:26)ZZ (cid:13) (cid:2) f ( x ( t ) , y ( t ) ) · ( x ( t ) cos θ ( t ) + y ( t ) sin θ ( t ) ) p · ( − x ( t ) sin θ ( t ) + y ( t ) cos θ ( t ) ) q (cid:3) dx ( t ) dy ( t ) (cid:9) dt = 1 T Z T (ZZ (cid:13) " f ( x ( t ) , y ( t ) ) · p X i =0 (cid:16) C ip · x i ( t ) cos i θ ( t ) · y p − i ( t ) sin p − i θ ( t ) (cid:17) · q X j =0 (cid:16) ( − j · C jq · x j ( t ) sin j θ ( t ) · y q − j ( t ) cos q − j θ ( t ) (cid:17) dx ( t ) dy ( t ) dt = 1 T Z T (ZZ (cid:13) " p + q X k =0 " k X i =0 (cid:16) C ip · cos i θ ( t ) · sin p − i θ ( t ) · ( − k − i · C k − iq · sin k − i θ ( t ) · cos q + i − k θ ( t ) (cid:17) · x k ( t ) y p + q − k ( t ) i · f ( x ( t ) , y ( t ) ) i dx ( t ) dy ( t ) o dt when i ≤ p and k − i ≤ q = p + q X k =0 ( m ( f ) k,p + q − k k X i =0 " ( − k − i C ip C k − iq · T Z T (cid:0) cos q +2 i − k θ ( t ) · sin p + k − i θ ( t ) (cid:1) dt (14)According the simplified result in the following definition (15), we can learnthat geometric central moments of rotational-motion-blurred image and original9mage are linearly relative. m ( g ) pq = p + q X k =0 n m ( f ) k,p + q − k · H ( p, q, k ) o H ( p, q, k ) = k X i =0 [ S p, q, k, i ) · S p, q, k, i )] S p, q, k, i ) = ( − k − i C ip C k − iq when i ≤ p and k − i ≤ q otherwiseS p, q, k, i ) = 1 T Z T (cid:0) cos q +2 i − k θ ( t ) · sin p + k − i θ ( t ) (cid:1) dt (15)Depending on definition (15), we make further efforts to simplify derivedresults, so we have: S p, q, k, i ) = 1 T Z T (cid:0) cos q +2 i − k θ ( t ) · sin p + k − i θ ( t ) (cid:1) dt = 1 T Z T (cid:0) cos p + q − x θ ( t ) · sin x θ ( t ) (cid:1) dt ( supposing x = p + k − i )= 1 ωT Z ωT (cid:0) cos p + q − x θ · sin x θ (cid:1) dθ = 1 ωT Z ωT (cid:0) cos p + q θ · tan x θ (cid:1) dθ (16)Based on the definition, we can obtain different moments of any order. Inthis paper, we give central moments from 1st to 2th order.1 st : m ( g )10 = 1 ωT h m ( f )10 · sin( ωT ) + m ( f )01 · (1 − cos( ωT )) i m ( g )01 = 1 ωT h m ( f )10 · ( − ωT )) + m ( f )01 · sin( ωT )) i nd : m ( g )20 = 12 ωT h m ( f )20 · (cos( ωT ) sin( ωT ) + ωT ) + 2 m ( f )11 · (sin( ωT )) + m ( f )02 · ( − cos( ωT ) sin( ωT ) + ωT ) i m ( g )11 = 12 ωT h − m ( f )20 · (sin( ωT )) + 2 m ( f )11 · cos( ωT ) sin( ωT ) + m ( f )02 · (sin( ωT )) i m ( g )02 = 12 ωT h m ( f )20 · ( − cos( ωT ) sin( ωT ) + ωT ) − m ( f )11 · (sin( ωT )) + m ( f )02 · (cos( ωT ) sin( ωT ) + ωT ) i (17)10 .2. Geometric moment invariants to rotational motion blur After screening out 6 Hu moment invariants and all rotation moment in-varaints that are given in Mo’s paper[10], we find two absolute invaraints andten relative invaraints that might have great invariace and robustness to rota-tional motion blur.
RMBMI − m
20 + m RMBMI − m
40 + 2 m
22 + m RMBMI − m · m
03 + m · m
21 + m · m
12 + m · m m + m RMBMI − m · m
12 + m · m − m · m − m · m m + m RMBMI − m + m ) + ( m + m ) m + m RMBMI − m · m
13 + m · m − m · m
11 + m · m − m · m − m · m m − m ) + 4 m RMBMI − m · m
13 + m · m − m · m · m
10 + m · m · m − m · m − m · m m · m − m · m · m
10 + m · m · m − m · m . Conclusions In this paper, we first combine the concept of geometric moment and mo-tion blur degradation model; then we prove that geometric moments of blurredimage and original image are linearly related. Depending on this property, wegive an approach of analysing whether an existing moment-based feature is in-variant to motion blur. Finally, we find some geometric moment invariants areinvariants to not only spatial transform but also motion blur. Meanwhile, wetest invariance and robustness of these invariants using synthetic and real blurimage datasets.
Acknowledgements
This work was partly funded by National Key R&D Program of China(no.2017YFB1002703), National Key Basic Research Planning Project of China(no. 2015CB554507) and National Natural Science Foundation of China (no.61227802and 61379082). 12 eferenceseferences