An application of a pseudo-parabolic modeling to texture image recognition
AAn application of a pseudo-parabolic modelingto texture image recognition (cid:63)
Joao B. Florindo − − − and Eduardo Abreu − − − Institute of Mathematics, Statistics and Scientific Computing - University ofCampinas, Rua S´ergio Buarque de Holanda, 651, Cidade Universit´aria ”Zeferino Vaz”- Distr. Bar˜ao Geraldo, CEP 13083-859, Campinas, SP, Brasil { florindo,eabreu } @unicamp.br Abstract.
In this work, we present a novel methodology for textureimage recognition using a partial differential equation modeling. Morespecifically, we employ the pseudo-parabolic Buckley-Leverett equationto provide a dynamics to the digital image representation and collect lo-cal descriptors from those images evolving in time. For the local descrip-tors we employ the magnitude and signal binary patterns and a simplehistogram of these features was capable of achieving promising resultsin a classification task. We compare the accuracy over well establishedbenchmark texture databases and the results demonstrate competitive-ness, even with the most modern deep learning approaches. The achievedresults open space for future investigation on this type of modeling forimage analysis, especially when there is no large amount of data for trai-ning deep learning models and therefore model-based approaches ariseas suitable alternatives.
Keywords:
Pseudo-parabolic equation · Texture recognition · Imageclassification · Computational Methods for PDEs.
Texture images (also known as visual textures) can be informally defined as thoseimages in which the most relevant information is not encapsulated within one ora limited set of well-defined objects, but rather all pixels share the same impor-tance in their description. This type of image has found numerous application inmaterial sciences [18], medicine [10], facial recognition [14], remote sensing [32],cybersecurity [26], and agriculture [23] to name but a few fields with increasingresearch activity.While deep learning approaches have achieved remarkable success in pro-blems of object recognition and variations of convolutional neural networks haveprevailed in the state-of-the-art for this task, texture recognition on the otherhand still remains a challenging problem and the classical paradigm of local (cid:63)
Supported by S˜ao Paulo Research Foundation (FAPESP), National Council for Sci-entific and Technological Development, Brazil (CNPq), and PETROBRAS - Brazil. a r X i v : . [ c s . C V ] F e b Florindo and Abreu image encoders still is competitive with the most modern deep neural networks,presenting some advantages over the last ones, like the fact that they can workwell even when there is little data available for training.In this context, here we present a local texture descriptor based on the actionof an operator derived from the Buckley-Leverett partial differential equation(PDE) (see [1,2] and references cited therein). PDE models have been employedin computer vision at least since the 1980’s, especially in image processing. Thescale-space theory developed by Witkin [30] and Koenderink [16] are remarkableexamples of such applications. The anisotropic diffusion equation of Perona andMalik [22] also represented great advancement in that research front, as it solvedthe problem of edge smoothing, common in classical diffusion models. Evolutionsof this model were later presented and a survey on this topic was developed in[29].Despite these applications of PDEs in image processing, substantially lessresearch has been devoted to recognition. As illustrated in [28], pseudo-parabolicPDEs are promising models for this purpose. An important characteristic of thesemodels is that jump discontinuities in the initial condition are replicated in thesolution [9]. This is an important feature in recognition as it allows some controlover the smoothing effect and would preserve relevant edges, which are knownto be very important in image description.Based on this context, we propose the use of Buckley-Leverett equation asan operator acting as a nonlinear filter over the texture image. That image isused as initial condition for the PDE problem and the solution obtained by anumerical scheme developed in [1] is used to compose the image representation.The solution at each time is encoded by a local descriptor. Extending the ideapresented in [28], here we propose two local features: the sign and the magni-tude binary patterns [12]. The final texture descriptors are provided by simpleconcatenation of histograms over each time.The effectiveness of the proposed descriptors is validated on the classificationof well established benchmark texture datasets, more exactly, KTH-TIPS-2b [13]and UIUC [17]. The accuracy is compared with the state-of-the-art in texturerecognition, including deep learning solutions, and the results demonstrate thepotential of our approach, being competitive with the most advanced solutionsrecently published on this topic.
We consider an advanced simulation approach for the pseudo-parabolic PDE ∂u∂t = ∇ · w , where w = g ( x, y, t ) ∇ (cid:18) u + τ ∂u∂t (cid:19) , (1)and let Ω ⊂ R denote a rectangular domain and u ( · , · , t ) : Ω → R be a se-quence of images that satisfies the pseudo-parabolic equation (1), in which theoriginal image at t = 0 corresponds to the initial condition, along with zero fluxcondition across the domain boundary ∂Ω , w · n = 0 , ( x , y ) ∈ ∂Ω . Following n application of a pseudo-parabolic modeling to texture image recognition 3 [28] (see also [27,1]), we consider the discretization modeling of the PDE (1) in auniform partition of Ω into rectangular subdomains Ω i,j , for i = 1 , . . . , m and j = 1 , . . . , l , with dimensions ∆x × ∆y . The center of each subdomain Ω i,j isdenoted by ( x i , y j ). Given a final time of simulation T , consider a uniform par-tition of the interval [0 , T ] into N subintervals, where the time step ∆t = T /N is subject to a stability condition (see [28,27] for details). We denote the textureconfiguration frames in the time levels t n = n∆t , for n = 0 , . . . , N . Let U ni,j and W n +1 i,j be a finite difference approximations for u ( x i , y j , t n ) and w , respectively,and both related to the pseudo-parabolic PDE modeling of (1). Motivated byour promising results in [28], we employ a stable cell-centered finite differencediscretization in space after applying the backward Euler method in time to (1),yielding U n +1 i,j − U ni,j ∆t = W n +1 i + ,j − W n +1 i − ,j ∆x + W n +1 i,j + − W n +1 i,j − ∆y . (2)Depending on the application as well as the calibration data and texture para-meters upon model (1), we will have linear or nonlinear diffusion models forimage processing (see, e.g., [22,5,29]). As a result of this process the discreteproblem (2) would be linear-like A n U n +1 = b n or nonlinear-like F ( U n +1 ) = 0and several interesting methods can be used (see, e.g., [2,28,27,22,5,29]). Wewould like to point out at this moment that our contribution relies on the PDEmodeling of (1) as well as on the calibration data and texture parameters associ-ated to the pseudo-parabolic modeling in conjunction with a fine tunning of thelocal descriptors for texture image recognition for the pertinent application un-der consideration. In summary, we have a family of parameter choice strategiesthat combines pseudo-parabolic modeling with texture image recognition.Here, we consider the diffusive flux as g ( x , y , t ) ≡
1, which results (1) tobe a linear pseudo-parabolic model. For a texture image classification based ona pseudo-parabolic diffusion model to be processed, we just consider that eachsubdomain Ω i,j corresponds to a pixel with ∆x = ∆y = 1. As we perform animplicit robust discretization in time (backward Euler), we simply choose thetime step ∆t = ∆x and the damping coefficient τ = 5. More details can befound in [28]; see also [27,1].Therefore, this description summarizes the basic key ideas of our compu-tational PDE modeling approach for texture image classification based on apseudo-parabolic diffusion model (1). Inspired by ideas presented in [28] and significantly extending comprehensionon that study, here we propose the development of a family of images { u k } Kk =1 .These images are obtained by introducing the original image u as initial con-dition for the 2D pseudo-parabolic numerical scheme presented in Section 2. u k is the numerical solution at each time t = t k . Here, K = 50 showed to be areasonable balance between computational performance and description quality. Florindo and Abreu
Following that, we collected two types of local binary descriptors [12] fromeach u k . More exactly, we used sign LBP S riu P,R and magnitude
LBP M riu P,R descriptors. In short, the local binary sign pattern
LBP S riu P,R for each imagepixel with gray level g c and whose neighbor pixels at distance R have intensities g p ( p = 1 , · · · , P ) is given by LBP S riu P,R = (cid:26) (cid:80) P − p =0 H ( g p − g c )2 p if U ( LBP
P,R ) ≥ P + 1 otherwise , (3)where H corresponds to the Heaviside step function ( H ( x ) = 1 if x ≥ H ( x ) = 0 if x <
0) and U is the uniformity function, defined by U ( LBP
P,R ) = | H ( g P − − g c ) − H ( g − g c ) | + P − (cid:88) p =1 | H ( g p − g c ) − H ( g p − − g c ) | . (4)Similarly, the magnitude local descriptor is defined by LBP M riu P,R = (cid:26) (cid:80) P − p =0 t ( | g p − g c | , C )2 p if U ( LBP
P,R ) ≥ P + 1 otherwise , (5)where C is the mean value of | g p − g c | over the whole image and t is a thresholdfunction, such that t ( x, c ) = 1 if x ≥ c and t ( x, c ) = 0, otherwise.Finally, we compute the histogram h of LBP S riu P,R ( u k ) and LBP M riu P,R ( u k )for the following pairs of ( P, R ) values: { (1 , , (2 , , (3 , , (4 , } . The pro-posed descriptors can be summarized by D ( u ) = (cid:91) type = { S,M } (cid:91) ( P,R )= { (8 , , (16 , , (24 , , (24 , } K (cid:91) k =0 h( LBP type riu P,R ( u k )) . (6)To reduce the dimensionality of the final descriptors, we also apply Karhunen-Lo`eve transform [21] before their use as input to the classifier algorithm. Thediagram depicted in Figure 1 illustrates the main steps involved in the proposedalgorithm. The performance of the proposed descriptors is assessed on the classification oftwo well-established benchmark datasets of texture images, namely, KTH-TIPS-2b [13] and UIUC [17].KTH-TIPS-2b is a challenging database focused on the real material depictedin each image rather than on the texture instance as most classical databases. Inthis way, images collected under different configurations (illumination, scale andpose) should be part of the same class. The database comprises a total of 4752color textures with resolution 200 ×
200 (here they are converted to gray scales), n application of a pseudo-parabolic modeling to texture image recognition 5
Fig. 1.
Main steps of the proposed method. equally divided into 11 classes. Each class is further divided into 4 samples (eachsample corresponds to a specific configuration). We adopt the most usual (andmost challenging) protocol of using one sample for training and the remainingthree samples for testing.UIUC is a gray-scale texture dataset composed by 1000 images with resolu-tion 256 ×
256 evenly divided into 25 classes. The images are photographed underuncontrolled natural conditions and contain variation in illumination, scale, pers-pective and albedo. For the training/testing split we also follow the most usualprotocol, which consists in half of the images (20 per class) randomly selectedfor training and the remaining half for testing. This procedure is repeated 10times to allow the computation of an average accuracy.For the final step of the process, which is the machine learning classifier, weuse Linear Discriminant Analysis [11], given its easy interpretation, absence ofhyper-parameters to be tuned and known success in this type of application [28].
Figures 2 and 3 show the average confusion matrices and accuracies (percentageof images correctly classified) for the proposed descriptors in the classificationof KTH-TIPS-2b and UIUC, respectively. The average is computed over all trai-ning/testing rounds, corresponding, respectively, to 4 rounds in KTH-TIPS-2band 10 rounds in UIUC. This is an interesting and intuitive graphical represen-tation of the most complicated classes and the most confusable pairs of classes.While in UIUC there is no pair of classes deserving particular attention (themaximum confusion is of one image), KTH-TIPS-2b exhibits a much more chal-lenging scenario. The confusion among classes 3, 5, 8, and 11 is the most cri-tical scenario for the proposed classification framework. It turns out that theseclasses correspond, respectively, to the materials “corduroy”, “cotton”, “linen”,and “wool”. Despite being different materials, they inevitably share similaritiesas at the end they are all types of fabrics. Furthermore, looking at the samplefrom these classes, we can also observe that the attribute “color”, that is not
Florindo and Abreu considered here, would be a useful class discriminant in that case. In general, theperformance of our proposal in this dataset is quite promising and the confusionmatrix and raw accuracy confirm our theoretical expectations.
Accuracy: 67.4%
Target Class O u t pu t C l a ss Fig. 2.
Average confusion matrix and accuracy for KTH-TIPS-2b.
Table 1 presents the accuracy compared with other methods in the literature,including several approaches that can be considered as part of the state-of-the-art in texture recognition. First of all, the advantage over the original CLBP,whose part of the descriptors are used here as local encoder, is remarkable, be-ing more than 10% in KTH-TIPS-2b. Other advanced encoders based on SIFTare also outperformed in both datasets (by a large margin in the most challen-ging textures of KTH-TIPS-2b). SIFT descriptors are complex object descriptorsand were considered the state-of-the-art in image recognition for several years.Compared with the most recent CNN-based approaches presented in [8], the re-sults are also competitive. In UIUC, the proposed approach outperforms CNNmethods like DeCAF and FC-CNN VGGM. These correspond to complex archi-tectures with a high number of layers and large requirements of computationalresources and whose results are pretty hard to be interpreted.Generally speaking, the proposed method provided results in texture clas-sification that confirm its potential as a texture image model. Indeed that wastheoretically expected from its ability of smoothing spurious noise at the sametime that preserves relevant discontinuities on the original image. The combina-tion with a powerful yet simple local encoder like CLBP yielded interesting and n application of a pseudo-parabolic modeling to texture image recognition 7
Accuracy: 98.0%
Target Class O u t pu t C l a ss Fig. 3.
Average confusion matrix and accuracy for UIUC.
Table 1.
Accuracy of the proposed descriptors compared with other texture descriptorsin the literature. A superscript in KTHTIPS-2b means training on three samples andtesting on the remainder (no published results for the setup used here).KTH-TIPS-2b UIUCMethod Acc. (%)VZ-MR8 [24] 46.3LBP [20] 50.5VZ-Joint [25] 53.3BSIF [15] 54.3LBP-FH [3] 54.6CLBP [12] 57.3SIFT+LLC [8] 57.6ELBP [19] 58.1SIFT + KCB [7] 58.3SIFT + BoVW [7] 58.4LBP riu /VAR [20] 58.5 PCANet (NNC) [6] 59.4 RandNet (NNC) [6] 60.7 SIFT + VLAD [7] 63.1ScatNet (NNC) [4] 63.7 FV-CNN AlexNet [8] 69.7
Proposed 67.4
Method Acc. (%)RandNet (NNC) [6] 56.6PCANet (NNC) [6] 57.7BSIF [15] 73.4VZ-Joint [25] 78.4LBP riu /VAR [20] 84.4LBP [20] 88.4ScatNet (NNC) [4] 88.6MRS4 [25] 90.3SIFT + KCB [7] 91.4MFS [31] 92.7VZ-MR8 [24] 92.8DeCAF [7] 94.2FC-CNN VGGM [8] 94.5CLBP [12] 95.7SIFT+BoVW [7] 96.1SIFT+LLC [8] 96.3 Proposed 98.0
Florindo and Abreu promising performance neither requiring large amount of data for training noradvanced computational resources. In general, such great performance combinedwith the straightforwardness of the model, that allows some interpretation of thetexture representation based on local homo/heterogeneous patterns, make theproposed descriptors a candidate for practical applications in texture analysis,especially when we have small to medium datasets and excessively complicatedalgorithms should be avoided.
In this study, we investigated the performance of a nonlinear PDE model (pseudo-parabolic) as an operator for the description of texture images. The operator wasapplied for a number of iterations (time evolution) and a local encoder was col-lected from each transformed image. The use of a basic histogram to pooling thelocal encoders was sufficient to provide competitive results.The proposed descriptors were evaluated over a practical task of textureclassification on benchmark datasets and the accuracy was compared with otherapproaches from the state-of-the-art. Our method outperformed several otherlocal descriptors that follow similar paradigm and even some learning-basedalgorithms employing complex versions of convolutional neural networks.The obtained results confirmed our expectations of a robust texture descrip-tor, explained by its ability of nonlinearly smoothing out spurious noise andunnecessary details, but preserving relevant information, especially those con-veyed by sharp discontinuities. In general, the results and the confirmation ofthe theoretical formulation suggest the suitability of applying such model inpractice, in tasks of texture recognition that require simple models, easy to beinterpreted and that do not require much data for training. This is a commonsituation in areas like medicine and several others.
Acknowledgements
J. B. Florindo gratefully acknowledges the financial support of S˜ao Paulo Re-search Foundation (FAPESP) (Grant n application of a pseudo-parabolic modeling to texture image recognition 9
References
1. Abreu, E., Vieira, J.: Computing numerical solutions of the pseudo-parabolicbuckley–leverett equation with dynamic capillary pressure. Math. Comput. Simul. , 29–48 (2017)2. Abreu, E., Ferraz, P., Vieira, J.: Numerical resolution of a pseudo-parabolicbuckley-leverett model with gravity and dynamic capillary pressure in het-erogeneous porous media. Journal of Computational Physics , 109395(2020). https://doi.org/https://doi.org/10.1016/j.jcp.2020.109395,
3. Ahonen, T., Matas, J., He, C., Pietik¨ainen, M.: Rotation invariant image descrip-tion with local binary pattern histogram fourier features. In: Salberg, A.B., Hard-eberg, J.Y., Jenssen, R. (eds.) Image Analysis. pp. 61–70. Springer Berlin Heidel-berg, Berlin, Heidelberg (2009)4. Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Transactionson Pattern Analysis and Machine Intelligence (8), 1872–1886 (2013)5. Catt´e, F., Lions, P.L., Morel, J.M., Coll, T.: Image selective smoothing and edgedetection by nonlinear diffusion. SIAM Journal on Numerical Analysis (1), 182–193 (1992)6. Chan, T., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: Pcanet: A simple deep learningbaseline for image classification? IEEE Transactions on Image Processing (12),5017–5032 (2015)7. Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing texturesin the wild. In: Proceedings of the 2014 IEEE Conference on Computer Vision andPattern Recognition. pp. 3606–3613. CVPR ’14, IEEE Computer Society, Wash-ington, DC, USA (2014)8. Cimpoi, M., Maji, S., Kokkinos, I., Vedaldi, A.: Deep filter banks for texture recog-nition, description, and segmentation. International Journal of Computer Vision (1), 65–94 (2016)9. Cuesta, C., Pop, I.: Numerical schemes for a pseudo-parabolic burgers equation:discontinuous data and long-time behaviour. J. Comput. Appl. Math. , 269–283(2009)10. Dhivyaa, C.R., Sangeetha, K., Balamurugan, M., Amaran, S., Vetriselvi, T., John-paul, P.: Skin lesion classification using decision trees and random forest algorithms.Journal of Ambient Intelligence and Humanized Computing (2020)11. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals ofEugenics (2), 179–188 (1936)12. Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary patternoperator for texture classification. Trans. Img. Proc. (6), 1657–1663 (2010)13. Hayman, E., Caputo, B., Fritz, M., Eklundh, J.O.: On the significance of real-worldconditions for material classification. In: Pajdla, T., Matas, J. (eds.) ComputerVision - ECCV 2004. pp. 253–266. Springer Berlin Heidelberg, Berlin, Heidelberg(2004)14. Jain, D.K., Zhang, Z., Huang, K.: Multi angle optimal pattern-based deep learningfor automatic facial expression recognition. Pattern Recognition Letters , 157–165 (2020)15. Kannala, J., Rahtu, E.: Bsif: Binarized statistical image features. In: ICPR. pp.1363–1366. IEEE Computer Society (2012)16. Koenderink, J.J.: The structure of images. Biological Cybernetics (5), 363–370(1984)0 Florindo and Abreu17. Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using localaffine regions. IEEE Transactions on Pattern Analysis and Machine Intelligence (8), 1265–1278 (2005)18. Lin, J., Pappas, T.N.: Structural texture similarity for material recognition. In:2019 IEEE International Conference on Image Processing (ICIP). pp. 4424–4428.IEEE International Conference on Image Processing ICIP, Inst Elect & Elect Engi-neers; Inst Elect & Elect Engineers Signal Proc Soc (2019), 26th IEEE InternationalConference on Image Processing (ICIP), Taipei, TAIWAN, SEP 22-25, 201919. Liu, L., Zhao, L., Long, Y., Kuang, G., Fieguth, P.: Extended local binary patternsfor texture classification. Image Vision Comput. (2), 86–99 (2012)20. Ojala, T., Pietik¨ainen, M., M¨aenp¨a¨a, T.: Multiresolution gray-scale and rotationinvariant texture classification with local binary patterns. IEEE Transactions onPattern Analysis and Machine Intelligence (7), 971–987 (2002)21. Pearson, F.K.: Liii. on lines and planes of closest fit to systems of points in space.The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Sci-ence (11), 559–572 (1901)22. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion.IEEE Trans. Pattern Anal. Mach. Intell. (7), 629–639 (1990)23. Robert Singh, K., Chaudhury, S.: Comparative analysis of texture feature extrac-tion techniques for rice grain classification. IET Image Processing (11), 2532–2540 (2020)24. Varma, M., Zisserman, A.: A statistical approach to texture classification fromsingle images. International Journal of Computer Vision (1), 61–81 (2005)25. Varma, M., Zisserman, A.: A statistical approach to material classification usingimage patch exemplars. IEEE Transactions on Pattern Analysis and Machine In-telligence (11), 2032–2047 (2009)26. Verma, V., Muttoo, S.K., Singh, V.B.: Multiclass malware classification via first-and second-order texture statistics. Computers & Security (2020)27. Vieira, J., Abreu, E.: Numerical modeling of the two-phase flow in porous mediawith dynamic capillary pressure. Ph.D. thesis, University of Campinas, Campinas,SP, Brazil (7 2018)28. Vieira, J., Abreu, E., Florindo, J.B.: Texture image classification based on apseudo-parabolic diffusion model (2020), Available at https://arxiv.org/abs/2011.07173
29. Weickert, J.: A review of nonlinear diffusion filtering. In: Proceedings of the FirstInternational Conference on Scale-Space Theory in Computer Vision. pp. 3–28.SCALE-SPACE ’97, Springer-Verlag, Berlin, Heidelberg (1997)30. Witkin, A.P.: Scale-space filtering. In: Proceedings of the Eighth InternationalJoint Conference on Artificial Intelligence - Volume 2. pp. 1019–1022. IJCAI’83,Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1983)31. Xu, Y., Ji, H., Ferm¨uller, C.: Viewpoint invariant texture description using fractalanalysis. International Journal of Computer Vision (1), 85–100 (2009)32. Zhao, G., Wang, X., Cheng, Y.: Hyperspectral image classification based on localbinary pattern and broad learning system. International Journal of Remote Sensing41