Auto-Detection of Tibial Plateau Angle in Canine Radiographs Using a Deep Learning Approach
Masuda Akter Tonima, F M Anim Hossain, Austin DeHart, Youmin Zhang
AAuto-Detection of Tibial Plateau Angle in CanineRadiographs Using a Deep Learning Approach
Masuda Akter Tonima , F M Anim Hossain , Austin DeHart and Youmin Zhang Gina Cody School of Engineering, Concordia University, Montreal, Quebec, Canada Innotech Medical Industries Corp., Vancouver, British Columbia, Canada * [email protected] Abstract —Stifle joint issues are a major cause of lamenessin dogs and it can be a significant marker for various formsof diseases or injuries. A known Tibial Plateau Angle (TPA)helps in the reduction of the diagnosis time of the cause. Withthe state of the art object detection algorithm YOLO, and itsvariants, this paper delves into identifying joints, their centroidsand other regions of interest to draw multiple line axes andfinally calculating the TPA. The methods investigated predictssuccessfully the TPA within the normal range for 80 percent ofthe images.
Index Terms —TPA, YOLO, object detection, canine, X-ray,knee X-ray, knee angle, deep learning
I. I
NTRODUCTION
Canine stifle joint issues are one of the oldest and most facedissues in the veterinary orthopedics sector [1]. The value ofTibial Plateau Angle (TPA) can be used to identify issues thatexist in a canine’s leg, how it may react to different forms ofinjuries or even if there is presence of pre-existing conditions[2]. Injuries such as the cranial cruciate ligament (CCL)rupture (CCLR) are very common, major and progressivelydeteriorate stifle joints of canines permanently; automatedTibial Plateau Angle (TPA) assessment can help in shorteningthe process of CCL detection during surgical procedures [3],additionally, as the TPA value plays an important role in thedistribution of force when a dog walks, it can also determinewhether or not there is excessive cranial tibial thrust that maypredispose canines to CCLR [1], [4].The utility of automation has become ubiquitous in themodern world where everything is electronically powered andwe, humans rely more and more on artificial intelligence forassistance. Automation in image annotation has become a ma-jor tool in the medical field, driving patient care decisions [5].The latest method of obtaining such decisions in an automatedmanner involves using a processor for digital image repre-sentation acquisition that simultaneously generates annotationsand determines the associations between multiple annotationsof objects of same class or group [5]. This same processorsimultaneously works on determining and representing the saidclasses while recording the annotations in its limited memory[5]. There has been variety of work regarding determinationTPA considering multiple factors such as age, sex, breed etc. asseen in studies [4], [3], [2], [6], however none thus far for theautomation of the process. This study delves into this aspect of application and completes the first stage of automation inTPA determination.II. TPA C
ALCULATION
TPA calculation has some prerequisites that must be fulfilled,these are:1) The Stifle and Tarsus must be 90°flexed;2) Functional Tibial line must be formed by connectinga) Centre of Talusb) The centre of intercondylar eminences shown inFig. 1;3) The Medial Tibial Plateau line should be drawn usingthe first two.Following the identification of points of interests, anddrawing of the lines of interest, i.e. the Functional Tibial Line(FTL) and the Medial Tibial Plateau Line (MTPL), anotherline is drawn such that the relation of new line and the FTLis 90°. Tibial plateau angle is the angle between this new lineand the MTPL; this is shown in Fig. 3. Normal TPA valuescan range from 18 to 25 degrees [1]; the large range can beattributed to the large range of breed, body weight, age etc.
A. Training Dataset
The dataset used for training was collected from variousveterinary clinics, as none is available in the public domain,and the objects of interests were manually annotated. Theresolution of these images vary largely as they are collected (a) (b) (c)
Fig. 1:
TPA requirements (a) Stifle (knee joint) and Tarsus (anklejoint) flex, (b) Green line representing the FTL and (c) MTPL pointedwith green arrow [7]. a r X i v : . [ c s . C V ] F e b a) (b) (c) Fig. 2:
Functional Tibial line drawing needs; (a) IntercondylarEminence point, (b) Intercondylar Eminence point reference and (c)Centre of Talus [7]. from various sources, thus for the sake of uniformity theimages are all scaled to fit the same dimensions. The firstpart of this project was to develop a lightweight radiographimage sorting algorithm reported in [8]. The images chosen forthe task described in this manuscript were all classified to belateral lower body images by that sorting algorithm. Examplesof manual annotations of objects of interests are given in Fig.4: here regions A, B and C identify the joints while the point’e’ identifies the centre of Talus and regions d1 and d2 identifythe points that form the MTPL.The images from the source dataset had issues and neededto be consolidated into a more usable framework. Theseissues were mostly due to inconsistencies in practices ofradiographers and movement by the animals during radio-graphy that resulted in radiographs with incomplete data,inconsistent image quality, such as varying contrast, brightnessand positioning of point of interest or images that failed tomeet the prerequisite conditions for this task. Examples ofdifficult data are shown in Fig. 5.Most of the source dataset of over a thousand uniqueknee radiographs were distorted, had poor patient positioning,or were otherwise unfit for annotating. 250 of the originalimages were of sufficient quality to contribute to this effort.These images were set to have 6 different classes for trainingpurposes, as mentioned and shown in Fig. 4. These were thentrained using YOLOv3 [10] and the results of the predictedannotations are shown in Table I. From these predictions theFig. 3:
Tibial Plateau Angle shown in relation with the new perpen-dicular line drawn and the MTPL [7].
Fig. 4:
Example of manual annotations, highlighting the regions ofinterests [9].
Fig. 5:
Some examples of difficult data centroids were extracted, which is then used to plot the FTLand MTPL. Then using the method mentioned in Section II,TPA is calculated. Examples of images, the region of interestsdetection and their respective TPA determination is shown inFig. 6 and Table I:
B. Activation Functions
As activation functions are explored in this paper for thepurpose of performance comparison, a small section has beendedicated for understanding of the roles and types of activationfunction that have been used. Activation function is simply thepathway that allows feeding of the input and output to andfrom the current neuron. This can range in variety of forms, a) (b)(c) (d)(e) (f)
Fig. 6:
Example of algorithm detecting, highlighting the regions ofinterests. from as simple as an on/off switch—i.e, Step function to ascomplex as the Sigmoid. In this section only the functionsused for results obtained are discussed:1) Linear activation function is exactly what the namesuggests. It takes an input, multiples it with the learnedweight and produces an output that is a function of theinput. f ( x ) = mx (1)2) ReLU is a rectified version of the linear function that TABLE I: TPA shown by trained system with calculations from thedetected annotations
Figure Number TPA
TABLE II:
Comparison of results with variation in algorithmversions
Image YOLOv3 YOLOv4-1 YOLOv4-2 YOLOv4-3 does not allow the negative inputs. f ( x ) = max (cid:40) x for x > otherwise (2)The graphical representation of this is shown in Fig.7(a).3) Leaky ReLU is a variation of ReLU that has a smallslope on the negative area and thus is more complacenttoward negative inputs. f ( x ) = max (cid:40) x for x > ax x < = 0 (3)The graphical representation of this is shown in Fig.7(b).4) Swish is a gated sigmoid function that has an interestingmathematical form. f ( x ) = x × σ ( β ( x )) (4)where the β can be a constant or a trainable quantity,depending on which the Swish may act either like ascaled linear (for β = 0 ) or a ReLU (for β → ∞ )function. The graphical representation of this is shownin Fig. 7(c).5) Mish is an improvement on the existing swish function;it is a smooth and non-monotonic function that can bedefined as: f ( x ) = x × tanh(ln(1 + e x ) (5)The graphical representation of this is shown in Fig.7(d). Since Mish and its predecessor are visibly indis-tinguishable, a comparison of the two is shown in Fig. 8that displays the negligible difference between the two.III. T ESTING AND A NALYSIS
For comparison purposes the radiographs have been testedwith YOLOv3, original YOLOv4 [12] and custom modifica-tions of the YOLOv4 by changing the activation functions a) ReLu (b) Leaky Relu(c) Swish (d) Mish
Fig. 7:
Graphical form of the activation functions [11].
Fig. 8:
Mish and Swish comparison for understanding how similarthey are. Blue line represent Swish and red represents Mish [11]. and the results are shown in Table II. The original versionof YOLOv4 (addressed as YOLOv4-1 in Table II) combinedthe Mish, Linear and Leaky activation functions, the versionsYOLOv4-2 and YOLOv4-3 used for the purpose of testing inthis paper have combinations of Mish, Linear, Swish and Mish,Linear and Relu activation functions respectively. Similarly,comparison of the results that did not fall under the presumednormal range shown in Fig. 9, for the algorithm, are shownwith variations in activation function in Table III. It can be seenhere that these images, are giving TPA predictions similar tothe original YOLOv3, i.e. outside of presumed range. TABLE III:
Comparison of ‘below range’ results with variation inalgorithm versions
Image YOLOv3 YOLOv4-1 YOLOv4-2 YOLOv4-3 (a) TPA=10.4° (b) TPA=10.3° (c) TPA=6.53°
Fig. 9:
Example of algorithm detecting, highlighting the regions ofinterests that are below known range of value.
IV. C
ONCLUSION AND F UTURE W ORK
Automated image annotation has become a major requirementin the medical field, since it can be a great tool to drive quick,intelligent and reliable patient care decisions [5]. Grady andSchaap [5], patented the idea of incorporating user input aspart of the learning process, as it is essential that the automatedresults are validated and appropriately corrected by a user, i.e.,a radiographer, when required. Results that do not fall underthe range mentioned by [1], are shown in Fig. 9 and Table III.These open a possible scope for improvement and future workas these samples might require human expert-intervention inorder to correct the annotations and relearn from the saidcorrections. For future work, a user based feedback loopwill be added to this system that may be used alongside thesepredictions, as input to train the system further which willresult in more accurate TPA calculation. Another degree tothis work could be automating this angle value calculation,i.e., using the TPA values as part of the information fed to thesystem so that it may be able to draw the lines along the bonejoint-axes, and calculate the TPA, all within the same deepneural network; simplifying the user interface to just insertingthe X-ray and, potentially resulting in a more accurate TPAcalculation.The core finding of this paper is that, even without expertintervention, automation of annotations was successfully per-formed to a significantly accurate degree, 4 out of every 5images tested in average. With this in mind, in addition tothe previously developed lightweight classifier [8], it can beconcluded that the second step of our development, of theautomation in diagnostic tool, is complete. This paper confirmsthat it is possible to automate the system via annotation whichan be improved with the formerly mentioned expert feedbackin the future. A
CKNOWLEDGMENT
The dataset used in this paper is provided by Innotech MedicalIndustries Xray. The research work is financially supported inpart by a MITACS Accelerate Project (no. FR56849) under thepartner organization Innotech Medical Industries Corp. and theNatural Sciences and Engineering Research Council of Canada(NSERC). R
EFERENCES[1] Beom Seok Seo, In Seong Jeong, Zhenglin Piao, Minju Kim, SehoonKim, Md Mahbubur Rahman, and Nam Soo Kim. Measurement of thetibial plateau angle of normal small-breed dogs and the application ofthe tibial plateau angle in cranial cruciate ligament rupture.
Journal ofAdvanced Veterinary and Animal Research , 7(2):220, 2020.[2] Sarah Sörensson. Evaluation of tibial plateau angle and otherfactors in cases of canines stifle joint diseases. Available athttps://hdl.handle.net/20.500.12512/109058, 2021.[3] Philip D Pacchiana, Ethan Morris, Sarah L Gillings, Carl R Jessen, andAlan J Lipowitz. Surgical and postoperative complications associatedwith tibial plateau leveling osteotomy in dogs with cranial cruciateligament rupture: 397 cases (1998–2001).
Journal of the AmericanVeterinary Medical Association , 222(2):184–193, 2003.[4] Choong Sup Kim, Su Young Heo, Jae Won Seol, Min Su Kim,Sang Hoon Lee, Nam Soo Kim, and Hae Beom Lee. Measurement of thetibial plateau angle in normal small breed dogs.
Journal of VeterinaryClinics , 32(3):231–234, 2015.[5] Leo Grady and Michiel Schaap. System and method for controllinguser repeatability and reproducibility of automated image annotationcorrection, April 30 2020. US Patent App. 16/725,649.[6] KW Moore and RA Read. Cranial cruciate ligament rupture in thedog— A retrospective study comparing surgical techniques.
AustralianVeterinary Journal , 2021. Sub-mitted.[9] Eric Liu. labelimg.py. Available athttps://github.com/eric612/AutoLabelImg/blob/master/labelImg.py,2018. this is a python code used for manual annotation.[10] Joseph Redmon and Ali Farhadi. YOLOv3: An incremental improve-ment. arXiv preprint arXiv:1804.02767 , 2018.[11] Sefik Serengil. Mish as neural networks activation func-tion. Available at https://sefiks.com/2019/10/28/mish-as-neural-networks-activation-function/, Feb. 2020.[12] Alexey Bochkovskiy, Chien Yao Wang, and Hong Yuan Mark Liao.YOLOv4: Optimal speed and accuracy of object detection. arXivpreprint arXiv:2004.10934arXivpreprint arXiv:2004.10934