Towards Direct Medical Image Analysis without Segmentation
aa r X i v : . [ c s . C V ] O c t Towards Direct Medical Image Analysis without Segmentation
Xiantong Zhen a , b and Shuo Li a , b , c ∗ a The University of Western Ontario, London, ON, Canada. b The Digital Image Group (DIG), London, ON, Canada. c Lawson Health Research Institute, London, ON, Canada.
Direct methods have recently emerged as an e ff ective ande ffi cient tool in automated medical image analysis and be-come a trend to solve diverse challenging tasks in clinical prac-tise. Compared to traditional methods, direct methods are ofmuch more clinical significance by straightly targeting to thefinal clinical goal rather than relying on any intermediate steps.These intermediate steps, e.g., segmentation, registration andtracking, are actually not necessary and only limited to veryconstrained tasks far from being used in practical clinical appli-cations; moreover they are computationally expensive and time-consuming, which causes a high waste of research resources.The advantages of direct methods stem from removal of in-termediate steps, e.g., segmentation, tracking and registration; avoidance of user inputs and initialization; reformulationof conventional challenging problems, e.g., inversion problem,with e ffi cient solutions.Direct methods in medical image analysis scenarios are de-fined as a series of methodologies that estimate clinical mea-surements directly from medical imaging data without relyingon any unnecessary, intermediate steps. The principle behinddirect methods is that there are intrinsic relationship existingbetween medical images and clinical measurements; these rela-tionship can be directly extracted and modeled to estimate clin-ical measurements for diagnosis and prognosis without relyingon any intermediate stages.Direct methods have recently demonstrated its great e ff ec-tiveness and e ffi ciency in many aspects for direct medical im-age analysis, especially on two important applications: volumeestimation and functional analysis. Direct estimation of cardiac volumes without segmentationhas shown remarkable e ff ectiveness due to its great advantagesover conventional segmentation based methods. For more than20 years, cardiac volume estimation has long been su ff eringfrom the intermediate, unreliable and even intractable segmen-tation steps in traditional methods. Segmentation has only beenfocused on a single ventricle, e.g., the left ventricle (LV) ona bi-ventricular view for a long time, and recently started towork on the right ventricle (RV), which remains unsolved, notto mention joint bi-ventricles, i.e., LV and RV, and even morechallenging four chambers, i.e., LV, RV, LA and RA. All thefour ventricular volumes, however, are routinely and intensivelyused in clinical practise for cardiac functional analysis. ∗ Corresponding author ([email protected]).
Direct volume estimation removes segmentation and esti-mate the volume directly from images by capturing and disen-tangling the hihgly non-linear relationship between image ap-pearance and volumes. Direct estimation provides a generalframework that can flexibly deal with a single ventricle [1],joint bi-ventricles [2, 3, 4] and simultaneous four chambers [5]in a more convenient way within a unified single framework. The initial success of direct volume estimation has beendemonstrated by LV volume estimation in [1]. Image featuresby statistics based on the Bhattacharyya coe ffi cient of similaritybetween image distributions; it is demonstrated that these statis-tics are non-linearly related to the LV volumes and can be usedfor ejection fraction estimation via neural networks directly. Futher success of direct volume estimation has beenachieved for joint bi-ventricular volume estimation [2, 3, 4].The first attempt [2] adopts a Bayesian model in which the LVand RV volumes are calculated as the weighted average overthe templates. Random forests are introduced in [3] for moree ffi cient joint bi-ventricular volume estimation without relyingon assumed models. Recently, a fully data-driven, learning-based framework for bi-ventricular volume estimation has beendeveloped in [4]. Direct volume estimation has been generalized and vali-dated on multiple ventricles for cardiac four chambers in [5].Based on supervised descriptor learning, multi-output regres-sion forests are adopted to directly and simultaneously estimatefour chamber volumes. This achieves for the first time achievesgeneral framework for direct volume estimation of either a sin-gle ventricle or multiple ventricles.
Direct diagnosis has started to generate increasing interest[2, 3] in cardiac image analysis which has long been severelyobstructed by segmentation, registration and tracking in con-ventional approaches. Direct diagnosis of cardiac functional abnormalities hasfurther demonstrated the power of direct methods [6, 7]. Tra-ditional methods are highly dependent on segmentation, regis-tration and tracking [8], which are conditionally expensive andinevitably induces cumulative errors. A direct method for re-gional assessment of the cardiac LV myocardial function viaclassification has been proposed in [6]. The method formulatesdiagnosis as a classification problem based on statistical fea-tures which are related to the proportion of blood within eachsegment and can characterize segmental contraction. Without
Preprint submitted to Elsevier October 22, 2015 racking or segmentation of regional boundaries, they obtain andirect assessment of cardiac segmental abnormality in real-timewith more accurate results than segmentation-based methods. Direct diagnosis has also been developed to resolve tradi-tional challenging problems, e.g., inverse problems, with moree ffi cient solutions. Diagnosis and treatment of dilated car-diomyopathy (DCM) is challenging due to the large variety ofdisease stages. Cardiac models of cardiac electrophysiology(EP) is commonly used to conduct the challenging personal-ization of model parameters. Traditional methods require tosolve an inverse problem which is challenging and computa-tionally demanding. Direct diagnosis and treatment of DCMhas been explored in [7] which achieves data-driven estimationof cardiac electrical di ff usivity from 12-lead ECG. The directmethod learns the inverse function by formulating as a polyno-mial regression problem. Compared to traditional approaches,the direct method can find model parameters directly from EEGsignals for specific patients and provides more accurate resultsin a more e ffi cient way.In addition to the great success of direct methods in volumeestimation and diagnosis, direct methods have also been de-veloped in many other important medical image applications,e.g., anatomy detection / localization [9], shape analysis [10] andclinical pattern prediction [11]. The great e ff ectiveness and up-rising role of direct methods in medical image analysis and clin-ical practise stem from multiple attractive merits in contrast totraditional indirect methods: • Direct methods enable to focus on the final clinical goalwithout relying on any unnecessary, unreliable, intermedi-ate steps. By directly modeling the relationship betweenimage appearance and medical measurements of clinicalinterest, direct methods estimate clinical measurements di-rectly from medical images in a more convenient, e ffi cientand accurate way. Therefore, direct methods can not onlysave high computational cost but also avoid errors inducedby any intermediate operations, which enables practicaluse in clinical applications. • Direct methods provide more e ffi cient resolutions of con-ventional problems, e.g., inversion problems, by statisticallearning. In contrary to their conventional counterparts,statistical models have more compact and exquisite for-mations and are highly generalized, and therefore possessextensive flexibility and generality for di ff erent clinical ap-plications. Moreover, due to the statistical learning, di-rect methods also enable us to richly explore data statisticswhich therefore o ff ers more meaningful and comprehen-sive clinical assessment. • Direct methods serve as a bridge between rapidly-updatingmachine learning algorithms and medical image analysis,which accelerates the translation of fundamental researchto clinical practise. Direct methods allow to leverage ad-vanced machine learning techniques from artificial intelli-gence for medical image analysis. By deploying machinelearning algorithms, direct methods are able to not only ex-plore distinguishing image features that are closely relatedto diseases but also take advantages of papulation-basedanalysis that helps accurate diagnosis. The established success of direct methods on diverse appli-cations has validate its e ff ectiveness in medical image anal-ysis and shown the great potential in clinical practice. Dueto the attractive merits of e ffi ciency, e ff ectiveness and conve-nience, direct methods provide innovative solutions for tradi-tional medical image analysis and will promise its great per-spective in even broader clinical applications, e.g., volume es-timation, functional analysis, clinical prediction and diagnosis.With direct methods, traditional time-consuming tasks will bereplaced with more e ffi cient methods; those complex tasks willbe simplified and handled in a more convenient way; and thoseintractable tasks will be resolved quickly with more e ff ectivedirect formulations.Direct methods will continue to show its legendary on solv-ing even more challenging clinical tasks with the rapid develop-ment and advancement of machine leaning techniques in artifi-cial intelligence, and proliferation of large amount of medicalimaging data. In the foresee future, we can expect the mar-riage of direct methods with advanced state-of-the-art machineleaning techniques, e.g., deep learning and statistical discrim-inative learning, which will further boost the development ofdirect medical image analysis and promote its wide use in clin-ical practice. References [1] M. Afshin, I. B. Ayed, A. Islam, A. Goela, T. M. Peters, S. Li, Globalassessment of cardiac function using image statistics in MRI, in: MedicalImage Computing and Computer-Assisted Intervention–MICCAI 2012,2012, pp. 535–543.[2] Z. Wang, M. Ben Salah, B. Gu, A. Islam, A. Goela, S. Li, Direct estima-tion of cardiac bi-ventricular volumes with an adapted bayesian formula-tion, IEEE Transactions on Biomedical Engineering (2014) 1251–1260.[3] X. Zhen, Z. Wang, A. Islam, I. Chan, S. Li, Direct estimation of cardiacbi-ventricular volumes with regression forests, in: Medical Image Com-puting and Computer-Assisted Intervention–MICCAI 2014, 2014.[4] X. Zhen, Z. Wang, A. Islam, M. Bhaduri, I. Chan, S. Li, Multi-scale deepnetworks and regression forests for direct bi-ventricular volume estima-tion, Medical Image Analysis, 2015.[5] X. Zhen, A. Islam, M. Bhaduri, I. Chan, S. Li, Direct and simultaneousfour-chamber volume estimatino by multi-output regression, in: MedicalImage Computing and Computer-Assisted Intervention–MICCAI 2015,Springer, 2015.[6] M. Afshin, I. Ben Ayed, K. Punithakumar, M. Law, A. Islam, A. Goela,T. Peters, S. Li, Regional assessment of cardiac left ventricular myocar-dial function via mri statistical features, IEEE Transactions on MedicalImaging.[7] O. Zettinig, T. Mansi, D. Neumann, B. Georgescu, S. Rapaka, P. Seegerer,E. Kayvanpour, F. Sedaghat-Hamedani, A. Amr, J. Haas, et al., Data-driven estimation of cardiac electrical di ff usivity from 12-lead ecg sig-nals, Medical image analysis.[8] K. Punithakumar, I. Ben Ayed, A. Islam, A. Goela, I. G. Ross, J. Chong,S. Li, Regional heart motion abnormality detection: An information the-oretic approach, Medical image analysis 17 (3) (2013) 311–324.[9] A. Criminisi, D. Robertson, E. Konukoglu, J. Shotton, S. Pathak,S. White, K. Siddiqui, Regression forests for e ffi cient anatomy detectionand localization in computed tomography scans, Medical image analysis17 (8) (2013) 1293–1303.[10] S. K. Zhou, Shape regression machine and e ffi cient segmentation of leftventricle endocardium from 2d b-mode echocardiogram, Medical ImageAnalysis 14 (4) (2010) 563–581.[11] A. I. Namburete, R. V. Stebbing, B. Kemp, M. Yaqub, A. T. Papa-georghiou, J. A. Noble, Learning-based prediction of gestational age fromultrasound images of the fetal brain, Medical image analysis 21 (1) (2015)72–86.cient segmentation of leftventricle endocardium from 2d b-mode echocardiogram, Medical ImageAnalysis 14 (4) (2010) 563–581.[11] A. I. Namburete, R. V. Stebbing, B. Kemp, M. Yaqub, A. T. Papa-georghiou, J. A. Noble, Learning-based prediction of gestational age fromultrasound images of the fetal brain, Medical image analysis 21 (1) (2015)72–86.