DICODerma: A practical approach for metadata management of images in dermatology
**correspondence: [email protected]
DICOD erma : A practical approach for metadata management of imagesin dermatology P reprint , compiled F ebruary
18, 2021
Bell Raj Eapen , Feroze Kaliyadan , and Ashique Karalikkattil T McMaster University, Hamilton, Ontario, L8S 4L8, Canada Faculty of Dermatology, King Faisal University, Saudi Arabia Amanza Skin Clinic, Perinthalmanna, Kerala, India A bstract Clinical images are vital for diagnosing and monitoring skin diseases, and their importance has increased withthe growing popularity of machine learning. Lack of standards has stifled innovation in dermatological imaging,unlike other image-intensive specialties such as radiology. We investigate the meta-requirements for utilizingthe popular DICOM standard for metadata management of images in dermatology. We propose practical designsolutions and provide open-source tools to integrate dermatologists’ workflow with enterprise imaging systems.Using the tool, dermatologists can tag, search, organize and convert clinical images to the DICOM format. Webelieve that our less disruptive approach will improve the adoption of standards in the specialty. ntroduction
Dermatology being a visual specialty, dermatologists rely onimages for documenting and evaluating patient outcomes. How-ever, unlike radiology that relies widely on accepted standardsfor imaging, dermatologists lack standardized methods for ac-quisition, transfer and archival of clinical images [1]. The lackof standardisation has been a major drawback when it comes tolarge-scale imaging and documentation in dermatology. Withmachine learning (ML) gaining momentum and popularity in therecent times, the need for standardised digital imaging has alsoincreased. Many of these emerging ML methods need e ffi cientand e ff ective management of images for training, testing andvalidating models.The lack of a well-established standard has an impact on patientprivacy as well [2]. Dermatologists do not have standards-basedsolutions such as the Picture Archival and Retrieval System(PACS) to rely on for sharing images among them and peers.Hence, they are often compelled to resort to less secure methodssuch as email and social media platforms. Most dermatologistsrely on their own personal methods for image archival. Hence,they find it di ffi cult to compile or retrieve images belonging toa specific category (example: images of mucosal lesions) fordiscussions, presentations or any academic activity, a task whichis very easily done by their radiology colleagues.Digital Imaging and Communications in Medicine (DICOM)is a widely accepted and comprehensive standard for imageacquisition, transmission and storage in radiology and relatedspecialties. Most devices for image acquisition and displaysupport the DICOM standard. Much work has been done toport the DICOM standard to dermatology, but the e ff orts so farhave been largely unsuccessful [3]. The consistent display of animage is less critical in dermatology for diagnosis and the imag-ing needs are (or traditionally were) less intensive compared toradiology. This led to the resistance in adopting DICOM -- acomprehensive and complex standard for image management.Unlike standard consumer image file formats such as JPEG andBMP, DICOM supports the storage of clinical metadata such as the patient demographics along with the image. Traditionallydermatologists rely on auxiliary systems such as the electronicmedical records (EMRs) for the clinical metadata.DICODerma is a tool and a preliminary standard to reconcile thebest of both worlds — the simplicity of consumer image toolsand the DICOM and PACS based enterprise imaging infrastruc-ture. DICODerma can encode some of the relevant DICOM tagsin the EXIF (Exchangeable Image File Format) header space ofordinary digital images. Using DICODerma we built a plugin forthe popular open-source image viewer for healthcare — ImageJ— to manage these metadata in digital images in dermatology.ImageJ has been used previously in dermatological applicationssuch as constructing three-dimensional images from optical co-herence tomography [4] and quantifying allergic and irritantpatch test reactions [5]. Using our plugin called DIT4IJ, meta-data can be added to any digital image, search images basedon the metadata and convert ordinary digital images to the DI-COM format. DIT4IJ stands for Dermatology Image Tagger forImageJ.DIR4IJ allows dermatologists to use the existing tools that theyare familiar with, and at the same time leverage some of the ad-vantages of an enterprise imaging infrastructure such as greaterpatient privacy, patient safety, and better compliance with leg-islative requirements for image retention.The rest of the article is structured as follows. First, we brieflydescribe the DICOM specifications and the associated terminolo-gies and how they pertain to dermatology. Then we systemati-cally explore the meta-requirements for extending the DICOMstandard to dermatology based on our personal experience. Next,we describe our meta-design — a java library for storing andretrieving patient metadata as EXIF tags called DICODerma.Then we describe how we used DICODerma to build an ImageJplugin for dermatologists (DIT4IJ) to tag and organize imagesand to convert them to the DICOM format. Finally, we discusssome of the advantages and limitations of our approach. a r X i v : . [ c s . I R ] F e b reprint – DICOD erma : A practical approach for metadata management of images in dermatology he DICOM S tandard
DICOM is one of the most widely used standards in healthcaredefining formats for images and structured data, workflow man-agement and network protocols [6]. The National ElectricalManufacturers Association (NEMA) foresees the administrationof the standard but has no license requirement for use. Someof the common terms associated with DICOM are the serviceobject pair (SOP) and the image object definition (IOD). ThoughIODs are generic classes, most IODs represent individual real-world entities such as X-rays and MRI along with the associatedmetadata. The combination of an IOD with a service such asstorage, print or query, is the SOP.The various metadata associated with the images includes pa-tient demographics, series (a group of closely related images),study (all series associated with one procedure) and the acquiredbinary image data. The metadata has a numerical key calledthe tag, data type called the value representation (VR) and thevalue multiplicity (VM) count. The metadata is organized intological groups such as the patient module. The list of thesespecifications that a product supports is called the conformancestatement. In short, DICOM specifies storage for storing, pro-cessing, transmitting and displaying imaging data. The DICOMheader is seen in Figure 1. maging standards in D ermatology Imaging standards have a crucial role in the clinical imagemanagement in dermatology owing to its highly visual nature.Dermatologists use di ff erent types of images ranging from der-moscopy to total-body maps. Sophisticated methods such asreflectance confocal microscopy are also becoming increasinglypopular. In this article, we give emphasis to the common digitalphotographs, but some of the discussions may apply to othermodalities as well.Image metadata is important in dermatology as in other do-mains. The useful metadata includes demographic details, clini-cal findings, device settings and image characteristics. Accuraterendering of images and acquisition context is important in der-matology as well [3]. Dermatology has a distinct ontology thatis used for an accurate textual description of lesions. The meta-data standards should support the domain-specific ontology ofdermatology and support the emerging modalities.Though dermatology is highly visual, dermatologists do not com-pletely rely on the captured images for diagnostic, prognosticand therapeutic decision making, and as such accuracy of colourand resolution is not very crucial. Images are mainly used fordocumentation, but with the increasing popularity of telederma-tology, parameters like resolution and color accuracy have alsobecome more important. Dermatologists, especially those work-ing in the community and those in limited resource settings, relyon consumer devices such as digital cameras and smartphonesfor image capture and documentation. Image capture mostlyhappens during a face-to-face consultation and routine physicalexaminations. Hence, though the DICOM standard can be usedas it is in dermatology, its overall adoption by vendors as wellas practitioners has not been very encouraging as of now. Thelack of adoption is mostly due to the large overhead required forthe implementation and adoption of a comprehensive standardsuch as DICOM. The workgroup 19 (WG19) of the DICOM consortium has ex-plored ways in which DICOM can be extended to dermatologicalapplications though the group did not propose a complete finalstandard [3]. The existing IODs such as the Visible Light (VL)and the Standard Capture (SC) can be used for dermatologicalapplications with little modifications. Device and acquisition-related metadata are captured by consumer-devices and encodedin the EXIF header supported by many digital image storageformats. There is some overlap between EXIF and DICOMheader tags. The growing popularity of machine learning (ML) and artifi-cial intelligence (AI) applications in dermatology has broughtnew requirements for image management [1]. The need forstandardized images, labelled with appropriate metadata, is anenabler for AI applications. The digital revolution encouragessharing of images with peers and experts from other disciplinesfor opinion and as such being part of the wider institutionalimage management infrastructure such as the picture archiv-ing and communication system (PACS). Adoption of ElectronicHealth Record (EHR) systems made it necessary to have a com-plete digital longitudinal patient record that includes clinicalimages captured during a dermatology encounter. The need foradopting enterprise-imaging standards is becoming increasinglyimportant in dermatology. ur A pproach Guided by the design science research methodology [7], wesystematically investigated the solution space for the problemof standardizing the digital image workflow for dermatology.Our aim was to find generalizable design knowledge that canguide system designers and policymakers. Though specificrequirements vary among di ff erent user groups of an informationsystem, they follow generic laws called meta-requirements [8].We identified some of the meta-requirements as below:1. The existing DICOM standard should be leveraged asmuch as possible so that existing solutions such asPACS can be directly used in dermatology.2. The users should be able to enter the DICOM ecosys-tem without adopting the entire standard, ideally usingsimple tools that are already in use.3. The solution should be usable even with no vendoradoption, but vendors who adopt the standard shouldhave an incentive to do so.4. The solution should support improved patient privacy.5. Search Engine Optimisation [SEO]: Search enginesand social media platforms have an increasingly im-portant role in knowledge dissemination in a privacy-preserving manner. Potential solutions should addressthe needs of these platforms [9].6. The standard should support emerging techniques suchas machine learning and artificial intelligence.7. The meta-design should be su ffi ciently abstract so thatit can be easily implemented by vendors and users tosupport new needs. reprint – DICOD erma : A practical approach for metadata management of images in dermatology
38. The standard should be simple and easy to adopt andadapt to, leveraging existing tools. esign
As potential users of DICODerma, we adopt a meta-designapproach to translate the generalizable meta-requirements asdescribed above into a prototype that can be extended. Wecreated two software artifacts (meta-design) in the solution spacethat aligns with the above meta-requirements. One is a javalibrary called DICODerma, to encode some of the importantDICOM tags as EXIF tags. The other is a plugin called DIT4IJfor the popular open-source biomedical image managementsoftware — ImageJ. Both are open-source available from theGitHub repository [10]. Before we describe our meta-designin detail, we will briefly introduce the EXIF standard and theImageJ platforms that form the building blocks for our meta-design.
EXIF tags (hereafter EXIF) are metadata tags added by con-sumer devices such as digital cameras to digital images capturedby these devices (this includes images captured on smartphonestoo). EXIF captures a variety of details ranging from date andtime information to camera settings such as aperture and shutterspeed, and GPS coordinates for the location of capture. EXIFis part of the TIFF specification and can be found in image filetypes such as JPG and PNG in addition to TIFF. The GIF formatdoes not support EXIF. Some tags such as the EXIF version aremandatory while most tags are optional such as the user com-ment tag. EXIF is a consumer specification and does not supportany of the clinical tags in the DICOM header. However, someof the EXIF tags overlap with headers in the DICOM IODs. Weadopt a design approach that leverages the EXIF for clinicaltags.
ImageJ is an image analysis program developed by the NationalInstitute of Health (NIH), widely used for biomedical imageanalysis [11]. ImageJ is an open-source JAVA-based softwarewith an extensible plug-in architecture. The first version whichwas released 25 years back was rewritten as ImageJ2 with addi-tional functionalities. ImageJ2 and Fiji (ImageJ bundled witha range of plugins that facilitate scientific image analysis) arewidely used for biomedical image management [12].DICOM SC IOD is for images that are converted from a non-DICOM format such as JPEG and PNG. It is a modality indepen-dent DICOM format with no constraints on the pixel data format.Though the initial specification was confined to single-frameimages, it has been expanded to include multi-frame images.As SC IOD is modality independent PACS will not assign anymodality [13].We mapped common demographic and study-related tags fromthe DICOM SC IOD to a JSON structure as shown in Figure 1.The DICODerma Java library (hereafter DICODerma) facilitateswriting the JSON, represented as a string, to the ‘UserComment’section of EXIF. DICODerma can read and parse the JSONstring from EXIF. This enables mapping useful DICOM tags to EXIF enabling the inclusion of patient metadata in consumer im-age files. DICODerma uses popular and open-source dcm4chejava library [14] for writing DICOM (dcm) files from JPEG fileformat, a popular format supported by most capture devices andimage editing software. These converted DICOM files can beused in any system that supports these standards.
ImageJ has several plugins that can display, edit, save and pro-cess digital images in various formats including DICOM. Owingto the extensible, plugin architecture of ImageJ, advanced usesnot natively supported by ImageJ can be added. The modulesare typically written in Java and can be installed from the ImageJuser interface or manually copied to the plugins folder in theImageJ folder structure. The additional functions introduced bythe plugins can be easily integrated into the ImageJ graphicaluser interface (GUI). The plugins, depending on their type andfunctions, implement certain abstract base classes in the ImageJcore and provide implementations for methods such as run and setup .DIT4IJ is an ImageJ plugin that adds the following four func-tions as submenus in the ImageJ. The ‘add tags’ function re-ceives the tags --- patient id, patient name, gender, date timeand diagnosis — from the user and converts them to a JSONstring and writes the string to the ‘User Comment’ EXIF tagof an image. The ‘StudyDescription’ tag is used to capture thediagnosis ( Figure 1). The ImageJ provides the interface forinputting these tags ( Figure 2). DIT4IJ can display these tagsfor any image and provides an interface to search for these tagsin a folder structure. For example, it can open all images of aparticular diagnosis such as lichen planus by searching in anyspecified file folder in the computer, including all subfoldersin the search. The consumer file formats such as JPEG can beconverted into DICOM and saved anywhere in the system. Thisconverted DICOM (dcm) file can be used with any DICOMaware application. See the attached video file to see the usagedemonstration.
We address the common limitation in the existing consumerimage formats — the lack of support for patient metadata. Thisneed is addressed without a ff ecting the images by the use ofEXIF. The clinicians can still continue to use their imaging toolsfor capture, processing and visualization of images. Some of thevisualization tools support viewing the EXIF metatags includingUserComment, though the JSON formatted string is not meantfor direct visualization.We introduce ImageJ, a popular biomedical imaging softwareto the dermatology community. ImageJ is currently not a popu-lar image viewer for clinical dermatology though it has use indermatopathology. Some of the image manipulation algorithmsfor clinical and cosmetic dermatology can be easily built us-ing the modular and extensible ImageJ framework. Some suchcommercial products are available [15]. We believe that thefunctions introduced by DIT4IJ will make ImageJ, a useful toolin dermatologists’ armamentarium and democratize imagingworkflows. reprint – DICOD erma : A practical approach for metadata management of images in dermatology reprint – DICOD erma : A practical approach for metadata management of images in dermatology DICODerma can only handle JPEG images with the traditionalEXIF structure. Sources that generate other file types such asPNG and GIF cannot be used with DICODerma. DICODermauses the dcm4che library [14] to convert JPEG images to com-pressed DICOM files. All DICOM readers do not yet supportcompressed DICOM files. The chance of inadvertently shar-ing sensitive patient information is a challenge in this methodthough encryption of EXIF is a solution, again at the cost ofincreasing the the complexity [16]. DICODerm needs furtherdevelopment to support other modalities such as dermoscopyand optical coherence tomography.The SC IOD is a general-purpose IOD for use with any digitalimage. As the SC IOD is not associated with any modality,some PACS systems may not handle them well. SC IOD lacksthe meta-data model to cater to dermatologists’ unique needssuch as patient positioning and lighting. However, unlike otherspecialties that need specialty-specific metadata model, the der-matological community’s needs may be minimal. The machine-learning algorithms may be less tolerant of variability in colourand lighting than human observers, and these requirements maychange in the future [17]. We believe that our approach willintroduce dermatologists to the many advantages of standardiza-tion and ignite interest in developing a specialty-specific IOD inthe future. iscussion
The standardization requirements for dermatological images arebeyond the handling of patient metadata. The proposed methodof using EXIF and interconversion with DICOM header fieldsare easily extensible to capture other relevant metadata. Main-stream [18] and specialized search engines [19] are becoming increasingly accurate and useful for dermatologists and residents.DICODerma method can improve the accuracy further becauseof the availability of standard metadata.Teledermatology is becoming increasingly important because ofthe scarcity of dermatologists, especially in resource-poor areas.The exchange of good quaity clinical images between patientsand dermatologists is vital in teledermatology [20]. The discus-sions related to skin findings in pandemics such as COVID-19is crucial for screening. DICODerma may improve the e ffi cientuse of images for these purposes [21].Smartphone based image acquisition is the new normal in derma-tology, with dermoscopic addons becoming available for hand-held devices [22]. Standardizing image capture from handhelddevices along with relevant metadata, is the need of the hour.Vendors can incorporate simple solutions using DICODerma inapps that dermatologists routinely use [23].We propose a simple method and tool for managing imagingmetadata in dermatological images. Our method is suitable foran encounter-based workflow, commonly seen in dermatology.In an encounter-based workflow, the imaging forms part ofother clinical documentation, unlike in an order-based workflowwhere the image-acquisition may be the primary purpose ofthe visit [24]. The possibility of integrating with the enterpriseimaging systems with minimal change to the traditional andstraightforward imaging methods that dermatologists are usedto might lead to the development of more elaborate standards. R eferences [1] Bell R. Eapen. Artificial Intelligence in Dermatology: APractical Introduction to a Paradigm Shift. Indian Derma-tology Online Journal , 11(6):881–889, November 2020.ISSN 2229-5178. doi: 10.4103 / idoj.IDOJ_388_20.[2] Bell Raj Eapen, Norm Archer, and Kamran Sartipi. Lesion-Map: A Method and Tool for the Semantic Annotation ofDermatological Lesions for Documentation and MachineLearning. JMIR Dermatology , 3(1):e18149, April 2020.doi: 10.2196 / ff ery, David Clunie, Clara Curiel-Lewandrowski,Josep Malvehy, H. Peter Soyer, and Allan C. Halpern.Transforming Dermatologic Imaging for the Digital Era:Metadata and Standards. Journal of Digital Imaging , 31(4):568–577, August 2018. ISSN 0897-1889. doi: 10.1007 / s10278-017-0045-8.[4] Taige Cao and Hong Liang Tey. High-definition opticalcoherence tomography – an aid to clinical practice and re-search in dermatology. JDDG: Journal der Deutschen Der-matologischen Gesellschaft , 13(9):886–890, 2015. ISSN1610-0387. doi: 10.1111 / ddg.12768.[5] H. Ohshima, S. Kinoshita, M. Futagawa, H. Takiwaki,K. Washizaki, A. Ishiko, and H. Kanto. Quantification ofallergic and irritant patch test reactions using ImageJ. SkinResearch and Technology , 20(2):177–181, 2014. ISSN1600-0846. doi: 10.1111 / srt.12103.[6] W. D. Bidgood and S. C. Horii. Modular extension of theACR-NEMA DICOM standard to support new diagnos-tic imaging modalities and services. Journal of Digital reprint – DICOD erma : A practical approach for metadata management of images in dermatology Imaging , 9(2):67–77, May 1996. ISSN 0897-1889. doi:10.1007 / BF03168859.[7] Hevner, March, Park, and Ram. Design Science in Infor-mation Systems Research.
MIS Quarterly , 28(1):75, 2004.ISSN 0276-7783. doi: 10.2307 / Procedia Computer Science ,126:1243–1252, January 2018. ISSN 1877-0509. doi:10.1016 / j.procs.2018.08.066.[9] Afsheen Sharifzadeh, Shinjita Das, and Gideon P. Smith.Google reverse image search using dermatology eConsultcases. Dermatologic Therapy , 33(6):e14372, 2020. ISSN1529-8019. doi: 10.1111 / dth.14372.[10] Bell Raj Eapen. DICOM for Dermatol-ogy: GitHub:dermatologist / dicom-dermatology.https: // github.com / dermatologist / dicom-dermatology,2021.[11] Caroline A. Schneider, Wayne S. Rasband, and Kevin W.Eliceiri. NIH Image to ImageJ: 25 years of image analysis. Nature Methods , 9(7):671–675, July 2012. ISSN 1548-7105. doi: 10.1038 / nmeth.2089.[12] Johannes Schindelin, Ignacio Arganda-Carreras, ErwinFrise, Verena Kaynig, Mark Longair, Tobias Pietzsch,Stephan Preibisch, Curtis Rueden, Stephan Saalfeld, Ben-jamin Schmid, Jean-Yves Tinevez, Daniel James White,Volker Hartenstein, Kevin Eliceiri, Pavel Tomancak, andAlbert Cardona. Fiji: An open-source platform forbiological-image analysis. Nature Methods , 9(7):676–682,July 2012. ISSN 1548-7105. doi: 10.1038 / nmeth.2019.[13] Oleg S. Pianykh. DICOM SOPs: Beyond Basic. InOleg S. Pianykh, editor, Digital Imaging and Communi-cations in Medicine (DICOM): A Practical Introductionand Survival Guide , pages 169–175. Springer, Berlin, Hei-delberg, 2012. ISBN 978-3-642-10850-1. doi: 10.1007 / Journal of Digital Imaging , 20 Suppl1:125–129, November 2007. ISSN 0897-1889. doi: 10.1007 / s10278-007-9064-1.[15] De-Tian Xu, Jian-Na Yan, Yong Cui, and Wei Liu. Quan-tifying facial skin erythema more precisely by analyz-ing color channels of The VISIA Red images. Jour-nal of Cosmetic and Laser Therapy: O ffi cial Publica-tion of the European Society for Laser Dermatology ,18(5):296–300, October 2016. ISSN 1476-4180. doi:10.3109 / ff ectin Protecting Image of Copyright Piracy. In Proceedingsof the 6th International Conference on Information Tech-nology: IoT and Smart City , ICIT 2018, pages 214–218,New York, NY, USA, December 2018. Association forComputing Machinery. ISBN 978-1-4503-6629-8. doi:10.1145 / Journal of Digital Imaging , 28(1):41–52, February 2015.ISSN 1618-727X. doi: 10.1007 / s10278-014-9721-0.[18] Montassar Amri and Kaliyadan Feroz. Google searcheshelp with diagnosis in dermatology. Informatics in PrimaryCare , 21(2):70–72, 2014. ISSN 1475-9985. doi: 10.14236 / jhi.v21i2.52.[19] P. Tschandl, G. Argenziano, M. Razmara, and J. Yap. Di-agnostic accuracy of content-based dermatoscopic imageretrieval with deep classification features. British Journalof Dermatology , 181(1):155–165, 2019. ISSN 1365-2133.doi: 10.1111 / bjd.17189.[20] Nadiya Chuchvara, Rachel Patel, Radhika Srivastava,Catherine Reilly, and Babar K. Rao. The growth ofteledermatology: Expanding to reach the underserved. Journal of the American Academy of Dermatology , 82(4):1025–1033, April 2020. ISSN 0190-9622. doi:10.1016 / j.jaad.2019.11.055.[21] Pavane L. Gorrepati and Gideon P. Smith. Analysis ofavailability, types, and implementation of teledermatol-ogy services during COVID-19. Journal of the AmericanAcademy of Dermatology , 83(3):958–959, September 2020.ISSN 0190-9622. doi: 10.1016 / j.jaad.2020.06.022.[22] Linda Tognetti, Diletta Fiorani, Giulia Tonini, LorenzoZuliani, Gennaro Cataldo, Alberto Balistreri, GabrieleCevenini, Elisa Cinotti, and Pietro Rubegni. Dermoscopy:Fundamentals and Technology Advances. In Michele Fimi-ani, Pietro Rubegni, and Elisa Cinotti, editors, Technologyin Practical Dermatology: Non-Invasive Imaging, Lasersand Ulcer Management , pages 3–24. Springer Interna-tional Publishing, Cham, 2020. ISBN 978-3-030-45351-0.doi: 10.1007 / JMIR mHealth and uHealth , 8(9):e20203, September 2020.doi: 10.2196 / Journalof Digital Imaging , 29(5):559–566, October 2016. ISSN1618-727X. doi: 10.1007 //