Explainable Spatial Clustering: Leveraging Spatial Data in Radiation Oncology
Andrew Wentzel, Guadalupe Canahuate, Lisanne van Dijk, Abdallah Mohamed, Clifton David Fuller, G.Elisabeta Marai
EExplainable Spatial Clustering:Leveraging Spatial Data in Radiation Oncology
Andrew Wentzel * University of Illinois at Chicago
Guadalupe Canahuate
University of Iowa
Lisanne V. van Dijk
University of Texas
Abdallah S.R. Mohamed
University of Texas
C.David Fuller
University of Texas
G.Elisabeta Marai
University of Illinois at Chicago A BSTRACT
Advances in data collection in radiation therapy have led to anabundance of opportunities for applying data mining and machinelearning techniques to promote new data-driven insights. In light ofthese advances, supporting collaboration between machine learningexperts and clinicians is important for facilitating better developmentand adoption of these models. Although many medical use-casesrely on spatial data, where understanding and visualizing the un-derlying structure of the data is important, little is known about theinterpretability of spatial clustering results by clinical audiences.In this work, we reflect on the design of visualizations for explain-ing novel approaches to clustering complex anatomical data fromhead and neck cancer patients. These visualizations were developed,through participatory design, for clinical audiences during a multi-year collaboration with radiation oncologists and statisticians. Wedistill this collaboration into a set of lessons learned for creatingvisual and explainable spatial clustering for clinical users.
Keywords:
Data Clustering and Aggregation, Life Sciences, Col-laboration, Mixed Initiative Human-Machine Analysis, Guidelines
NTRODUCTION
One of the most important applications of machine learning (ML)techniques to oncological healthcare is patient stratification. Stratifi-cation is the division of a patient population (group) into subgroups,or ”strata”. Each strata represents a particular section of that patientpopulation. The strata are typically correlated with specific demo-graphic or disease traits, and specific outcomes including survival orside effects in response to specific treatments. The nature of patientstratification makes it well suited for clustering—an unsuperviseddata mining technique that groups patients based on some measureof distance between them. When the distance measure and clusteringalgorithm is well chosen, clustering can generate novel insights andhelp discover previously undiscovered structure in the data.Oncological data is often tied to a patient’s anatomy, which com-plicates the construction of a similarity measure between patientsand the selection of a clustering algorithm. In cancer patients, thespatial information of the tumor and surrounding anatomy is vital indeciding optimal treatment and forecasting patient endpoints. Thus,understanding the underlying spatial structure of the data duringthe clustering process is important. Despite a widespread interestin sophisticated clustering techniques for patient stratification, theadoption of clustering in oncology is stifled by the difficulty inunderstanding the inner workings of spatially-informed clustering.In this work, we examine a participatory design of explanatoryvisual encodings born out of a long-term collaboration between * e-mail: [email protected] oncology, data mining, and data visualization practitioners perform-ing analysis on a cohort of head and neck cancer patients [23, 29].Specifically, this work looks at interpreting clusters of stratified headand neck cancer patients based on secondary disease spread to thelymph nodes, with the goal of helping clinical users understand thestrata and use them to help predict the toxicity outcome of diseasetreatment. We reflect on the process of creating domain-specific vi-sual encodings through participatory design to help ”bridge the gap”between the data experts and healthcare experts [15]. We furtherexplore obstacles and successes when creating visual encodings forinterpreting data mining techniques, and for communicating withoncology experts with limited background in both visualization andin artificial intelligence. ELATED W ORK
Cluster Explainability
Interpretation and visualization of clustersis a common analysis task tightly integrated with dimensionality re-duction in general, but is less understood than traditional explainableAI (XAI) approaches, which are generally focused on supervisedlearning. A task analysis of 10 data analysts [3] included 3 tasksrelated to clusters: verifying clusters, naming clusters, and matchingclusters to existing classes. General methods of cluster visualizinghave typically been linked to low-dimensionality embedding, whereclasses are shown plotted in a 2 or 3 dimensional space, and cluster-membership is shown on top of the data in the lower-dimensionspace [1,11,31]. Hierarchical clustering methods, where clusters areiteratively created at different levels of granularity, have commonlybeen visualized as dendrograms. When dimensionality reductionisn’t appropriate, general methods of multivariate data visualizationare used, such as parallel coordinate plots [8] or specialized glyphencodings [5]. Other systems synthesize existing methods to supportvisual steering and clustering for scientists [6, 7, 25]. While somerecent work has dealt with clustering ensemble geospatial data [19],we are not aware of any methods that deal explicitly with clusteringanatomical or 3-d data as in this work.
Vis in Healthcare
Visualization approaches to healthcare problemsoften focus on supporting data exploration, rather than understandingpredictive models [2, 4, 17]. Certain systems for model explorationhave been developed to aid in the development of regression modelsbased on the workflows of biostatisticians [10, 28]. Other systemshave applied visualization for clustering cancer data [25], and pre-dicting infection spread in hospital wards [26]. For spatial data,Grossmann et al. [13] incorporated methods for visualizing clustersbased on bladder shape to support a retrospective study on prostatecancer patients. Some works have attempted to identify design con-siderations when working with domain experts in healthcare [20,27].However, with the exception of Raidou et al. [27], most of theseconsiderations do not apply to clustering or spatial data, and arelargely focused on analytics and electronic health record data. Asa result, there is a dearth of papers discussing how to approachunsupervised XAI models to reach clinical audiences. a r X i v : . [ c s . H C ] O c t B ACKGROUND
In many cancer patients, tumors metastasize into the lymphaticsystem, causing lymph nodes to become ”involved”—affected bysecondary nodal tumors. The lymphatic system forms a complexchain of lymph nodes, and these secondary tumors spread alongthese chains to adjacent regions stochastically. Affected lymphnodes are a long-established factor in determining patient outcomesin head and neck cancer [16]. Current predictive systems use astaging system based on the size and number of nodal tumors, butmiss more nuanced predictions about how the different patternsof nodal spread may affect toxicity outcomes [14, 35]. No priormachine learning methods correctly handle this type of spatial data,due to a lack of spatial similarity measures [12, 18].Our data comes from a cohort of 582 head and neck cancer pa-tients collected retrospectively from the MD Anderson Cancer Cen-ter. All patients survived for at least 6 months after treatment. Datawas collected on the presence of 2 severe side effects: feeding tubedependency, and aspiration - fluid in the lungs that requires removal.We mainly consider the presence of either of these side effects, whichwe define as as radiation-associated dysphagia (RAD) [9]. The dataalso encodes the disease spread to 9 connected regions (denoted aslevels 1A-6) on each side of the head, along with the disconnectedretropharyngeal lymph node (RP or RPN). Many patients in thiscohort had unique patterns of disease spread to the lymph nodes.The project consisted of 2 phases with distinct design require-ments. In phase 1 (model development), we worked alongside sixdomain experts in radiation oncology, and two data analysts withdata mining and biostatistics backgrounds, over four years. Duringthis time we developed, validated, and deployed an anatomically-informed patient stratification method based on each patient’s pat-terns of diseased lymph nodes [18]. To demonstrate the importantrole of spatiality, the stratification used only anatomical features. Wemet with representatives from this group up to three times per weekvia teleconferencing, as well as in quarterly face to face meetings. Inphase 2 (model dissemination), our results needed to be analyzed anddelivered to the larger radiation oncology community. In this stage,we received feedback from three additional radiation oncologistsand two bioinformaticians with expertise in head and neck cancer.The final stratification approach is available to clinicians through anopen-source interface [23]. Below, we reflect on the design process,which focused on an activity-centered design paradigm [22], alongwith feedback from the domain experts.
ODEL D EVELOPMENT P HASE
In phase 1, we worked to identify a meaningful, anatomically-informed distance measure between patients, as well as an appro-priate method of clustering the patients. We developed an approachin which each side of the head was treated as a graph. Nodes inthis graph corresponded with regions in the head that aligned withthose used in existing oncology literature, and regions that wereanatomically adjacent in the head were connected in the graph as anedge. Each patient was treated as two sub-graphs, one for each sideof the head, containing only the nodes with nodal tumors. A distancemeasure based on these graphs then needed to be identified, along-side a clustering technique that led to meaningful clusters (activity2). Clustering was performed using only the spatial disease spreadcaptured by the graph model. Because identifying relevant struc-tures in oncological data is nontrivial, defining this methodologyrequired iterative experimentation with different features, clusteringtechniques, numbers of clusters, and other parameters [30]. Weidentified the following activities that required visual support:1. Identify and analyze the relevant spatial data features underly-ing one datapoint (i.e. patient).2. Analyze the effects of different spatial similarity measures on
Figure 1: (A) Lymph nodes overlaid over a diagram of the neck. (B)Example graphs of diseased nodes for 2 individual patients (datapointrepresentation). (C) Example consensus graph for 1 cluster (clusterrepresentation). The top-right graph shows disease spread with 66+%of patients on the right nodes in 1B, 2A, 2B, and 3, and disease in1-33% of patients in right node 4. The bottom-right graph similarlyindicates involvement of >
66% and <
33% of patients in left nodes 4and 3, respectively. clustering (i.e. why two patients are considered to be similarunder a specific measure).3. Analyze the representative patterns and pattern variation withineach cluster.
Datapoint Representation
The first design followed a graphmetaphor to encode the diseased regions for each patient (activity1). A compact graph that followed an anatomical map of lymphnode chains for half the head (because the problem is symmetric)was used as a template for each patient (Figure 1-A), based on ideasfrom biological network visualization [24, 32]. For each patient,two envelopes were drawn over their diseased nodes. Green andpurple envelopes were used for the left and right side of the head,respectively. Areas where envelopes overlap are shown in blue anddenote regions where tumors occur on both sides of the head, whichare of particular interest to oncologists (Figure 1-B).This design allowed for a compact representation of a complexspatial feature space, while following the mathematical intuitionbehind different distance measures. These graphs were incorporatedinto an interface that shows patients and compares them to their mostsimilar matches. The compact representation was useful in identi-fying the spatial features of each datapoint, as well as interpretingdistance between patients.
Cluster Representation
In a first attempt to characterize eachcluster, we selected a representative patient for each cluster: i.e., thepatient closest to the cluster centroid (activity 3). The representativepatient, however, did not capture any intra-cluster variability. Sub-sequently, we created a new representative encoding by placing themost commonly affected nodes for a cluster in a ”consensus” graph.Nodes where of the patients in that cluster had nodal tumors wereoutlined in envelopes. However, in this new representation it wasunclear why certain clusters were not merged. In a third iteration,we added a different marker (squares) for nodes where less than ofthe patients in that cluster, but at least one patient had nodal tumors(Figure 1-C). We used shape, rather than color, because hue alreadyencoded disease laterality, and further intensity variation was notlegible given the small scale.However, at small scale, the markers and colors for multiple clus-ters became hard to distinguish. Additionally, outside clinicians andbioinformaticians mis-interpreted the third encoding as representing igure 2: Part of an augmented dendrogram of lymph node clusters(clusters 1-3 not shown; the full dendrogram is available in Lucianiet al. Leaves of the tree are smaller clusters that merge at higherlevels according to the agglomerative clustering algorithm. Clustersare id-ed by colors in the graph. Clusters are further augmented withbreakdowns of relevant clinical covariates of interest (F.T.: FeedingTube; Asp.: Aspiration). only one patient in that cluster, and in one case, as clusters contain-ing identical patients. In the fourth design, two stacked graphs wereused for each side of the head for each cluster, and visual scaffold-ing [21] was used to explain the progression from a single datapointrepresentation to the consensus graph. The consensus graphs wereplaced within dendrograms, which showed the consensus graphsof smaller component clusters within each larger cluster of interest(Figure 2). To further clarify the hierarchical clustering process,we added explicit color-coding of the dendrograms, with labels andcolors showing the cluster names and tracing the merging process, aswell as small statistics tables showing the patient toxicity outcomeswithin each larger cluster. LINICAL M ODEL D ISSEMINATION P HASE
In the second phase, our results needed to be able to reach theirintended audience: clinical radiation oncologists. While the method-ological development was concerned with the clinical validity ofthe analysis, clinical readers are more concerned with significanceof the results, and place more importance on feasibility, trust in theunderlying covariates, and the implications of the results [33, 34],rather than the methodology used, which had already been peer-reviewed [18]. In this phase, we used four clusters to align withexisting staging systems, and the clustering still only consideredspatial disease spread. In order to effectively communicate results,we identified the following activities to support:1. Describe patient clusters from an anatomical perspective.2. Identify each cluster’s underlying structure.3. Connect structural cluster differences to clinical covariates.4. Explain plausible causal relationships between the clusters andcorrelated patient outcomes.
Cluster Conditionals
The first design relied on two synergisticencodings for each cluster. The first encoding expanded on the origi-nal anatomical diagram to show the most discriminative features ineach cluster (conditionals). To do this, a decision tree was trainedon the cohort to predict cluster membership with 100% accuracy
Figure 3: (A) Cluster conditionals. (Top-left) Map of the regions in theneck. Color indicates when the decision tree classified a patient intothe cluster based on if the region had no disease (pale red), tumorsin one side of the head (red), both sides of the head (dark red), ora combination of two options. (Bottom-left) Radar chart showing thepercentage of patients in the cluster with nodal tumors in a givenregion. Color indicates the presence of tumors in exactly one (palered) or two (dark red) sides of the head. (B) Second iteration of clusterconditionals. (Top-right) Membership diagram showing the regionsin the head. Color indicates when all (red), a subset of (yellow), ornone of (blue) the patients in a cluster had nodal tumors in a region.(Bottom-right) Decision-tree based diagram. Colors indicate when adecision tree classified a patient into that cluster. using the number of sides of the head with a nodal tumor in eachregion of the head and neck, which could be 0 (no disease), 1 (unilat-eral disease), or 2 (bilateral disease). Because experts who had notparticipated in the methodology design process had trouble under-standing the graph-based encoding, the set of variables consideredsufficient to any patient in the training data into a given cluster wasthen encoded into an anatomical region diagram of one side of theneck (Figure 3-A). By focusing on the regions that the decision treeconsidered, the diagram highlighted the regions that best identifiedthe key differences between clusters, while omitting regions withcommonalities between then, in order to support activities 2 and 3.The second encoding was a radar plot of the percentage of people ina cluster with either unilateral or bilateral disease spread in a givenregion of the neck. This representation allowed for a more detailedview of the overall distribution of tumors in each cluster (activity 1).The initial cluster visualization design using trees was found tointuitively make sense to clinical collaborators. However, they haddifficulty understanding the underlying explanation of the diagramsand how they were generated within the space of a figure caption,as they had limited experience with decision trees. Collaboratorsincorrectly assumed that all combinations of nodal disease in thediagrams were shared between all patients in a given cluster. Ad-ditionally, our collaborators pointed out that while the one-sideddiagram of the neck was common for surgical applications, radiationoncologists often visualized the neck in terms of a front view thatincluded both sides of the head simultaneously.In the second design (Figure 3-B), each cluster is encoded usinga frontal view anatomical diagram. A red-yellow-blue categoricalcolor scheme was used to mark which regions were diseased in allpatients, some patients, or no patients within the cluster, respectively,following the original intuition of our collaborators. An additionalanatomical diagram based on the decision tree was included for eachcluster below the membership diagrams. Since the new diagram igure 4: Designs for two high-risk cluster conditionals. (Top) Spatialheatmaps showing the portion of patients with nodal tumors in eachregion for at least one (left) or both (right) sides of the head. Regionsmost informative in determining cluster membership are outlined in athick dark border. (Bottom) Radar charts showing the percentage ofpatients within the cluster with a given toxicity outcome (FT/RAD/AS),and those within an existing risk-staging group (T1/T4/N2a/N2b/N2c). included both sides of the head, color was used to show when thedecision tree split the cluster based on the presence of disease (red),or absence of disease (gray) in a given region, while white regionswere not considered in the model.
Cluster Membership
The conditional designs were better-received by the clinicians, but difficulties in understanding the col-ormap and the lack of detail in the cluster membership made itchallenging to correctly draw insights. To address these concerns,we designed a new heatmap diagram of the neck (Figure 4), whichused a sequential white-red color scheme to encode the number ofpatients in a cluster with disease in a given region (activity 1). Wenote that head and neck oncologists account for symmetry whendiscussing similar patients, and thus a symmetric encoding was adesired feature. A simplified decision tree was trained to identifythe regions that contained the most information about cluster mem-bership, which were outlined with a dark border in the heatmaps(activity 2). Additional labels were included, to indicate the left/rightsides of the diagram show unilateral vs. bilateral involvement, ratherthan the literal left/right sides of the head.To help indicate the relationship between the clusters and otherclinical data, covariates and outcomes that were the most interestingto clinicians were included in a radar chart alongside the heatmapsfor each cluster. The inclusion of these data helped with the collabo-rators’ ability to discuss potential relationships between the structureof the clusters and correlated outcomes (activities 3 and 4).
ESIGN L ESSONS
Through the course of these iterations, we have distilled designlessons for interpretable clustering with spatial data.
L1.
Use visual scaffolding based on users’ spatial background.
Spatial representations were, as expected, essential to understandingthe clustering. Furthermore, encodings were better received whenthey mapped directly to the users’ model of the problem, particularlywhen the users did not participate in the design. Using a graph-based encoding for the patient lymph node chains allowed us to drawparallels to graph theory, which was useful when testing similaritymeasures that were based on graph matching methods. In contrast,when designing for the wider oncology community, the encodingbest received was created by visually scaffolding the graph directlyonto an anatomical diagram of the neck from clinical literature.
L2.
Incorporate visual details specific to the user’s activities.
Whendesigning for the methodology development, we focused on develop-ing the clustering algorithm and ensuring that the results were moremeaningful than existing methods. Placing the cluster visualizationswithin a dendrogram allowed the users to scrutinize the inner work-ings of the clusters at different scales. In contrast, clinicians weremore results-focused. Namely, their key interests focused on thespatial structure underlying each cluster, how the clusters related tooutcomes and existing clinical categories, and if these correlationscould be explained in a way that was supported by clinical intuition.Thus, the design benefited from incorporating anatomical detailsand additional clinical covariates that were not considered whendesigning the model.
L3.
Show secondary variables and outcomes.
Design iterations thatfailed to include explicit labeling of results directly into the figureled to confusion. In the initial dendrograms, viewers had troubleconnecting the clusters directly to other statistical analysis. For theclinical figures, collaborators often assumed that there were directcausal relationships between variables shown in the figure. In thiscase, it was useful to include potential confounding variables, toallow the readers to come up with alternative hypotheses.
L4.
Design for both interactive and static visualization.
In ourexperience, we started out with interactive designs aiming to assista relatively small group of domain experts, who participated in thedesign process. Relatively quickly, it became obvious that the spatialclustering had to be explained to a broader audience that expectedstatic visualizations, in the style of biomedical illustrations. Futureworks will stay closer to the illustrative style during the interactivemodel development phase, to reduce the cost of later redesign.
L5.
Build decision trees and conditionals to help explain spatialcluster differences.
When working with the broader audience, wefound that the easiest way to explain cluster differences required ex-plicit construction of decision trees, and ”conditionals” based on thestructure of the data—attempting to directly encode the differenceswas infeasible.
ONCLUSION
This work reflects on the process of designing visualizations forclustering with anatomical spatial data. These designs were devel-oped in two phases over several years, using participatory designalongside collaborators with background in bioinformatics and radi-ation oncology. Through these designs iterations, we distill a set oflessons learned. While we focus on a particular problem, our designapproach can be generalized to other type of cancer with spatiallydependent data. These designs are part of a larger body of workborne out of a multi-year collaboration with domain experts withanatomical cancer data. By incorporating additional insights fromsibling projects, we aim to develop a comprehensive set of designguidelines for visualizing clusters of spatial data and effectivelydisseminating these results to domain expert audiences outside ofthe visualization community. A CKNOWLEDGMENTS
This work is supported by the US National Institutes ofHealth, through awards NIH NCI-R01CA214825 and NIH NCI-R01CA2251. We thank all members of the Electronic VisualizationLaboratory, members of the MD Anderson Head and Neck CancerQuantitative Imaging Collaborative Group, and our collaborators atthe University of Iowa and University of Minnesota.
EFERENCES [1] B. Alper, N. Riche, G. Ramos, and M. Czerwinski. Design study oflinesets, a novel set visualization technique.
IEEE Trans. Vis. Comp.Graph. (TCVG) , 2011.[2] J. Bernard, D. Sessler, J. Kohlhammer, and R. A. Ruddle. Usingdashboard networks to visualize multiple patient histories: a designstudy on post-operative prostate cancer.
IEEE Trans. Vis. Comp. Graph.(TCVG) , 2018.[3] M. Brehmer, M. Sedlmair, S. Ingram, and T. Munzner. Visualizingdimensionally-reduced data: Interviews with analysts and a charac-terization of task sequences. In
Proc. 5th Work. Beyond Time Errors:Novel Eval. Meth. Vis. , 2014.[4] A. A. Bui, D. R. Aberle, and H. Kangarloo. Timeline: visualizingintegrated patient records.
IEEE Trans. Info. Techn Biomed. , 2007.[5] N. Cao, D. Gotz, J. Sun, and H. Qu. Dicon: Interactive visual analysisof multidimensional clusters.
IEEE Trans. Vis. Comp. Graph. (TCVG) ,2011.[6] R. Cava, C. M. D. S. Freitas, and M. Winckler. Clustervis: visualizingnodes attributes in multivariate graphs. In
Proc. Symp. App. Comp. ,2017.[7] M. Cavallo and C¸ . Demiralp. Clustrophile 2: Guided visual clusteringanalysis.
IEEE Trans. Vis. Comp. Graph. (TCVG) , 2018.[8] S.-Y. Chou, S.-W. Lin, and C.-S. Yeh. Cluster identification withparallel coordinates.
Patt. Recog. Letters , 1999.[9] K. M. Christopherson, A. Ghosh, A. S. R. Mohamed, M. Kamal, G. B.Gunn, T. Dale, J. Kalpathy-Cramer, J. Messer, A. S. Garden, H. Elha-lawani, et al. Chronic radiation-associated dysphagia in oropharyngealcancer survivors: Towards age-adjusted dose constraints for deglutitivemuscles.
Clinical Transl. Rad. Onco. , 18:16–22, 2019.[10] D. Dingen, M. van’t Veer, P. Houthuizen, E. H. Mestrom, E. H. Korsten,A. R. Bouwman, and J. Van Wijk. RegressionExplorer: Interactiveexploration of logistic regression models with subgroup analysis.
IEEETrans. Vis. Comp. Graph. (TVCG) , 2018.[11] A. Efrat, Y. Hu, S. G. Kobourov, and S. Pupyrev. Mapsets: visualizingembedded and clustered graphs. In
Inte. Symp. Graph Draw.
Springer,2014.[12] H. Elhalawani, T. A. Lin, S. Volpe, A. S. Mohamed, A. L. White,J. Zafereo, A. J. Wong, J. E. Berends, S. AboHashem, B. Williams, et al.Machine learning applications in head and neck radiation oncology:lessons from open-source radiomics challenges.
Front. Onco. , 8:294,2018.[13] N. Grossmann, O. Casares-Magaz, L. P. Muren, V. Moiseenko, J. P.Einck, M. E. Gr¨oller, and R. G. Raidou. Pelvis runner: Visualizingpelvic organ variability in a cohort of radiotherapy patients.
Vis. Comp.Bio. Med. Conf. (VCBM) , 2019.[14] S. H. Huang and B. O’Sullivan. Overview of the 8th edition tnmclassification for head and neck cancer.
Curr. Treat. Op. Onco. , 2017.[15] S. J¨anicke, P. Kaur, P. Kuzmicki, and J. Schmidt. Participatory visual-ization design as an approach to minimize the gap between research andapplication. In
Gap bet. Vis. Res. Vis. Soft. (VisGap) . The EurographicsAssociation, 2020.[16] C. R. Leemans, R. Tiwari, J. J. Nauta, I. V. D. Waal, and G. B. Snow.Recurrence at the primary site in head and neck cancer and the signifi-cance of neck lymph node metastases as a prognostic factor.
Cancer ,1994.[17] M. H. Loorak, C. Perin, N. Kamal, M. Hill, and S. Carpendale. Times-pan: Using visualization to explore temporal multi-dimensional data ofstroke patients.
IEEE Trans. Vis. Comp. Graph. (TCVG) , 2015.[18] T. Luciani, A. Wentzel, B. Elgohari, H. Elhalawani, A. Mohamed,G. Canahuate, D. M. Vock, C. D. Fuller, and G. E. Marai. A spatialneighborhood methodology for computing and analyzing lymph nodecarcinoma similarity in precision medicine.
J. Biomed. Inform. , 2020.[19] B. Ma and A. Entezari. An interactive framework for visualization ofweather forecast ensembles.
IEEE Trans. Vis. Comp. Graph. (TCVG) ,2018.[20] R. G. C. Maack, D. Saur, H. Hagen, G. Scheuermann, and C. Gill-man. Towards Closing the Gap of Medical Visualization Researchand Clinical Daily Routine. In C. Gillmann, M. Krone, G. Reina,and T. Wischgoll, eds.,
Gap bet. Vis. Res. Vis. Soft. (VisGap) . The Eurographics Association, 2020.[21] G. E. Marai. Visual scaffolding in integrated spatial and nonspatialanalysis. In
EuroVis Works. Vis. Ana. (EuroVA) . The EurographicsAssociation, 2015. doi: 10.2312/eurova.20151097[22] G. E. Marai. Activity-centered domain characterization for problem-driven scientific visualization.
IEEE Trans. Vis. Comp. Graph. (TCVG) ,2017.[23] G. E. Marai, C. Ma, A. T. Burks, F. Pellolio, G. Canahuate, D. M. Vock,A. S. Mohamed, and C. D. Fuller. Precision risk analysis of cancertherapy with interactive nomograms and survival plots.
IEEE Trans.Vis. Comp. Graph. (TCVG) , 2018.[24] G. E. Marai, B. Pinaud, K. B¨uhler, A. Lex, and J. H. Morris. Tensimple rules to create biological network figures for communication,2019. doi: doi.org/10.1371/journal.pcbi.1007244[25] T. Metsalu and J. Vilo. Clustvis: a web tool for visualizing clusteringof multivariate data using principal component analysis and heatmap.
Nucleic acids research , 2015.[26] M. M¨uller, M. Petzold, M. Wunderlich, T. Baumgartl, M. H¨ohn,V. Eichel, N. Mutters, S. Scheithauer, M. Marschollek, and T. vonLandesberger. Visual analysis for hospital infection control using arnn model. In
EuroVis Work. Vis. Ana. (EuroVA) . The EurographicsAssociation, 2020.[27] R. G. Raidou, K. Furmanova, N. Grossmann, O. Casares-Magaz,V. Moiseenko, J. P. Einck, M. Gr¨oller, and L. P. Muren. Lessons learntfrom developing visual analytics applications for adaptive prostatecancer radiotherapy. In
Gap bet. Vis. Res. Vis. Soft. (VisGap) . TheEurographics Association, 2020.[28] D. Rojo Garcia, N. N. Htun, and K. Verbert. GaCoVi: a CorrelationVisualization to Support Interpretability-Aware Feature Selection.
Proc.EuroVis 2020 Short Papers , 2020.[29] T. Sheu, D. M. Vock, A. S. Mohamed, N. Gross, C. Mulcahy,M. Zafereo, G. B. Gunn, A. S. Garden, P. Sevak, J. Phan, et al. Con-ditional survival analysis of patients with locally advanced laryngealcancer: construction of a dynamic risk model and clinical nomogram.
Sci. reports , 7:43928, 2017.[30] J. Tosado, L. Zdilar, H. Elhalawani, B. Elgohari, D. M. Vock, G. E.Marai, C. Fuller, A. S. Mohamed, and G. Canahuate. Clustering oflargely right-censored oropharyngeal head and neck cancer patients fordiscriminative groupings to improve outcome prediction.
Sci. reports ,10(1):1–14, 2020.[31] J. Wenskovitch, I. Crandell, N. Ramakrishnan, L. House, and C. North.Towards a systematic combination of dimension reduction and clus-tering in visual analytics.
IEEE Trans. Vis. Comp. Graph. (TCVG) ,2017.[32] J. E. Wenskovitch, L. A. Harris, J.-J. Tapia, J. R. Faeder, and G. E.Marai. MOSBIE: a tool for comparison and analysis of rule-basedbiochemical models.
BMC Bioinfo. , 15(1):1–16, 2014.[33] A. Wentzel, P. Hanula, T. Luciani, B. Elgohari, H. Elhalawani,G. Canahuate, D. Vock, C. D. Fuller, and G. E. Marai. Cohort-basedT-SSIM visual computing for radiation therapy prediction and explo-ration.
IEEE Trans. Vis. Comp. Graph. (TCVG) , 26(1):949–959, 2019.[34] A. Wentzel, P. Hanula, L. V. van Dijk, B. Elgohari, A. S. Mohamed,C. E. Cardenas, C. D. Fuller, D. M. Vock, G. Canahuate, and G. E.Marai. Precision toxicity correlates of tumor spatial proximity to organsat risk in cancer patients receiving intensity-modulated radiotherapy.
Radiotherapy & Onco. , 2020.[35] J. Wu, M. F. Gensheimer, N. Zhang, F. Han, R. Liang, Y. Qian,C. Zhang, N. Fischbein, E. L. Pollom, B. Beadle, et al. Integratingtumor and nodal imaging characteristics at baseline and mid-treatmentcomputed tomography scans to predict distant metastasis in oropha-ryngeal cancer treated with concurrent chemoradiotherapy.