Visualization of Human Spine Biomechanics for Spinal Surgery
VVisualization of Human Spine Biomechanics for Spinal Surgery
Pepe Eulzer, Sabine Bauer, Francis Kilian, Kai Lawonn
Fig. 1. Overview of exploration techniques for a spinal disc deformation simulation. We extract patient movement as an animation (left),which we connect with detailed simulation results (middle), including force impact directions. Each value-over-time plot is associatedwith the corresponding spinal disc using patient-specific anatomy. A simplified depiction allows for at-a-glance assessments andensemble comparisons (right).
Abstract — We propose a visualization application, designed for the exploration of human spine simulation data. Our goal is to supportresearch in biomechanical spine simulation and advance efforts to implement simulation-backed analysis in surgical applications.Biomechanical simulation is a state-of-the-art technique for analyzing load distributions of spinal structures. Through the inclusionof patient-specific data, such simulations may facilitate personalized treatment and customized surgical interventions. Difficulties inspine modelling and simulation can be partly attributed to poor result representation, which may also be a hindrance when introducingsuch techniques into a clinical environment. Comparisons of measurements across multiple similar anatomical structures and theintegration of temporal data make commonly available diagrams and charts insufficient for an intuitive and systematic display ofresults. Therefore, we facilitate methods such as multiple coordinated views, abstraction and focus and context to display simulationoutcomes in a dedicated tool. By linking the result data with patient-specific anatomy, we make relevant parameters tangible forclinicians. Furthermore, we introduce new concepts to show the directions of impact force vectors, which were not accessible before.We integrated our toolset into a spine segmentation and simulation pipeline and evaluated our methods with both surgeons andbiomechanical researchers. When comparing our methods against standard representations that are currently in use, we foundincreases in accuracy and speed in data exploration tasks. In a qualitative review, domain experts deemed the tool highly useful whendealing with simulation result data, which typically combines time-dependent patient movement and the resulting force distributions onspinal structures.
Index Terms —Medical visualization, bioinformatics, coordinated views, focus and context, biomechanical simulation.
NTRODUCTION
With an estimated 80% of the population being affected by back painat some point in their lives [2, 27], the prevalence of this illness hasincreased rapidly in recent years [24]. Its underlying causes can rangefrom diseases and personal anatomical characteristics [26] to modernera risk factors associated with low back or neck pain, such as over-weight [29, 47] or the extensive use of smartphones, resulting in a • Pepe Eulzer is with the University of Jena, Germany. E-mail:[email protected].• Sabine Bauer is with the University of Koblenz-Landau, Germany. E-mail:[email protected].• Francis Kilian is with the Department of Spine Surgery at Cath. ClinicKoblenz-Montabaur, Germany. E-mail: [email protected].• Kai Lawonn is with the University of Jena, Germany. E-mail:[email protected] received xx xxx. 201x; accepted xx xxx. 201x. Date of Publicationxx xxx. 201x; date of current version xx xxx. 201x. For information onobtaining reprints of this article, please send e-mail to: [email protected] Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx condition that received the name “text neck” [12]. Consequences arestrained joints, bones and tissue, which may degenerate over time andcause chronic pain. Treatment options are as versatile as causes, withspinal surgery being an established option to treat severe cases [51].Current research in biomechanical simulation is focused on provid-ing a means to better understand the cause and effect relationship ofspinal disorders, as well as opening up the possibility of comparingand personalizing treatment options. Modelling forces and torquesoccurring in different spinal structures under various load cases canprovide valuable information to clinicians and researchers. For ex-ample, it is problematic to perform in vivo measurements to test ifa procedure like lumbar fusion has negative consequences for adja-cent structures [25, 38]. A simulation, on the other hand, is certainlyviable [6, 53]. Similarly, more abstract research questions can be exam-ined, such as the effects of different weight classes on spinal load [9].Recent advances in personalized medicine, like implants build withrapid prototyping techniques [3, 52, 57], raise the feasibility of patient-specific treatments. Ahead of an intervention, simulation can providevaluable feedback on, e.g., force distribution, when using differentimplant types [8].With increasingly sophisticated biomechanical simulations emergingrapidly, the computed results are also becoming more complex. When a r X i v : . [ c s . G R ] S e p imulating properties of spinal structures, researchers are faced with thechallenge of understanding and relating a large quantity of parameterscalculated for the individual vertebrae, joints and discs. Especially theintroduction of a temporal dimension, e.g., by introducing patient move-ment, complicates this analysis process. Furthermore, we will latershow that medical practitioners have difficulty to understand typicalsimulation output, which may be a major hindrance when such systemsare to be adapted in clinical environments. We propose a visualizationframework, which facilitates intuitive exploration of clinically relevantresults from biomechanical spine simulation (cf. Fig. 1). We integratedour methods into an existing segmentation and simulation pipeline. Ourmain contributions are components of this application:• We facilitate exploration of all pipeline outputs: patient-specificspinal anatomy, animation of movements and temporal simulationresult data.• We designed anatomically aligned plots for intuitive data analysisacross multiple vertebrae.• We propose a 3D depiction for anatomical assessments, a 2D viewfor singular datasets and a simplified representation for ensemblecomparisons.• Additionally, we introduce embedded glyphs to encode forcedirections, which were not accessible before. EDICAL AND S IMULATION B ACKGROUND
We will now summarize necessary context and terminology from boththe medical and simulation domains.
Anatomy of the spine.
Along its length an adult’s spine follows fourtypical curvatures, according to which the vertebrae are anatomicallygrouped. From neck to pelvis, these groups are the cervical spine(vertebrae C1-C7), the thoracic spine (vertebrae Th1-Th12), the lumbarspine (vertebrae L1-L5) and the Os sacrum. Clinically relevant areparticularly the transition regions, as they are often predilection sitesfor spinal diseases (e.g., disc prolapse). With exception of the first andsecond cervical vertebra, all vertebrae follow a common blueprint (cf.Fig. 2). The vertebral bodies are stacked enclosing discs, which cushionthe surrounding bones and allow the torso to perform bending motions.Each vertebra also connects to the structure above and below throughfacet joints located to the left and right of the vertebral body. Dependingon the spine segment, they are tilted in different directions, facilitatingvarious ranges of motion. Facets and discs are the spinal structures mostprone to injury and degeneration, as they support the body’s weightand distribute forces during movements. The spine is stabilized fromall sides by a complex network of ligaments and muscles, which attachto the vertebrae.
Fig. 2. A vertebra from a cranial point of view (left) and two stackedvertebrae as seen from the right lateral side (right). The vertebraeconnect through two joints and a disc.
Biomechanical Spine Simulation.
Computer simulations of spinalmechanics can provide information that helps to answer medical re-search questions and might become an important asset in planning surgical interventions. Recent advances in multibody simulation allowfor the modelling of patient-specific spinal structures, including thesimulation of complex motion sequences [4–6]. Typically, parameterslike stiffness and damping of the modelled bodies are taken from stud-ies performed on real tissue [48,61] and additional aspects can be takeninto account, such as image-based tissue degeneration scores [30, 62].A biomechanical simulation then computes the resulting forces, de-formations and movements present in the scene and updates them peranimation tick, i.e., minimal time-increment, starting from some initialconfiguration. Researchers and clinicians are interested in the particu-larities of the resulting values, how they affect each structure and howthey react to external factors, e.g., patient movement. In this process,a range of difficulties needs to be addressed. For instance, values ofparameters in literature often fluctuate significantly [7] and humanspine anatomy has a high variance [55]. Therefore, comparable andintuitively understandable simulation results are of utmost importanceto overcome these challenges.
ELATED W ORK
Related work is comprised of the current state of the art in repre-sentation of simulation output. Also, we summarize efforts made invisualization of anatomical features, in particular of the human spine.
A number of methods have been proposed to deal with complex datain 3D space, e.g., resulting from simulation. A common method is toemploy ensemble visualizations and comparative approaches, whichshow multiple data sets within a mutual domain to indicate differentvalues of attributes. Potter et al. [49] proposed a framework for visualanalysis of ensemble data, consisting of linked views, which present anoverview first, then guide the user towards details. Konyha et al. [35]explored methods for analyzing scalar attributes, which they called fam-ilies of curves . They show different levels of data interaction throughbrushing, starting with single views (e.g., scatter plots), then they transi-tion to brushing in multiple views. Demir et al. [14] combined bar andline plots to facilitate visual exploration of 3D ensemble fields. They,too, display an overview first, where detail is provided on demand bynarrowing the view of the camera. Weissenb¨ock et al. [60] extendedthis work by implementing nonlinear scaling of the 1D curves. Thisleads to a more compact view, however, both depictions miss a spatialcorrespondence to the original volume data, where brushing is requiredto indicate areas of interest.A common theme in research on medical imaging and visualizationare techniques for spatial comparison of 3D data, which may also betime-varying, i.e., spatio-temporal. Hermann et al. [28] demonstratedhow image warping can be used to show variability in ensembles ofbiomedical images and Raidou et al. [50] proposed a tool for visual ex-ploration of bladder shape variation during prostate cancer radiotherapy.Another approach is to merge data captured at different time steps to,for example, display a static overview, allowing direct comparison [21].Recently, time-warping was proposed, as a method of selectively defin-ing regions of interest around spatio-temporal events [56]. A number ofmethods utilized in spatial comparisons can be found in the survey ofKim et al. [31], who derive four elemental ways of visualizing such data:Juxtaposition, superimposition, interchange and explicit encoding.A further domain worth mentioning in this context is glyph visu-alization. Different value fields in a 3D domain can be representedusing glyphs placed to preserve spatial correspondence. For example,a typical application area is visualization of tensor fields [63]. Borgoet al. [11] defined guidelines for glyph visualizations and Ropinski etal. [54] introduced a taxonomy for glyphs in medical applications.There exist a number of approaches to depict 3D simulation data,which generally rely on the same fundamental principles as the worksabove. Krekel et al. [36, 37] showed how simulated range of motiondata derived from patient-specific anatomy can be displayed using acombination of interactable 3D representations with embedded glyphsand additional statistical views. In a pre-operative planning system,such tools can be used to assist surgeons in complex decision-makingprocesses. Related techniques can be found in the works of Dick et ig. 3. Superposed line plots (1), a typical representation that has been used to analyze simulation results. Meaningful comparisons are hardlypossible, even with only five spinal discs displayed. The same data set can be visualized with our proposed tool (3). In this example, the main window(B) shows area charts of computed parameters over time for eight spinal discs. Each chart is associated with corresponding patient anatomy. Theblack line highlights the selected time step and shows the plot values. Options, such as spacing between plots, can be chosen from the control panel(A). The animation window (C) links the chosen point in time with the performed movement (2). al. [15–17], who employ color maps on 3D structures, illustrative stresstensor field visualizations and glyphs indicating object distances tofacilitate planning of surgical procedures. Possible approaches aresimulated, supporting clinicians in finding an optimal treatment op-tion through visual assessment of resulting attribute values. Depictingsimulation results often becomes more intricate when the data is time-dependent. The works of Doleisch et al. [18–20] address this problemin particular. They utilize focus and context techniques, where the userdefines a target domain in, e.g., statistical representations, which isthen visualized or highlighted in a linked 3D depiction of the simula-tion. Similar cases have been explored in the medical domain whensimulating blood flow [39, 40, 45, 46]. Glyphs, color maps, illustrativetechniques and integration of 3D models and statistical data representa-tion methods, such as plots and charts, are used to visualize a range ofparameters within the 3D domain. This means the anatomical structurecan be used to enable a direct link between attribute values and a pa-tient’s anatomy. For an overview of illustrative visualization techniquesand focus and context depictions, we refer to the surveys of Lawonn etal. [41–43].
Patient-specific anatomical features are commonly explored using cross-sectional imaging modalities, which in turn can serve as the basis forvolume renderings or segmented 3D models. The particular domainof spinal structure visualization, however, has only been sparsely ex-plored. Notable are the works of Klemm et al. [32–34], which evolvearound visual analysis of lumbar spine cohort data sets. They use asemi-automatic detection of the lumbar spine from volume images, re-sulting in 3D models that serve as a foundation for advanced processing.For instance, they can be used to visualize spinal canal variability incohort study data, allowing to draw associations between anatomy anddemographic or biological factors. Klemm et al. demonstrate how thiscan be achieved through clustered 3D streamlines and also geometricabstractions that only require a 2D representation [33]. Later, they gen-eralized their methods towards a visual analytics workflow, allowingepidemiologists to generate and validate hypotheses. The lumbar spinecohort visualization methods served as a demonstration [34]. Further,they showed how patient-specific properties can be measured and vi-sualized using geometric spine models. These can be used to analyzemutual dependencies between shape-describing parameters and other variables, allowing new insights into spine data sets [32].To the best of our knowledge, there have been no works specificallytargeting visualization of spine simulations. Common mechanicalsimulation tools [13, 22] generally offer a result representation basedon standard plotting, e.g., line or bar charts, which can be inadequatefor understanding complex systems like spinal anatomy.
EQUIREMENT A NALYSIS
We worked closely with domain experts when distilling requirementsfor our framework. Our intention is to provide a useful tool for bothspinal simulation researchers and medical practitioners, as we believemaking the output data intuitively understandable could benefit eitherand may prove useful to bridge the gap between technical research andclinical application. For this reason, we consulted an expert in eachfield. In the domain of biomechanical simulation we worked with aresearcher who has 15 years of experience in spine simulation. We thendiscussed possible directions with an orthopedic and neurosurgeon,specialized in spinal surgery with 36 years of experience.As a first step, we assessed the current workflows applied in biome-chanical spine simulation research. In patient-specific simulations,segmentations of vertebra geometry are created based on common med-ical volume imaging, e.g., computed tomography (CT). Pre-processingprepares the models for spine simulation, for instance, ligament at-tachments and origins can be detected and marked. Then, the model’sbiomechanical properties are simulated, which includes forces, dis-placements, and deformations. This results in attributes computedper simulation tick, i.e., smallest time increment, for each anatomicalstructure over a specified period of time. Until now, these results weredisplayed using line charts, for instance, as total force over time plotsfor selected structures. In current workflows, multiple structures arecompared by using superpositions of several line charts (Fig. 3 (1)).This only allows for a limited number of comparisons and quicklyresults in a cluttered view. Another problem we found in discussionwith physicians is that they have difficulties to grasp anatomical corre-spondence when presented with these kinds of representations. As soonas multiple line charts were displayed, they were hesitant when talkingabout the underlying anatomical structures and sometimes even referredto wrong ones. This not only made the clinical experts reluctant toimplement such a simulation system into clinical routine but could alsobe potentially harmful. ig. 4. The full pipeline. From medical volume images (1) vertebra models are segmented (2). Then, they undergo pre-processing where anatomicmarkers, e.g., origin and attachment of ligaments are detected (3). Based on this information the biomechanical simulation is performed (4). Inthis work, we show how the results can be visualized (5). We target to facilitate inclusion of patient-specific motion data, which is currently beingintegrated into the pipeline.
To provide an improved representation of the simulation results,tailored to the needs of researchers and clinicians working in thisdomain, we first narrowed down which simulation attribute values areof importance. Medically relevant are particularly force distributionson spinal discs and facet joins, as they are often the sources of chronicpain. This can be a result of unusually high or unbalanced forces, whichare mechanical conditions that can be appropriately simulated. Otherimportant clinical parameters are the resulting deformations of thespinal discs. These are typically measured using imaging techniquesand can be simulated through model stiffness, based on factors likedegree of degeneration and patient age. Visualizing these aspects mightcontribute to a better clinical analysis, for instance, when pathologicalcases can be more accurately identified and classified and adequatetreatment options can be reviewed or even simulated.In our discussions, we attempted to define the data analysis tasksresearchers and clinicians wish to perform and from these tasks speci-fied a number of requirements a visualization framework would needto fulfill. Both biomechanical researchers and physicians, desire anoverview of individual data sets, i.e., distribution of a selected attributeover multiple vertebrae. In addition to the simulation output, the expertsagreed that a display of the segmented geometry is of major importance,as data interpretation is highly dependent on the specific anatomicalfeatures. In order to identify imbalances, they want to directly compareforces acting on the left and right facet joints. Furthermore, it wouldmake sense to allow for comparison of multiple data sets. While physi-cians foremost intend to compare different simulated treatment options,biomechanical researchers would also like to contrast distinct sets ofinput parameters. The display of total forces is particularly significant,however, the experts found it promising to also incorporate force direc-tion on spinal discs, as vertical forces can be more easily compensated,while non-orthogonal or shear forces can lead to injury. Moreover, theexperts agreed that an exploration of the temporal dimension of thesimulation data is critical. A current research goal of biomechanicalspine simulation is to account for movement patterns of the patient.This is also an important clinical aspect, since spinal structures areessential for almost all daily motions and load distribution is dependenton the way movements are performed.While many potential exploration tasks seem similar between ourtarget groups, we also encountered some differences. When examiningvalues, physicians tend to be less interested in exact numerical outputand more in averages and spikes in data. Especially, they requireirregular force patterns to be easily discernible. For researchers, on theother hand, reading out exact values is a necessity, since the results ofdifferent models need to be compared quantitatively. They also need toquickly identify faulty or missing data. In a visualization framework,we believe these differences could be accounted for using optionalfeatures that can be chosen according to the task.The experts expressed specific requirements, when we discussed howa visualization framework should facilitate the gathered data explorationtasks. Especially for medical practitioners the relation between thedata and anatomical structures should be intuitive, e.g., the connectionbetween a force over time plot and the corresponding spinal disc should be clear. Both experts argued that intuitive comparisons are mostcrucial for similar structures, for example neighboring spinal discs orleft and right facet joints. To facilitate understanding of the temporaldimension, we found a direct connection with the movement that isactually performed to be a requirement. Last but not least, we concludedthat the important component of a force direction visualization wouldbe to encode how vertically forces are impacting each spinal disc orwhether shear forces are present. As many tasks and requirementsare aligned, we believe a framework targeting both user groups is apromising direction. Ultimately, it could also form a common basisfor communicating results. This is why we propose a visualizationframework to facilitate the following summarized requirements: R1 There should be a clear correspondence between simulation resultdata and patient anatomy. R2 Intuitive comparisons of result values should be possible acrosssimilar structures. R3 The displayed data is intrinsically time-dependent, requiring an ex-plicit connection between patient movement and attribute values. R4 The directions of forces impacting a patient’s spinal discs shouldbe accessible.
ETHODS
We integrated our visualization tools into a state-of-the-art pipeline forpatient-specific biomechanical spine simulation (Fig. 4).The pipeline involves an automatic segmentation process [1], fol-lowed by data pre-processing, and a multi-body-simulation [5, 8].Biomechanical researchers are currently incorporating patient-specificspinal motion data, captured with visible light techniques, into the sim-ulation model. This would allow a clinical analysis of both individualanatomy and movements, in a combined system. For the visualizationof the results, we attempt to integrate all components of this pipeline,which are relevant to data analysis. We, therefore, combine the numeri-cal results, spine model, and motion data into one framework. As wecannot utilize real motion data yet, we use artificial head movementsby adding external forces to the simulation. This demonstrates thefunctionality and allows us to evaluate the tool, making it useful tosimulation experts already.While our proposed system can be implemented with an arbitraryamount of vertebrae, we decided to use a model of the cervical spinefirst. In clinical procedures and analysis, as well as property modelling,often the cervical, thoracic, or lumbar spine (and the correspondingtransition areas) are focused. This allows for a more targeted point ofview. In the following, the used model consists of vertebrae C1-C7 andthe transition to Th1-Th3. Our methods should be similarly applicableto the thoracic or lumbar spine.
We output raw data from the simulation in form of large matrices, oneper observed value, e.g., force on spinal discs. Each row corresponds ig. 5. A 3D depiction allows free interaction with the patient’s spinalanatomy, while the stacked charts remain linked to their target structure.Fig. 6. Values computed on left and right facet joints are shown oneach side respectively. The left time axis is flipped to achieve a mirror-symmetry effect and improve comparability of the two sides. Note howthis patient appears to have an imbalance in force distribution across thefirst facets (C1-C4). with a simulation tick interval, while every column represents a struc-ture, for instance a particular disc. This means, we can extract a valuelike force or deformation as a function of time per focus structure.Fundamentally, multiple value-over-time plots can be displayedside-by-side (juxtaposition), within a common coordinate system (su-perposition), or through interchange of a selected plot. The latter isnot desirable, as we intend to address the comparability requirement( R2 ). This leaves two main options: using a single coordinate systemor one for each plot. A superposition in two dimensions, as it is usedin current workflows, is not suited to compare a high number of plots.A possibility would be to stack the charts in a third dimension. Thisis generally not an ideal method, as it requires direct scene interactionfrom the user and assessments may be inaccurate due to perspective dis-tortion. Still, we implemented this layout as an option, since it showedto have an advantage when addressing R1 : the anatomical context ofthe data can be displayed in the 3D domain, directly next to the corre-sponding charts (Fig. 5). The spine can be rendered in its anatomicallycorrect state and freely rotated, while correspondence to the simulationresult data remain clear. The domain experts found this representationhelpful to gain a first impression of the combined data and anatomy. Toaddress the shortcomings of this depiction, e.g., possible perspectivedistortion, we implemented the second option as the default view toexplore the result values. We use juxtaposed charts, aligned on a sharedtime-axis, which we display in 2D (Fig. 3 (3)). Comparing area andline charts, it quickly became apparent that area charts are better suited,as they intuitively convey value dimensions, even with many charts inone scene. We can still keep the anatomical context, by aligning the Fig. 7. This data set shows facet force distributions following a lateralhead bend. It is directly compared against a static simulation withoutmovement (gray area plot). It can be seen how tilting the head results ina decisively higher load on one side of the vertebrae. charts beside the vertically drawn spine geometry and keeping data-points right next to their respective anatomical structure. The drawbackof this representation is that the geometry cannot be interacted withfreely, without losing correspondence. Another challenge is that thedata may encompass a wide range of values, with some plots fillingtheir coordinate systems while others remain close to zero. To enablecomparison, all axes need to be equally scaled, but a low overall scalemay impact assessment of small plots. We solve this problem by givingthe user the option to adjust the distance between plots, i.e., the lengthof the plots’ value-axes. In order not to lose anatomical context, weexpand the spine geometry accordingly by pulling the vertebrae apart,i.e., translating them along the vertical axis.We determined that users need to be able to read out quantitativevalues and also intuitively understand general value ranges. There areseveral ways to achieve this, with the simplest being to use labelledvalue axes for each plot with, e.g., gridlines. We tested this option butdecided to leave it disabled per default. With around ten juxtaposedcharts the scene becomes cluttered and values are hard to read. Instead,we opted for a combination of colormapping and point selection. Weemploy a consistent colormap of data values across all views, whichtargets to visualize the approximate value ranges and help with generalassessments. We use the viridis colormap, as its colors are perceptuallyuniform and can also be perceived by most forms of color blindness [44,59]. To see quantitative values for each chart in the scene, the usermay select a point in time of the simulation via a continuous slider (cf.Fig. 3 (B)).
Spine simulations contain various structures of medical interest. Spinaldiscs are a typical example, but as described above, facet joints canbe similarly important when evaluating stress distributions. In ourproposed layout, spinal disc data can be shown in the standard way ofright-facing plots. Facet joints, however, pose an additional challenge,as two facets exist between each vertebra pair. For values correspondingto facets, we therefore display right- and left-facing plots, with thepatient’s spine acting as a central vertical axis (Fig. 6). Our main ig. 8. The simplified representation relies on a color-only encoding and facilitates comparison of simulation results over multiple data sets. Here,we show how this could be utilized to compare different input parameters. The same motion was used to simulate spinal disc deformations with anincreasing degeneration degree from (1) to (5). The discretized colormap was added in a later iteration and allows to quickly discern value ranges.For instance, discs with degrees (1) to (3) appear to deform less than 2 mm. goal regarding facet data is to facilitate left-right comparisons, i.e., tovisualize whether a load is equally distributed on both sides. To enhanceperceptual recognition of differences between the two sides, we applythe Gestalt Principle of Symmetry [23]. Use of symmetry has beenshown to improve readability and understanding, for instance in graphlayouts [10, 58]. Thus, we create a reflective symmetry between valueson the left and right side, by mirroring the time-axis of the left-handdata charts. This results in a view, in which for all charts early pointsin time are closer to the central axis (where the vertebrae are rendered)and later points in time are further away.
Up until now, the time-dependency of the load distribution can beobserved in direction of the time-axis. To connect these results withthe underlying movement ( R3 ), we show an animation of the model’srotations and translations, which can be derived from the simulationin the same format as the value matrices. As the already displayedvertebrae models serve as anatomical references, using them in ananimation would result in a tangled view without clear correspondences.Therefore, we show the animation in an additional smaller window (asin Fig. 3 (C)), which is also interactable, in case different viewingangles are desired. If animation data was generated, the time pointselection is automatically linked to the respective animation time step.This is indicated in the main window by the black selection line, whichalso shows the individual plot values (cf. Fig. 3 (B)). In some application scenarios it might be desirous to perform compar-isons across multiple data sets or ensembles. For instance, in simulationresearch, this applies to contrasting different initial configurations orparameter sets. In medical practice, the results of possible treatment op-tions or implant types on force distributions are ideally directly compa-rable. Data set ensembles may also arise from cohort studies evaluatingif anatomical factors contribute to the manifestation of certain patholo-gies. Even simple tasks, like the comparison of two movement patterns,require more than one data set to be shown in the scene. Especially thelatter case can be covered by rendering a second simulation outcomewithin the existing coordinate systems. In our tool, such a data set canbe displayed in gray on top of the original area charts (Fig. 7). In caseof occlusion, i.e., when the comparison plot value is higher, the originalchart is shown as a line within the gray area.For ensembles we need to apply a more scalable solution, whichwe will call a simplified view. To facilitate coherence, our approachretains the general layout, i.e., the spine anatomy stays as a contextualreference in the center and the value-over-time graphs are presentedtowards the left (and right if necessary). As the number of charts goesup, the positional encoding of area plots is increasingly hard to read.Therefore, we reduce the data value encoding to color only. This allowsfor a quick overview and comparison of many data sets (Fig. 8).
Standard charts allow an interpretation of a total value or its compo-nents over time. For instance, a force vector’s length or one of its x , y , z -components can be interpreted. This makes it effectively impos-sible to understand from which spatial direction a force is impactingan anatomical structure. Since we render a 3D representation of thepatient’s spine already, we propose the integration of markers showingthe direction of forces within the scene to address R4 .We display arrow glyphs as a simple and intuitive shape to representforce vectors. Each glyph targets the barycenter of its focus structure,e.g., spinal disc model, akin to the internal representation of forces inthe multi-body-simulation. This is also in accordance with the feature-driven glyph placement approach typical for medical applications [11].The arrow direction is extracted according to the selected point intime. An option might be to scale the arrow glyphs proportional totheir total force, but we propose to keep their length uniform, as thisleaves less ambiguity w.r.t. their spatial direction. Even then, thedirection of arrow glyphs is difficult to interpret in 3D and dependson the viewing angle. Consulting with our domain experts, we foundthe most important aspect to be a clear indication of whether shearforces are present, i.e., if force vectors are impacting spinal discs morefrom the sides than above. Therefore, to make the arrow orientationsbetter comparable to the respective spinal disc, we propose to add anorthogonal disc located at the arrow tip (Fig 9). It can be thought of asa “force plane”, with the impact vector being its normal. This planarcomponent simplifies interpretation of the direction and gives spatialcues, even when little to no 3D interaction is used. The orientation ofthe glyph disc can be visually compared against the respective spinaldisc, giving an impression of how vertically the load is distributedat the selected time interval. To avoid visual clutter, these glyphsare only shown when the vertebrae models are pulled apart (throughthe axis scaling adjustment) and disappear when the vertebrae arecondensed towards their anatomically correct “stacked” positions. Allglyph components are updated according to the time-point selection,resulting in an interactive animation of force directions (Fig. 10).One obstacle to keep in mind is that occurring forces in a simulationare possibly not measured in the local coordinate system of the targetstructure. In our case, each force vector f is extracted in global coordi-nates, while the target object’s position is determined by some rotationmatrix φ , followed by a translation. As we display the direction glyphsin a model with fixed orientation, we need to correct the force vector’sorientation by ˆ f = φ T f , before positioning the glyph in the scene. VALUATION
During development of our methods we iterated several feedback cycleswith domain experts. We will now describe the main evaluation of ourinitial prototype. This evaluation was particularly beneficial, as ityielded some relevant design decisions.We conducted individual interviews with six domain experts (1 fe-male, 5 male; 26-63 years old; median 41 years). The expert groupcan be divided into three biomedical simulation researchers (5, 2 and 3years of experience) and three physicians specialized in spine surgery ig. 9. Glyphs encoding the direction of forces impacting the spinaldiscs. They become visible through expansion of the spine geometry,which is simplified to a silhouette representation. The selected pointin time is marked with an arrow and a black disc in the “force plane”,i.e., orthogonal to the impact direction. After the evaluation, we alsoadded isolines on the spinal disc surface, which are rendered parallel tothis plane, allowing a better interpretation of how vertically the load isdistributed on the spinal discs. (16, 20 and 36 years of experience as orthopedics or neurosurgeons).None of the experts had used the proposed tool before. Also, they werenot part of the preceding discussions, except for one of the interviewedsurgeons, who had also been present for the original requirement analy-sis. Participants were introduced to the concept of the full acquisition,simulation and visualization pipeline. We showed them examples ofsuperposed result plots and explained how these are currently used tointerpret the simulation output. Then, they were presented with thesame data sets in our framework and we introduced them to all features,while they were able to freely explore the data. We encouraged partici-pants to think aloud while they interacted with the tool and noted downtheir spoken comments and suggestions, particularly any difficultiesthey encountered. In an attempt to recreate scenarios that might ariseduring typical use of the tool, we integrated a number of tasks in thisprocess. Participants wrote their answers on paper, while interactingwith all data representations on a single-screen laptop setup.The first task targeted simple readouts of values at predeterminedpoints in time of the simulation. This is a common task for simu-lation researchers, who need to perform quantitative measurementsand comparisons. For each value, participants were to indicate howconfident they felt with their statement on a scale from 0% (very un-certain) to 100% (highly certain). To compare our tools with formerrepresentations, participants conducted the task twice, once using theold representation and once with our visualization framework, i.e., as inFig. 3 (1) and (3). For comparison, we used the same data set, however,we specified different time points to avoid a learning effect. We alsoswitched the order of methods for each new participant, i.e., whetherthe old representation or our framework was used first.The second task was conducted similarly, but focused on generalimpressions and assessments that participants were able to draw fromthe shown data. We matched our questions to theoretical explorationgoals of clinicians, which we gathered during the requirement analysis.For instance, we asked participants whether they found the displayeddisc deformation value for a number of determined spinal discs to behigher or lower on average, as compared to the rest. We also used facetdata sets, where for each facet pair between two vertebrae participantswere to decide if load distribution was skewed to the patient’s left orright side or whether it was approximately equally balanced. They
Table 1. Results of the first and second task, comparing the formerrepresentation style (old) against our proposed exploration framework(new). Values are averaged over physicians and biomechanical simula-tion experts, as well as all participants (total).
Task 1 Mode Total Physicians ExpertsAverage error old 2.55N 5.12N 0.2Nnew 0.19N 0.4N 0.03NSubjective certainty old 52.8% 36.1% 69.4%new 97.2% 94.4% 100.0%Time old 6m 4s 6m 19s 5m 50snew 3m 31s 3m 41s 3m 21sTask 2Correct assessments old 66.7% 38.9% 94.4%new 100.0% 100.0% 100.0%Subjective certainty old 45.8% 16.7% 75.0%new 95.8% 91.7% 100.0%Time old 3m 9s 2m 49s 3m 28snew 1m 22s 1m 33s 1m 11scould also note down missing data sets (we removed data from onefacet joint) and mark their certainty, as before. Again, the participantsperformed the task twice, once with the new and once with the old datarepresentation.To determine the effectiveness of the directional glyphs, we askedthe participants to describe the orientation of force directions for severalspecified spinal discs. We inquired how difficult they found these kindsof assessments and whether they believed directional encodings to be auseful addition to the tool.After participants had used the tools during introduction and execu-tion of tasks, we let them answer a questionnaire, where we asked howwell they perceived correspondence with anatomical structures ( R1 ),comparability of result data ( R2 ), selection of points in time ( R3 ) andthe depiction of force directions ( R4 ). We associated each requirementwith four to six statements the participants were to rate on a five-pointLikert scale ( −− , − , ◦ , + , ++ ). We concluded with a discussion aboutpossible use cases for the proposed tool, to see, if the experts’ opinionswould match our designated application scenarios. This was the onlypoint in the evaluation where we distinguished between technical ex-perts and medical practitioners, as their typical application domainswould naturally differ. ESULTS AND D ISCUSSION
All participants quickly understood most of the framework’s function-ality and were able to explore datasets without requiring assistance.Unanimously, they deemed the tool a valuable asset they would liketo use for analyzing spine simulations. In the following, we describesome selected insights gained from discussion with the domain experts.Additionally, we performed some measurements to compare the oldrepresentation style with our new tool and to validate participants’ im-pressions. These results do not represent a full quantitative study, butare meant to complement some findings of our qualitative interviews.They are summarized in Table 1. An overview of the questionnaireresults is shown in Fig. 11.
Anatomical correspondence.
Domain experts agreed that the visu-alization helps to foster a correspondence between simulation resultsand patient anatomy. Especially the physicians pointed out that theclear connection between data and anatomy felt more intuitive to themthan in the old representation. They noted that this made them moreinclined to actually use biomechanical simulation in practice. We canaffirm these impressions when looking at the average error (in NewtonN) participants made when reading out values in the first task (see Ta-ble 1). Even though both representations allow for quantitative readoutsof results, errors occur when the wrong plot is selected. This observa-tion was even more evident during the second task: Physicians drewerroneous conclusions in more than half of the cases, when data waspresented in the old format. However, all six experts identified everysingle case correctly, when they were using our tool. The most probablereason, why medical practicioners fared comparatively worse, is that ig. 10. The directional glyphs are animated according to the time-selection. In this example, the impact originates from the top-left side (0s), thenmoves to a vertical direction (1s) and lastly results in a left-frontal shear force (3s). After the evaluation we improved the representation by drawingthe trajectory of the arrow glyph as a fading surface. This makes the jittering of the impact direction at around 2s visible, even without an activeanimation. our technical experts were already used to the old representation formatand thus had an advantage. The errors physicians made were almost alldue to them inadvertently reading out the wrong plot or misinterpretingthe scaling of an axis. This disparity in accuracy (and also subjectivecertainty) could likely be overcome through training. However, a directconnection between data and anatomy allowed physicians to imme-diately explore results, without having noticeable problems. In thequestionnaire, participants rated our representation of patient-specificvertebrae to be effective in facilitating intuitive assignments of datato structures (S( ++ ) = 5, S( + ) = 1). They also found it easy to iden-tify the type of displayed data, i.e., what the target structures are andwhich parameter is shown (S( ++ ) = 6). This corresponds to our firstrequirement ( R1 ). Comparisons.
All experts noted how our tool explicitly helped tocompare result values of multiple spinal discs or facets. They foundthe area charts to be comparable at a glance (S( ++ ) = 4, S( + ) = 2) andmost rated the color map to be helpful to find data points with highloads (S( ++ ) = 4, S( + ) = 1, S( ◦ ) = 1). Every domain expert determinedthe two-sided plots to be highly useful when assessing facet parameters(S( ++ ) = 6). Many stressed this point in particular already whenperforming the tasks. Participants further confirmed that they coulddirectly identify missing or faulty data (S( ++ ) = 6). We can affirmthis, as during the tasks one simulation expert and one clinician couldnot identify a missing facet data set when using the old representation,even when encouraged to do so. However, all participants immediatelypointed out this error when using the visualization. Animation.
The connection between movement and result valueswas also perceived well. Participants said they found the real-timeanimation we display in a corner to be highly effective in this regard(S( ++ ) = 6). They could quickly select a sought time step (S( ++ ) =4, S( + ) = 2) and they felt it was obvious to understand which point intime was currently active (S( ++ ) = 6). Directional glyphs.
While overall responses were generally highlypositive, we found participants to have most difficulties in the thirdtask, when using the impact direction glyphs. Some experts expresseda need to further familiarize themselves with the depiction, before theycould accurately use it. Still, most found the direction of acting forceson a spinal disc to be assessable (S( ++ ) = 1, S( + ) = 4, S( ◦ ) = 1) andstated that they could determine the orientation of load distributionacross several discs (S( ++ ) = 2, S( + ) = 2, S( ◦ ) = 1). We also observedthat experts usually interpreted the direction correctly, however, manyrequired 3D scene interaction to do so. One physician mentioned thatit would be advantageous if shear forces could be spotted more easily.Note that at this point some features were not yet implemented, suchas the isolines we now draw onto the spinal disc surfaces. An aspectto take into account is that glyphs are different to methods currentlyused in practice. In a real scenario, experts would therefore requiremore time to learn how a visualized force distribution appears in aphysiological versus pathological case. This means that simulationexperts may already utilize such representations, for instance, to see ifforces behave as expected and to detect errors. For clinical use, it would be necessary to further explore what force direction characteristicsdifferent pathological cases exhibit, as compared to normal spines.Then, it may be evaluated if such a depiction can be used to make betterassessments and facilitate understanding of mechanical causes w.r.t.spine pathologies. Simplified view.
Experts mostly found the simplified depiction toprovide a faster overview (S( ++ ) = 1, S( + ) = 5). However, only someof them used this view during exploration. We discussed probablereasons with the experts, who noted that it would be easier to under-stand data magnitudes when using the positional encoding, since theheight differences of plots seemed more natural for them to compareas when they had to rely on color only. Nonetheless, the simplifiedview has a definitive advantage regarding required screen space. Evensmall depictions, as in Fig. 8, remain readable. Therefore, we came tothe consensus that this representation would be particularly suited tocompare multiple data sets. Future evaluations should thus target thisuse case explicitly. Application scenarios.
When discussing possible application sce-narios for the proposed visualizations, all physicians pointed out thepossibility of treatment evaluation through comparison of data col-lected pre- and post-surgery. They could also imagine the simulation ofdifferent treatment options, e.g., contrasting different possibilities forimplants and analyzing resulting loads on critical structures. Some pro-posed supplementing traditional diagnostics by examining simulatedloads on a patient’s spine and the possibility of supporting physician-patient communication through intuitive data representations. Biome-chanical simulation experts deemed the tool to be highly promising forevaluating result data, identifying computation errors and visually com-paring effects of different simulation parameters. They also suggestedthe use in interdisciplinary communication of simulation results. Thesescenarios are reflective of what we intended the framework to be usedfor.
MPROVEMENTS
Based on the evaluation we revised and modified some of the employedencodings. These improvements were subsequently discussed withthe domain experts we collaborated with for the initial requirementanalysis.
Directional glyphs.
As participants appeared to require 3D sceneinteraction in order to read the directional glyphs, we adapted the rep-resentation. When the glyphs are rendered, we reduce the vertebraegeometry to its silhouette projection, in order to visually guide focustowards the directional glyphs. We debated adding some form of en-coding showing deviation of force directions from a spinal centerline,similar to the ideas of Klemm et al. [32–34]. The problem we encoun-tered is that a centerline would, again, be somewhat dependent on theviewing angle. From a frontal (coronal) direction, deviations towardsthe left and right could be shown. However, deviations towards thefront or back would require another perspective. Instead, we proposeprojecting isolines in parallel direction to the force plane onto the spinaldisc model. These lines make vertical forces immediately discernible olors for Likert score: − − − ◦ + + +Anatomical correspondence: Use of patient anatomyData type recognizabilityAssignment of plot to structure
Comparability:
Identification of missing dataPlots are comparable at a glanceSpacing adjustment is helpfulColormap shows value rangesLeft-right comparability (facets)Simplification enables overview
Temporal aspects:
Selection of desired timestepMovement animationQuantitative readoutsPoint in time unambiguous
Directional glyphs:
Orientation readableOrthogonality of force visibleDistribution of forces over time
Physicians SimulationExperts
Fig. 11. Results of the questionnaire with color-encoded Likert scores.Each box shows the answer of one participant. from shear forces (cf. Fig. 9, 10). Additionally they can be displayedeven if the spine is in its original state. We believe that with some expe-rience a domain expert would already be able to hint at force directionsby interpreting the isolines, without having to artificially increase thedistance between vertebra, in order to see the full directional glyphs.We deliberately use arrows, discs and isolines as redundant mappingsof spatial direction. We found their combination to reduce the risk ofinformation loss and make the glyphs comprehensible from a staticpoint of view, as demonstrated in Fig. 10. One simulation expert statedthat it might be useful if the amount of jitter in force direction could bevisualized, as this would be a valuable indicator of simulation stability.To emphasize the glyph movements, we render the arrows’ trajectoryin form of a traced surface geometry, which fades away at selection oflater time-points. A jittering force direction can be immediately spotted,as it results in a wide surface. Impact directions, which are stable overtime, produce only a small surface trace.The additions were positively received by the experts, who foundthe difference between shear forces and vertical impacts to be readablemore clearly. They also agreed that the isolines are a helpful extensionto understand the directions even in a “compressed” model of the spine,where the full glyphs cannot be shown.
Simplified view.
Realizing the potential of the simplified view modefor ensemble and multi-set visualizations, we emphasized this use-caseby removing detail from the scene when the user intends to comparemore than two data sets. Individual values are probably less importantin this scenario, so we disabled the output of quantified values at theselected time-step per default. Also, the vertebral geometry is de-emphasized by using a flat projection. Additionally, we found subtlechanges in the color gradient to be hard to discern in these downscaledviews, which is why we employed a discretized color scheme. Thevalue range of the color map and the discretization level is configurableby the user. The experts acknowledged that the discretization helps tobetter distinguish values in small scale depictions. They affirmed thatthis also enables to spot where certain thresholds are exceeded, which is helpful to simulation experts and clinicians alike.
ONCLUSION AND F UTURE W ORK
In this design study, we presented a framework for visual explorationof human spine biomechanics. The visualizations target intuitive rep-resentations of time-dependent parameters, which are simulated usingpatient-specific anatomy. We aim to support simulation researchers inunderstanding computed data and make clinically relevant propertiesaccessible to spinal surgeons. We proposed a combination of interac-tive charts, glyphs and a simplified representation to make simulationresults explorable in depth, as well as to enhance data overview andcomparison.An evaluation with six domain experts showed that our tool haspotential to complement research in biomechanical spine simulationand may provide a way to introduce simulations into medical practice.We suggested novel glyph-based depictions of spatial force distributionsthat were not apparent from the data before. The visualizations solelyrely on data that is already used in the simulation pipeline: the rawoutput data, the segmented vertebrae models, and (optional) movementpatterns. This makes our methods generalizable regarding additionalspinal parameters and means they could also be used within differentspine simulation workflows.We believe that some of our insights could be transferred to otherareas as well. In particular, this applies to the concept of connecting datawith anatomy in the field of medical visualization, which appears to be astrong motivator for clinicians to adapt new encodings, as they can moreintuitively understand them. Embedding of abstract data representationsin a context familiar to the user allows for a structural interpretation.This is a research direction that we believe still holds potential, not onlyin the medical domain. Making sense of large amounts of data, acquiredthrough sensors or simulation, can be facilitated through a combinationof techniques from information and scientific visualization, resulting inhybrid representations, as they are shown in this work. A challenge isthe combined display of data with differing dimensions, e.g., 2D graphsin a 3D scene. A possible solution shown here is to fix 3D viewpointsand to manipulate or deform geometry to enforce data alignment. Toimprove readability of such visualizations, we found the followingprinciples to be particularly useful: colormaps for data overview andabstraction, focus and context to adapt to specific exploration tasks, anduse of symmetry to foster intuitive comparison if two sides are naturallygiven. Also, the use of embedded glyphs to visualize impact directionscould be transferred to more types of mechanical or biomechanicalsimulations, where researchers would benefit from getting a betterunderstanding of force vector orientations.In the future, we would like to evaluate an expansion of our methodsw.r.t. to full spines, as compared to the cervical spine examples weused in this work. Further, we would like to incorporate additionalinformation into the animation window. Computing and displaying im-portant structural relations, angles and degrees of freedom could makemultiple data sets quantifiably comparable regarding motion data. Forinstance, this would allow to contrast resulting forces on spinal discs independency of a defined bending angle. Also, we would like to exploreadditional methods for a combined overview of spine geometry andsimulation results, in order to replace the potentially flawed depictionof stacked graphs. As mentioned before, we currently simulate patientmovements through external forces as a proof-of-concept. While ourresults should scale well with movement data from visible light scans,which is currently being integrated into the pipeline, we would like totest patient-specific motion data with the proposed tools once this be-comes an option. This way, we aim to advance personalized medicine,by simulating accurate patient spine geometry in combination withindividual movement patterns and enabling physicians to intuitivelyexplore the resulting data. A CKNOWLEDGMENTS
This work was partially funded by the Carl Zeiss Foundation and theFederal Ministry for Economic Affairs and Energy of Germany.
EFERENCES [1] I. Al-Dhamari, S. Bauer, and D. Paulus. Automatic multi-modal cervicalspine image atlas segmentation. In A. Maier, T. M. Deserno, H. Handels,K. H. Maier-Hein, C. Palm, and T. Tolxdorff, eds.,
Bildverarbeitung f¨urdie Medizin , pp. 303–308. Springer, 2018. doi: 10.1007/978-3-662-56537-7 80[2] G. B. Andersson. Epidemiological features of chronic low-back pain.
Lancet , 354(9178):581–585, 1999.[3] M. Ay, T. Kubat, C. Delilbasi, B. Ekici, H. E. Yuzbasioglu, and S. Har-tomacioglu. 3D Bio-CAD modeling of human mandible and fabricationby rapid-prototyping technology.
Usak University Journal of MaterialSciences , 2:135–145, 2013. doi: 10.12748/uujms.201324255[4] S. Bauer. Basics of multibody systems: Presented by practical simulationexamples of spine models. In R. Lopez-Ruiz, ed.,
Numerical Simulation -From Brain Imaging to Turbulent Flows . IntechOpen, London, 2016. doi:10.5772/62864[5] S. Bauer and U. Buchholz. Biomechanical effects of spinal fusion toadjacent vertebral segments. In , pp.158–163. IEEE, 2013. doi: 10.1109/EMS.2013.28[6] S. Bauer and D. Paulus. Analysis of the biomechanical effects of spinalfusion to adjacent vertebral segments of the lumbar spine using multi bodysimulation.
International Journal of Simulation-Systems, Science andTechnology , 15, 2015.[7] S. Bauer and D. Paulus. How does the intervertebral discs parametervariation affect the biomechanical behavior of spinal structures? resultsof a detailed study of multibody simulation sensitivity.
InternationalJournal of Engineering and Applied Sciences , 2(9):37–42, 2015. doi: 10.1515/cdbme-2015-0092[8] S. Bauer and D. Paulus. Computational simulation as an innovative ap-proach in personalized medicine. In J. Bettany-Saltikov and S. Schreiber,eds.,
Computational Simulation as an Innovative Approach in Personal-ized Medicine, Innovations in Spinal Deformities and Postural Disorders .IntechOpen, London, 2017. doi: 10.5772/intechopen.68835[9] S. Bauer, C. Wasserhess, and D. Paulus. Quantification of loads on thelumbar spine of children with different body weight a comparative studywith the help of computer modelling.
Biomedical Engineering , 59:910–1027, 2014. doi: 10.1515/bmt-2014-5012[10] C. Bennett, J. Ryall, L. Spalteholz, and A. Gooch. The aesthetics of graphvisualization.
Computational Aesthetics , 2007:57–64, 2007.[11] R. Borgo, J. Kehrer, D. H. Chung, E. Maguire, and et al. Glyph-basedvisualization: Foundations, design guidelines, techniques and applications.
Eurographics , pp. 39–63, 2013.[12] J. M. Cu´ellar and T. H. Lanman. text neck: an epidemic of the modern eraof cell phones?
The Spine Journal , 17(6):901–902, 2017. doi: 10.1016/j.spinee.2017.03.009[13] S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, A. Habib, C. T. John,E. Guendelman, and D. G. Thelen. Opensim: open-source software tocreate and analyze dynamic simulations of movement.
IEEE Transactionson Biomedical Engineering , 54(11):1940–1950, 2007.[14] I. Demir, C. Dick, and R. Westermann. Multi-charts for comparative 3Densemble visualization.
IEEE Trans Vis Comput Graph , 20(12):2694–2703, 2014.[15] C. Dick, R. Burgkart, and R. Westermann. Distance visualization for inter-active 3D implant planning.
IEEE Trans Vis Comput Graph , 17(12):2173–2182, 2011.[16] C. Dick, J. Georgii, R. Burgkart, and R. Westermann. A 3D simulationsystem for hip joint replacement planning. In
Proceedings of WorldCongress on Medical Physics and Biomedical Engineering 2009 , vol.25/IV of
IFMBE Proceedings , pp. 363–366, 2009.[17] C. Dick, J. Georgii, R. Burgkart, and R. Westermann. Stress tensor fieldvisualization for implant planning in orthopedics.
IEEE Trans Vis ComputGraph , 15(6):1399–1406, 2009.[18] H. Doleisch. Simvis: Interactive visual analysis of large and time-dependent 3D simulation data. In
Winter Simulation Conference , pp.712–20, 2007.[19] H. Doleisch, M. Gasser, and H. Hauser. Interactive feature specificationfor focus+context visualization of complex simulation data. In
Proc. ofIEEE/EG VisSym , pp. 239–48, 2003.[20] H. Doleisch, M. Mayer, M. Gasser, R. Wanker, and H. Hauser. Case study:Visual analysis of complex, time-dependent simulation results of a dieselexhaust system. In
Proc. of IEEE/EG VisSym , pp. 91–6, 2004.[21] P. Eulzer, S. Engelhardt, N. Lichtenberg, R. De Simone, and K. Lawonn. Temporal views of flattened mitral valve geometries.
IEEE Trans VisComput Graph , 26(1):971–980, 2019.[22] P. A. Fishwick. Simpack: getting started with simulation programming inc and c++. In
Winter Simulation Conference , pp. 154–162, 1992.[23] J. Freyd and B. Tversky. Force of symmetry in form perception.
TheAmerican Journal of Psychology , pp. 109–126, 1984.[24] R. J. Gatchel. The continuing and growing epidemic of chronic low backpain.
Healthcare , 3(3):838–845, 2015. doi: 10.3390/healthcare3030838[25] G. Ghiselli, J. C. Wang, N. N. Bhatia, W. K. Hsu, and E. G. Dawson.Adjacent segment degeneration in the lumbar spine.
JBJS , 86(7):1497–1503, 2004.[26] D. Grob, H. Frauenfelder, and A. Mannion. The association between cervi-cal spine curvature and neck pain.
European Spine Journal , 16(5):669–678,2007. doi: 10.1007/s00586-006-0254-1[27] W. S. Group. The burden of musculoskeletal conditions at the start of thenew millennium. Technical Report 919, World Health Organization, 2003.[28] M. Hermann, A. C. Schunke, T. Schultz, and R. Klein. Accurate interactivevisualization of large deformations and variability in biomedical imageensembles.
IEEE Trans Vis Comput Graph , 22(1):708–717, 2016.[29] I. Heuch, I. Heuch, K. Hagen, and J.-A. Zwart. Body mass index as a riskfactor for developing chronic low back pain.
Spine , 38(2):133–139, 2013.doi: 10.1097/BRS.0b013e3182647af2[30] A. Kettler, F. Rohlmann, C. Neidlinger-Wilke, K. Werner, L. Claes, andH.-J. Wilke. Validity and interobserver agreement of a new radiographicgrading system for intervertebral disc degeneration: Part ii. cervical spine.
European Spine Journal , 15(6):732–741, 2006. doi: 10.1007/s00586-005-1037-9[31] K. Kim, J. V. Carlis, and D. F. Keefe. Comparison techniques utilizedin spatial 3D and 4D data visualizations: A survey and future directions.
Computers & Graphics , 67:138–147, 2017.[32] P. Klemm, S. Glaßer, K. Lawonn, M. Rak, H. V¨olzke, K. Hegenscheid,and B. Preim. Interactive visual analysis of lumbar back pain. In
Proc. ofthe 6th International Conference on Information Visualization Theory andApplications , pp. 85–92. Berlin, 2015.[33] P. Klemm, K. Lawonn, M. Rak, B. Preim, K. T¨onnies, K. Hegenscheid,H. V¨olzke, and S. Oeltze. Visualization and analysis of lumbar spine canalvariability in cohort study data. In Michael Bronstein, Jean Favre, and KaiHormann, ed.,
VMV 2013 - Vision, Modeling, Visualization , pp. 121–128.Lugano, 2013.[34] P. Klemm, S. Oeltze-Jafra, K. Lawonn, K. Hegenscheid, H. V¨olzke, andB. Preim. Interactive visual analysis of image-centric cohort study data.
IEEE Trans Vis Comput Graph , pp. 1673–1682, 2014. doi: 10.1109/TVCG.2014.2346591[35] Z. Konyha, A. Leˇz, K. Matkovi´c, M. Jelovi´c, and H. Hauser. Interactivevisual analysis of families of curves using data aggregation and derivation.In
Proc. of the Conference on Knowledge Management and KnowledgeTechnologies , p. 24, 2012.[36] P. R. Krekel, C. P. Botha, E. R. Valstar, P. W. de Bruin, P. M. Rozing, andF. H. Post. Interactive simulation and comparative visualisation of thebone-determined range of motion of the human shoulder. In
SimVis , pp.275–288, 2006.[37] P. R. Krekel, E. R. Valstar, J. H. D. Groot, F. H. Post, R. G. H. H. Nelissen,and C. P. Botha. Visual analysis of multi-joint kinematic data.
ComputGraph Forum , 29:1123–1132, 2010.[38] M. Kumar, A. Baklanov, and D. Chopin. Correlation between sagittal planechanges and adjacent segment degeneration following lumbar spine fusion.
European Spine Journal , 10(4):314–319, 2001. doi: 10.1007/s005860000[39] K. Lawonn, R. Gasteiger, and B. Preim. Adaptive surface visualizationof vessels with animated blood flow.
Comput Graph Forum , 33(8):16–27,2014.[40] K. Lawonn, S. Glaßer, A. Vilanova, B. Preim, and T. Isenberg. Occlusion-free blood flow animation with wall thickness visualization.
IEEE TransVis Comput Graph , 22 (1)(1):728–737, 2015.[41] K. Lawonn and B. Preim.
Feature Lines for Illustrating Medical SurfaceModels: Mathematical Background and Survey , chap. Visualization inMedicine in Life Sciences III, pp. 93–132. Springer Verlag, 2016.[42] K. Lawonn, N. N. Smit, K. Bhler, and B. Preim. A survey on multimodalmedical data visualization.
Comput Graph Forum , 37(1):413–438, 2018.doi: 10.1111/cgf.13306[43] K. Lawonn, I. Viola, B. Preim, and T. Isenberg. A survey of surface-basedillustrative rendering for visualization.
Comput Graph Forum , 37(6):205–234, 2018.[44] Y. Liu and J. Heer. Somewhere over the rainbow: An empirical assessmentf quantitative colormaps. In
Proceedings of the 2018 CHI Conference onHuman Factors in Computing Systems , pp. 1–12, 2018.[45] M. Meuschke, S. Voß, O. Beuing, B. Preim, and K. Lawonn. Com-bined visualization of vessel deformation and hemodynamics in cerebralaneurysms.
IEEE Trans Vis Comput Graph , 23(1):761–770, 2017.[46] M. Meuschke, S. Voß, O. Beuing, B. Preim, and K. Lawonn. Glyph-basedcomparative stress tensor visualization in cerebral aneurysms.
ComputGraph Forum , 36(3):99–108, June 2017. doi: 10.1111/cgf.13171[47] P. H. Mikkonen, J. Laitinen, J. Remes, T. Tammelin, S. Taimela, K. Kaikko-nen, P. Zitting, R. Korpelainen, and J. Karppinen. Association betweenoverweight and low back pain: a population-based prospective cohortstudy of adolescents.
Spine , 38(12):1026–1033, 2013. doi: 10.1093/aje/kwp356[48] M. Panjabi and A. White. Clinical biomechanics of the spine.
Kinematicsof the Spine , pp. 85–127, 1990.[49] K. Potter, A. Wilson, P.-T. Bremer, D. Williams, C. Doutriaux, V. Pas-cucci, and C. R. Johnson. Ensemble-vis: A framework for the statisticalvisualization of ensemble data. In
IEEE International Conference on DataMining Workshops , pp. 233–40, 2009.[50] R. G. Raidou, O. Casares-Magaz, A. Amirkhanov, V. Moiseenko, L. P.Muren, J. P. Einck, A. Vilanova, and M. E. Gr¨oller. Bladder runner: Visualanalytics for the exploration of rt-induced bladder toxicity in a cohortstudy. In
Comput Graph Forum , vol. 37, pp. 205–216, 2018.[51] S. S. Rajaee, H. W. Bae, L. E. Kanim, and R. B. Delamarter. Spinalfusion in the united states: Analysis of trends from 1998 to 2008.
Spine ,37(1):67–76, 2012. doi: 10.1097/BRS.0b013e31820cccfb[52] F. Rengier, A. Mehndiratta, H. Von Tengg-Kobligk, C. M. Zechmann,R. Unterhinninghofen, H.-U. Kauczor, and F. L. Giesel. 3D printing basedon imaging data: review of medical applications.
International Journal ofComputer Assisted Radiology and Surgery , 5(4):335–341, 2010. doi: 10.1007/s11548-010-0476-x 2010[53] A. Rohlmann, J. Calisse, G. Bergmann, and U. Weber. Internal spinalfixator stiffness has only a minor influence on stresses in the adjacent discs.
Spine , 24(12):1192–1196, 1999.[54] T. Ropinski, S. Oeltze, and B. Preim. Survey of glyph-based visualizationtechniques for spatial multivariate medical data. In
Computers & Graphics ,vol. 35, pp. 392–401, 2011.[55] P. Roussouly, S. Gollogly, E. Berthonnaud, and J. Dimnet. Classificationof the normal variation in the sagittal alignment of the human lumbar spineand pelvis in the standing position.
Spine , 30(3):346–353, 2005. doi: 10.1097/01.brs.0000152379.54463.65[56] V. Solteszova, N. N. Smit, S. Stoppel, R. Gr¨uner, and S. Bruckner. Me-mento: Localized time-warping for spatio-temporal selection. In
ComputerGraphics Forum , vol. 39, pp. 231–243, 2020.[57] W. Sun, B. Starly, J. Nam, and A. Darling. Bio-cad modeling and its ap-plications in computer-aided tissue engineering.
Computer-Aided Design ,37(11):1097–1114, 2005.[58] F. van Ham and B. Rogowitz. Perceptual organization in user-generatedgraph layouts.
IEEE Trans Vis Comput Graph , 14(6):1333–1339, 2008.[59] C. Ware, T. L. Turton, R. Bujack, F. Samsel, P. Shrivastava, and D. H.Rogers. Measuring and modeling the feature detection threshold functionsof colormaps.
IEEE Trans Vis Comput Graph , 25(9):2777–2790, 2018.[60] J. Weissenb¨ock, B. Fr¨ohler, E. Gr¨oller, J. Kastner, and C. Heinzl. Dynamicvolume lines: Visual comparison of 3D volumes through space-fillingcurves.
IEEE Trans Vis Comput Graph , 25(1):1040–49, 2018.[61] H.-J. Wilke, L. Claes, H. Schmitt, and S. Wolf. A universal spine testerfor in vitro experiments with muscle force simulation.
European SpineJournal , 3(2):91–97, 1994.[62] H.-J. Wilke, F. Rohlmann, C. Neidlinger-Wilke, K. Werner, L. Claes, andA. Kettler. Validity and interobserver agreement of a new radiographicgrading system for intervertebral disc degeneration: Part i. lumbar spine.
European Spine Journal , 15(6):720–730, 2006. doi: 10.1007/s00586-005-1029-9[63] C. Zhang, T. Schultz, K. Lawonn, E. Eisemann, and A. Vilanova. Glyph-based comparative visualization for diffusion tensor fields.