Exploring the Design Space of Aesthetics with the Repertory Grid Technique
EExploring the Design Space of Aesthetics withthe Repertory Grid Technique
David Baum − − − Leipzig University, Grimmaische Strae 12, 04109 Leipzig, Germany
Abstract.
By optimizing aesthetics, graph diagrams can be generatedthat are easier to read and understand. However, the challenge lies inidentifying suitable aesthetics. We present a novel approach based onrepertory grids to explore the design space of aesthetics systematically.We applied our approach with three independent groups of participantsto systematically identify graph aesthetics. In all three cases, we wereable to reproduce the aesthetics with positively evaluated influence onreadability without any prior knowledge. We also applied our approachto two- and three-dimensional domain-specific software visualizations todemonstrate its versatility. In this case, we were also able to acquireseveral aesthetics that are relevant for perceiving the visualization.
Keywords:
Aesthetics · Graph · Repertory Grid Technique · SoftwareVisualization · Visual Analytics
Making visualizations easier to read and to understand is a challenging taskand has been researched for decades [7]. Aesthetics are a suitable method toaddress this problem [30]. They represent heuristics to predict human perceptionof the visualization. Aesthetics are visual metrics that must be both objectivelymeasurable and perceptible to the observer [1]. They are independent of thesemantic context of a visualization and refer only to visual properties.For graph layouts consisting of nodes and edges, aesthetics are well re-searched. Typical aesthetics are, e.g., edge crossings and cutting angles of edges [30].These criteria are used as optimization goals, e.g., minimizing the number of edgecrossings or maximizing the average cutting angle to generate perceivable andcomprehensible graph layouts. Aesthetics have been adapted to other visualiza-tions, e.g., different sorts of diagrams [33, 8] as well as complex graphical userinterfaces such as websites [26]. Each type of visualization has its own aesthetics.Therefore, the state of the art research process has to be repeated for every typeof visualization. The process is always similar and comprises the following steps.1.
Define one or multiple aesthetics.
Every aesthetic must be measurable.There is no established way to derive aesthetics. Many aesthetics are onlychosen because they seem to be plausible, so this step is subjective. a r X i v : . [ c s . H C ] A ug . Evaluate impact of proposed aesthetics empirically.
In this step, par-ticipants solve tasks using different visualizations, measuring error rate andtime to complete the task. It is necessary to be able to trace possible differ-ences in solving the tasks back to different aesthetics. This can be achieved,for example, by changing one aesthetic while keeping all others approxi-mately constant. This is often only possible to a limited extent due to de-pendencies between different aesthetics.3.
Implement layout algorithm.
To make the positively evaluated aestheticsusable in practice, it is necessary to provide a suitable layout algorithm.It should have a reasonable runtime behavior and take care of conflictingoptimization goals.The whole process is iterative. Depending on the procedure, step 3 mightbe performed before step 2. When new aesthetics are defined, the subsequentsteps have to be repeated accordingly. However, this approach leads to significantproblems. Without being aware of all relevant aesthetics, interactions betweenthem cannot be considered. Unknown but relevant aesthetics might distort theoutcome of empirical evaluations significantly [17]. In addition, some aestheticsare not obvious, especially for complex visualizations with many different visualprimitives. Hence, there is a risk that important aesthetics may be overlooked.The whole process is very tedious because aesthetics are also defined and exam-ined that have no measurable effect on readability.In this paper, we want to improve the identification of aesthetics by makingthe process more reproducible and less based on the researcher’s intuition. Weuse a novel approach based on the repertory grid technique (RGT). This is aninterview technique that triggers the participants’ creativity to describe verballythe differences between certain elements. These descriptions then serve as a basisfor the definition of aesthetics. Therefore, more relevant aesthetics are knownwhen it comes to conducting the evaluations. This will simplify the outlinedresearch process and help to overcome the mentioned problems.
Several models and guidelines exist for designing and evaluating visualizations [25,22, 23]. However, these process models do not use any aesthetics. The only frame-work known to us that takes aesthetics into account is [21]. It assumes thataesthetics and its effects are already known. Most aesthetics are selected basedon intuition without giving an explicit rationale. Bennett et al. [3] justify estab-lished aesthetics with Gestalt principles. However, they do not show how newaesthetics can be derived from Gestalt principles.We are only aware of one approach to improve the iterative process by makingit less subjective and more efficient: drawings [27]. The participants are askedto draw visualizations with a given structure, often node-link diagrams. Sub-sequently, it is examined by statistical means which aesthetics the respondentapplied to their drawing. Drawings can help to some extent to weigh aestheticsr identify any irrelevant aesthetics. However, the capabilities of this approachto explore the aesthetics design space are limited. This approach requires well-defined aesthetics to check if they have been used by the subject or not. Also,drawings will not work for complex or three-dimensional visualizations, sincemost participants will be unable to express their mental model adequate in adrawing of such visualizations.We see drawings as a step towards improving the described research process.Nevertheless, some problems remain unsolved, which we address within this pa-per. Our approach is based on the RGT. We are not aware that this method hasalready been used in the context of aesthetics. In our previous work [1] we usedRGT to identify neglected and overemphasized information in visualizations.
The RGT is an empirical and qualitative research method. Its basic assumption isthat everybody describes and evaluates elements based on a large set of personalconstructs that can be expressed by using bipolar constructs [10, p. 15]. Elementsare for example objects, persons, experiences, or even products. A construct isdefined as “a way in which two or more things are alike and thereby different froma third or more things” [19, p. 61]. These constructs consist of two opposite poles,e.g., “clear” and “confusing” as well as a construct continuum in between, i.e.,different degrees of clarity. The RGT is an approach to make these constructsexplicit and visible. The process is reproducible and facilitates the structuredexploration of an unknown domain. To apply the RGT, multiple design decisionshave to be made, e.g., how elements and constructs are selected. In the following,we will discuss the research design that corresponds to our research questions.We will not discuss variants that are not reasonable for exploring design spacessuch as constructs provided by the researcher.
Every interview is done with the same set of elements. They are selected bythe researcher and should represent as much breadth of the domain as possible.The RGT helps to recognize differences between those elements. Something thatall elements have in common will most likely not be taken into account by theparticipants. For example, if all visualizations only consist of black entities, noconstructs for color mapping can be expected. The constructs obtained in thisway are still valid, but it is possible that they only describe a subset of thedomain. This threat can be reduced by asking the subject to provide additionalelements that differ from those given [12]. Placeholder elements such as “idealvisualization” or “worst visualization” can also be used to ensure adequate cover-age of the domain [9]. The elements can be based on real or artificially generateddata. .2 Construct elicitation
The constructs are not predefined. It will be investigated which constructs theparticipants use to describe the elements shown to them. For this purpose, threeelements are randomly selected and presented at once to the participant (cf. 1.The participant has to answer the following question: “How are any two of thesealike in some way?”, complemented by “What is the opposite of that?” [11]. Theanswers to both questions are the respective poles. For example, a participantmight describe those visualizations with two bipolar constructs, “helpful unhelp-ful” and “ugly beautiful”. For them, these are the relevant attributes in whichthe two visualizations differ. Constructs differ in their level of abstraction. Someconstructs are abstract, e.g., “ugly beautiful”, others are very concrete, e.g., “noedge crossings many edge crossings”. Abstract constructs are less helpful for ourresearch question since they are subjective and hard to measure. These abstractconstructs might lead to furtheer constructs if they are investigated in depth.It is not uncommon that a construct implies another construct. They only varyin their level of abstraction. The process of using a construct to attain a moreconcrete construct is called laddering and is a common part of the repertory gridinterview [12, 9]. This can be done by asking “Why does this visualization appearmore beautiful to you?”. For example, the answer could lead to the construct“symmetrical asymmetrical”. The whole procedure is repeated by using otherrandomly selected elements as long as the participant creates new constructsto distinguish between the elements. It is not feasible to use all possible com-binations during the interview, hence a reasonable stop criterion is necessary.We advise stopping the interview when three times in a row the participant didnot use any new constructs. This will lead to enough constructs and does notprolong the interview unnecessarily. During the interview, the participants haveno access to any constructs they used before. Otherwise, participants may try toavoid repetition or use synonyms to find as many constructs as possible. Further,there is no restriction on how many constructs may be named.The interviewer must understand what the participant describes with a con-struct. For this reason, informal communication between both persons is a reg-ular and intended part of the RGT. This may include further explanations bythe participant, showing examples or simple drawings. The RGT demands highstandards of the interviewer and the research design. The interviewer should befamiliar with the established guidelines for conducting the interviews. We havemainly followed the recommendation of Kurzhals et al. [20] and Fransella [9].Normally, a repertory grid interview also includes the creation of the name-giving grids. The participant evaluates for each element and each construct whichpole is more appropriate. For us, however, this information is of little value aswe are interested in the constructs used. For this reason, we have skipped thisstep.
The output of the interview is a list of constructs, that has to be further analyzed.Some constructs will represent aesthetics directly, but many constructs are notnteresting to us. This is expected and cannot be avoided. Kurzhals et al. proposethe following categorization to analyze the constructs of repertory grid interviewsto explore the design space [20]: – Visual Mapping
This category covers all constructs, that refer to the useof visual primitives (e.g., straight edges bent edges) and color mapping. – Composition
This category consists of constructs that refer to the compo-sition of visualization elements, i.e., layout, alignment, and visual density. – Data-related
Constructs are data-related and therefore belong to the thirdcategory, if they depend on the underlying data, such as “few nodes manynodes”. – Visual experience
The last category describes the hedonistic qualities ofvisualizations, such as “ugly beautiful” [14].For our research question, only the first two categories are interesting sincethey represent aesthetics. Data-related constructs do not describe the propertiesof the visualization but of the underlying data. Constructs of the last categoryare often vague and used as a starting point for laddering during the interview.In the process, more concrete constructs can be revealed that refer to visualmapping or composition. The last step is to reformulate the constructs as aes-thetics. “straight edges bend edges” becomes “edge curve” and so on. This stepis straight forward and should not cause any problems. If ambiguities shouldarise here, the laddering was not sufficiently performed. The final result is acomprehensive list of aesthetics. The method of extraction ensures that all aes-thetics are perceivable for human beings. However, there is no guarantee that allof them will have a significant influence on the readability of the visualization.
Many graph aesthetics have been proposed, and some of them have been evalu-ated in empirical studies [5]. We define positively evaluated aesthetics as aesthet-ics for which a significant influence on readability has already been empiricallydemonstrated. We applied the RGT to the domain of graph visualization tocheck the following hypotheses: – H1:
With RGT all positively evaluated aesthetics can be reproduced. – H2:
The results of RGT can be reproduced when using different elementsand different participants. H1 is used to check whether the RGT provides valid results. With H2 , wecheck whether the results are reproducible or depend on the selected elements orparticipants. We are not aware of any other approach to systematically explorethe design space of aesthetics. A comparative evaluation with other approachesis therefore not possible. able 1. In- and exclusion criteria for literature studyDatabase Search Term Inclusion (+) and Exclusion () CriteriaScienceDirect graph aesthetics + Publication Type:
Research Article + Journal:
Computer Aided Design + Journal:
Journal of Visual Languages
ACM (+graph +aesthetics)IEEE graph aesthetics Publication Type:
Book
SpringerLink graph aesthetics + Publication Type: Conference Papers+ Discipline:
Computer Science + Subdiscipline:
Information Systems Appl. + Subdiscipline:
User Interfaces and HCI
To verify the results of our evaluation, we have conducted an extensive literaturestudy following the guideline from vom Brocke et al. [38] to establish a groundtruth for H1 . It contains all the aesthetics proposed in the literature and whethera significant influence on readability could be empirically evaluated. We havesearched the databases available to us with the search terms listed in Table 1.The additional inclusion and exclusion criteria are necessary because theterm aesthetics is used in many different disciplines with different meanings. Intotal, we received 519 hits, 47 from ScienceDirect, 69 from ACM, 42 from IEEE,and 373 from SpringerLink. Two entries had to be removed due to duplicates,leaving 517 entries. We then manually sorted out the publications where theterm aesthetics is not used in the sense mentioned here. Then, we performed abackward search on the 95 remaining publications. This was necessary becausemany publications use aesthetics, but it was not the original source in whichthe metric was proposed. We also included the summaries from Taylor [36] andBennett [3], who did a similar literature study with a smaller focus. The firstthree columns of Table 2 summarize the results of our literature study. All in all,we identified 29 different graph aesthetics proposed in 14 different publications.For 13 aesthetics we could find an empirical evaluation that showed a significantinfluence on readability. For some aesthetics, we were not able to trace themback to exactly one source. In such a case we listed all found publications.Most aesthetics refer to the position of the nodes, edge intersections, thelength and curvature of the edges, and the angles between them. Some aes-thetics refer to paths, i.e., combinations of edges. For example, path bendinessdescribes how straight a path is or how many bends it has. Most aesthetics canbe sorted into the “Composition” category since they refer to layouting. Only afew aesthetics belong to the “Visual Mapping” category, they are highlighted inTable 2. For each group, we used 12 undirected graphs as elements. They canbe seen in the appendix. They consist only of black nodes and black undirected able 2.
List of all aesthetics derived from literature. Entries of the category “VisualMapping” are highlighted.Name Source Evaluation Group A Group B Group CAngular resolution [31, 36, 7] [17] 4 3 3Area [35, 36] [32] 10 8 8Aspect ratio [7] 3 4 3Cluster similar nodes [35, 36] [15] 5 5 4Convex faces [35]Consistent flow direction [31] 3 4 6Crossing angle [16, 39, 17] [39, 17] 8 9 7Degree of edge bends [31, 35, 6] [30, 29, 32] 9 9 10Difference between angles [18]Distribute nodes evenely [35, 36] 6 8 8Edge orthogonality [31] [32] 5 4 4Global symmetry [35, 4] [30] 4 3 4Keep nodes apart from edges [6] 3 6 7Local symmetry [35, 4] [32] 8 10 8Maximum bends [7] 9 9 8Maximum edge length [7, 36, 35] 6 4 4Node orthogonality [31] 3Nodes should not overlap [34] 4 3 3Number of bends [7] 3 3 4Number of branches [39] [39] 5 3 5Number of edge crossings [35, 36, 31, 6, 4] [30, 29, 32, 28] 6 3 8Path bendiness [39] [39] 3 3 5Shortest path length [39] [39] 4 3 3SD of crossing angles [17]SD of angular resolution [17]Total edge length [35, 36]Uniform edge bends [36] 3 3 4Uniform edge lengths [35, 13, 4, 6] 4 3 3Whitespace to ink ratio [28, 37] [28] 3 3 6 edges. We did not use any text labels or color mappings to keep the graphs assimple as possible. The graphs are not based on real but on artificially generateddata. We used the igraph library for R to generate random graphs. The smallestgraph contains 5 edges, the largest graph contains 69 edges. Each node positionwas assigned randomly, i.e., overlaps could and did occur. For each edge, thedegree and direction of edge curvature were determined randomly as well aswhich nodes the edge connects. No other properties were taken into account.Fig. 1 shows three of the used graphs. Participants
In total, we interviewed 30 participants. Initially, these partici-pants were divided into three groups to check H2 . We decided on a group sizeof 10 because it has proven to be sufficient in many studies. If the method is https://igraph.org/r/ ig. 1. User Interface for Repertory Grid Interview showing three graphs
Table 3.
List of all novel aesthetics elicited in the evaluationName Group A Group B Group CFace area 2 3 3Uniform faces 4 3 3 widely applied, it may be possible to find a convergence point at which additionalparticipants do not add any value. All participants were bachelor or master stu-dents of economics and have received an expense allowance. They were all nativespeakers of German, which was also the language of the interviews. In group A,the students were between 19 and 40 years old (mean: 23.3 years). 50% werefemale, 50% male. In group B, the students were between 18 and 29 years old(mean: 21.9 years). 40% were female, 60% male. In group C, the students werebetween 19 and 25 years old (mean: 21.5 years). 60% were female, 40% male.Participants of the same group have worked with the same elements.
Interview
The complete evaluation was done using the evaluation server ofGetaviz [2]. It displays three random graphs at the same time (see Fig. 1). Theparticipant cannot interact with the visualizations, i.e., there are no tooltips andit is not possible to navigate or zoom in and out. In the prestudy, we noticedthat sometimes rather vague terms such as “simple” or “complex” were usedas constructs. To improve the laddering, we asked the participants to draw forinstance a “very simple” or “very complex” graph and used it as an additionalelement. Having additional elements with extreme properties helps the partici-pant to name differences between the elements [19]. Besides that, we conductedthe interview as described in the method section.
The interview procedure led to a set of 56 different constructs from all par-ticipants. These constructs are divided into the four categories as follows: Vi-sual Mapping (4 constructs), Composition (21 constructs), Data-related (11 con-structs), Visual Experience (20 constructs). The distribution of the categoriess similar to previous studies but with fewer constructs referring to visual map-ping [20]. That was to be expected since the visual mapping was given by usingnode-link diagrams and corresponds to the distribution of published graph aes-thetics, which refer to the composition in most cases as well. The further analysiswill focus on the 25 constructs from the first two categories since the other con-structs are not relevant concerning aesthetics. For each aesthetic in Table 2 itis indicated which groups have used it. We can fully confirm hypothesis H1 .An aesthetic was used by 51.7% of the participants on average (min: 33.3%,max: 93.3%) With a softer stop criterion, some aesthetics might have been usedby more participants. It is neither necessary nor likely that all participants useidentical constructs.We were able to reproduce all published graph aesthetics that have an empir-ically verified impact on readability with all three groups. Group A reproduced82%, Group B reproduced 86%, and Group C reproduced 82% of published graphaesthetics. The five aesthetics not mentioned were not positively evaluated with-out exception. In the case of differences between smallest and optimal crossingangle , standard deviation of crossing angles , and standard deviation of angularresolution this is not surprising. Participants of all groups referred to crossingangles quite often, but not in such a mathematical way.Table 3 lists all elicited aesthetics that are novel, which means that we couldnot find a corresponding aesthetic in our literature study. Both novel aestheticsrefer to faces, i.e., the empty white areas that are bordered by edges. So farin the literature, it has only been suggested to consider whether the faces areconvex or concave. This was not relevant for any of the participants. However,participants of all groups distinguished between faces with a small area and faceswith a huge area. They also took into account, whether the graph consists offaces with a similar shape or not. The results of our evaluation indicate thatthe area and shape of the faces might influence how a graph is perceived. It has (a) “Ideal Graph” (b) “Worst Graph” Fig. 2.
Two example graphs drawn by participants o be verified empirically whether these aesthetics have a significant impact onunderstandability and readability.With one exception, the used aesthetics are consistent among all three groups.Only participants of Group 2 used node orthogonality to differentiate betweenthe elements. Therefore, we can accept hypothesis H2 conditionally. For the interviews, we have specified the elements and deliberately used ran-dom values for different properties of the graphs. There is a risk that therebythe aesthetics are predetermined and reflect only our assumptions. However, theparticipants mentioned aesthetics that have no direct connection to the ran-domized graph properties. For example, all groups used global symmetry as aconstruct. None of the given elements was symmetric or designed with respectto symmetry. However, many self-drawn graphs were symmetrical as shown in 2a,making them different from the elements provided.All interviews were conducted by the same person, therefore there is a risk ofconfirmation bias. Other potential confounding factors are the background anddegree of experience of the participants.
Software visualization is a subdomain of information visualization about visual-izing the structure, behavior, and evolution of software systems. These visualiza-tions are used in visual analytics tools to support software developers, projectmanagers, and other stakeholders to improve their understanding of develop-ment artifacts and corresponding activities. Software visualizations are complexdomain-specific diagrams that might contain multiple thousand data points, var-ious relationships between them, and a multitude of different visual primitives.Presenting this amount of information in such a way that it can be processedwell by a human being is a central challenge of this domain. The Recursive Disk(RD) Metaphor (Fig. 3a) [24] and the City Metaphor (Fig. 3b) [40] are twoapproaches to adequately visualize these data.Both metaphors are hierarchical visualizations that represent the internalstructure of a software system. The RD Metaphor is an abstract two-dimensionalmetaphor. It consists of two different kinds of disks (gray and purple) as well astwo different kinds of disk segments (blue and yellow). The disks can be nestedto represent contains-relationships between the elements as shown in Fig. 3a.The area of the disks and disk segments is also used to visualize the propertiesof the software system. The City metaphor is a three-dimensional real-worldmetaphor. It consists of gray districts and purple buildings as shown in Fig. 3b.The building’s height and base area also represent the properties of the softwaresystem.A high degree of readability and comprehensibility is a central requirementfor these kinds of diagrams. To improve them, however, no aesthetics have beenonsidered to date, i.e., there are no known aesthetics at all for this kind of visu-alization. One of the reasons for this is that the described problems of the currentresearch process are even greater with such complex visualizations. Therefore, weapply our approach to software visualizations to elicit aesthetics that will helpimprove readability and comprehensibility in the future. We have conducted onestudy on RD visualizations and one on City visualizations. Both studies are in-dependent of each other. However, since the study design is very similar, we willdescribe both studies together.
For each study, we used 12 visualizations as elements. We chose 12 differentsoftware systems based on software metrics (number of packages, number ofclasses, number of methods, number of attributes, and number of statements) tocover a wide range. We used Getaviz to generate the corresponding visualizationsfor each system. For RD, we conducted interviews with ten participants (50%male, 50% female). Their age varies between 19 and 52 years (mean: 22.8 years).For City, we conducted interviews with ten different participants (70% male, 30%female). Their age varies between 18 and 38 years (mean: 24 years). During theconstruct elicitation we gave participants the possibility to navigate, i.e., rotatethe visualization as well as zoom in and out, so they could view the visualizationas they liked. Otherwise, it would not be possible to perceive all visual entitiessince some entities might be occluded or too small to perceive. Apart from that,the interviews were the same as described in Section 4.2.
We elicited 53 constructs during the RD interviews, 19 of them qualified as aes-thetics. Each participant used 15.5 constructs on average. To describe the city (a) RD Metaphor (b) City Metaphor
Fig. 3.
Software visualizations generated by Getaviz able 4.
Elicited Aesthetics for RD and City MetaphorRD Aesthetics City AestheticsArea AreaBlue segments evenly distributed (global) Aspect ratio (global)Blue segments evenly distributed (local) Aspect ratio of districts (local)Centered focus Buildings in a rowEdge thickness Building densityFace area Clustering of similar buildingsGlobal symmetry Empty district areaLength of spiral windings Gap between buildingsLocal symmetry Largest difference in building heightNesting depth Nesting depthNumber of spiral turns Share of empty areaShare of empty area Sort buildings by heightSorting of purple disks (local) Uniform base area of buildingsUniform size of gray disks Uniform buildingsUniform size of purple disks Uniform facesUniform structure of gray disksUniform structure of purple disksYellow segments evenly distributed (global)Yellow segments evenly distributed (local) metaphor, 45 constructs were used, 15 of them are aesthetics. Each participantused 13.5 constructs on average. Table 4 lists all elicited aesthetics for bothmetaphors. The aesthetics for the RD visualizations are mostly about color dis-tribution and nesting, i.e., many aesthetics refer to a local context. This makessense considering the recursive structure of the visualizations. City visualizationshave a similar structure, but buildings are clearly dominant since most aesthet-ics refer to buildings. Some aesthetics refer to the three-dimensionality of thevisualization, where the height of the buildings plays a major role.It is particularly noticeable that fewer constructs were used compared tographs, both per interview and overall. This is most likely because the edges ofthe graphs have many degrees of freedom that are not present in the RD andCity visualizations. Due to the semantic constraints, e.g., a building must alwaysbe located in a district, the design space is not as extensive as it is for graphs.The elicited aesthetics serve as a starting point to design better layout al-gorithms. In previous work, only density and area were considered. The elicitedaesthetics must now be empirically evaluated to find out which of them have asignificant influence on readability.
Our approach to explore aesthetics design space using repertory grids has beeneffective. We have evaluated the approach as far as possible and were able to showin an empirical study that with only 10 participants all published and positivelyvaluated aesthetics can be identified. We could also show that our approachdelivers reproducible results and can be applied to diverse visualizations. Thequality and validity of the results depend above all on the selection of the suitableelements. The inclusion of drawings and placeholder elements was particularlyhelpful. However, the assessment of a domain expert is still necessary to createand select suitable elements. Nevertheless, the process is much less subjectiveand intuition-based than before.The analysis of the repertory grid data applied in this paper is rather simpleand could be enhanced in the future. For example, we did not analyze how oftencertain aesthetics have been used by participants. In our future work, we willevaluate the derived aesthetics from software visualizations to further validatethe results.
References
1. Baum, D.: Introducing aesthetics to software visualization. In: 23rd InternationalConference in Central Europe on Computer Graphics, Visualization and Com-puter Vision, WSCG 2015 - Short Papers Proceedings. vol. 23, pp. 65–73 (2015),http://wscg.zcu.cz/WSCG2015/! 2015 WSCG SHORT proceedings.pdf2. Baum, D., Schilbach, J., Kovacs, P., Eisenecker, U., Muller, R.: GETAVIZ:Generating Structural, Behavioral, and Evolutionary Views of Software Systemsfor Empirical Evaluation. In: Proceedings - 2017 IEEE Working Conference onSoftware Visualization, VISSOFT 2017. vol. 2017-Octob, pp. 114–118 (2017).https://doi.org/10.1109/VISSOFT.2017.123. Bennett, C., Ryall, J., Spalteholz, L., Gooch, A.: The aesthetics ofgraph visualization. In: Proceedings of the 2007 Computational Aes-thetics in Graphics, Visualization, and Imaging. pp. 57–64 (2007).https://doi.org/10.2312/COMPAESTH/COMPAESTH07/057-0644. Biedl, T., Marks, J., Ryall, K., Whitesides, S.: Graph Multidrawing: Finding NiceDrawings Without Defining Nice. Lecture Notes in Computer Science (includingsubseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformat-ics) , 347–355 (1999). https://doi.org/10.1007/3-540-37623-2 265. Chen, C.: Top 10 unsolved information visualization problems.IEEE Computer Graphics and Applications (4), 12–16 (2005).https://doi.org/10.1109/MCG.2005.916. Davidson, R., Harel, D.: Drawing Graphs Nicely Using SimulatedAnnealing. ACM Transactions on Graphics (4), 301–331 (1996).https://doi.org/10.1145/234535.2345387. Di Battista, G., Eades, P., Tamassia, R., Tollis, I.G.: Graph Drawing: Algorithmsfor the Visualization of Graphs (1999). https://doi.org/10.1007/BFb0021783,http://books.google.com/books?id=Dt4eAQAAIAAJ&printsec=frontcover8. Effinger, P., Jogsch, N., Seiz, S.: On a Study of Layout Aesthetics for BusinessProcess Models Using BPMN pp. 31–45 (2010)9. Fransella, F.: Fransella - 2003 - Some skills and tools for personal constructpract.pdf. International Handbook of Personal Construct Psychology pp. 105–122(2005)10. Fransella, F.: International Handbook of Personal Construct Psychology (2005).https://doi.org/10.1002/04700133701. Fransella, F., Neimeyer, R.A.: George Alexander Kelly: The Man and His The-ory. International Handbook of Personal Construct Psychology pp. 21–31 (2005).https://doi.org/10.1002/0470013370.ch212. Fried, R., Mayer, M.F.: Grid technique as tool for improving health services to insti-tutionalized children: A ten-year experience. Journal of the American Medical As-sociation (1), 1–5 (1956). https://doi.org/10.1001/jama.1956.0297001000300113. Gansner, E.R., Koren, Y., North, S.: Graph drawing by stress majorization. LectureNotes in Computer Science (1), 239–250 (2004). https://doi.org/10.1007/978-3-540-31843-9 2514. Hassenzahl, M., Wessler, R.: Capturing design space from a user perspective: Therepertory grid technique revisited. International Journal of Human-Computer In-teraction (3-4), 441–459 (2000)15. Huang, W., Eades, P., Hong, S.H.: Layout effects: Comparison of sociogram draw-ing conventions (575) (2005)16. Huang, W., Eades, P., Hong, S.H.: Larger crossing angles makegraphs easier to read. Journal of Visual Languages and Comput-ing (4), 452–465 (2014). https://doi.org/10.1016/j.jvlc.2014.03.001,http://dx.doi.org/10.1016/j.jvlc.2014.03.00117. Huang, W., Eadesy, P., Hongy, S.H., Linz, C.C.: Improving force-directed graphdrawings by making compromises between aesthetics. Proceedings - 2010 IEEESymposium on Visual Languages and Human-Centric Computing, VL/HCC 2010pp. 176–183 (2010). https://doi.org/10.1109/VLHCC.2010.3218. Hutchison, D., Mitchell, J.C.: Graph drawing: 12th international symposium, GD2004, New York, NY, USA, September 29 - October 2, 2004 ; revised selectedpapers. p. 536 (2004). https://doi.org/10.1007/3-540-68339-9 3419. Kelly, G.: A Theory of Personality: The Psychology of Personal Constructs (1963)20. Kurzhals, K., Weiskopf, D.: Exploring the Visualization Design Spacewith Repertory Grids. Computer Graphics Forum (3), 133–144 (2018).https://doi.org/10.1111/cgf.1340721. Lau, A., Moere, A.V.: Towards a model of information aesthetics in informationvisualization. Proceedings of the International Conference on Information Visual-isation pp. 87–92 (2007). https://doi.org/10.1109/IV.2007.11422. McKenna, S., Mazur, D., Agutter, J., Meyer, M.: Design activity framework forvisualization design. IEEE Transactions on Visualization and Computer Graphics (12), 2191–2200 (2014). https://doi.org/10.1109/TVCG.2014.234633123. Meyer, M., Sedlmair, M., Quinan, P.S., Munzner, T.: The nested blocksand guidelines model. Information Visualization (3), 234–249 (2015).https://doi.org/10.1177/147387161351042924. M¨uller, R., Zeckzer, D.: The recursive disk metaphor: A glyph-based approachfor software visualization. In: IVAPP 2015 - 6th International Conference on In-formation Visualization Theory and Applications; VISIGRAPP, Proceedings. pp.171–176. SciTePress, Set´ubal (2015). https://doi.org/10.5220/000534270171017625. Munzner, T.: A Nested Model for Visualization Design and Vali-dation. IEEE Transactions on Visualization and Computer Graph-ics (6), 921–928 (nov 2009). https://doi.org/10.1109/TVCG.2009.111,http://dx.doi.org/10.1109/TVCG.2009.11126. Pajusalu, M., Torres, R., Lamas, D.: The Evaluation of User Interface Aestheticsp. 74 (2012)27. Papadopoulos, C., Voglis, C.: Untangling graphs representing spatial relationshipsdriven by drawing aesthetics. ACM International Conference Proceeding Series pp.158–165 (2013). https://doi.org/10.1145/2491845.24918538. Polisciuc, E., Cruz, A., Machado, P., Arrais, J.P.: On the role of aesthetics in ge-netic algorithms applied to graph drawing. GECCO 2017 - Proceedings of the Ge-netic and Evolutionary Computation Conference Companion pp. 1713–1720 (2017).https://doi.org/10.1145/3067695.308255229. Purchase, H.C., James, M.I., Cohen, R.F.: An Experimental Study of the Basisfor Graph Drawing Algorithms. ACM Journal of Experimental Algorithmics , 4(1997). https://doi.org/10.1145/264216.26422230. Purchase, H.: Which aesthetic has the greatest effect on human understanding? In:Lecture Notes in Computer Science (including subseries Lecture Notes in ArtificialIntelligence and Lecture Notes in Bioinformatics). vol. 1353, pp. 248–261 (1997).https://doi.org/10.1007/3-540-63938-1 6731. Purchase, H.C.: Metrics for graph drawing aesthetics. Journal of Visual Lan-guages and Computing (5), 501–516 (2002). https://doi.org/10.1016/S1045-926X(02)90232-632. Purchase, H.C., Carrington, D., Allder, J.A.: Empirical evaluation of aesthetics-based graph layout. Empirical Software Engineering (3), 233–255 (2002).https://doi.org/10.1023/A:101634421561033. Purchase, H.C., Mcgili, M., Colpoys, L., Carrington, D.: Graph drawing aestheticsand the comprehension of UML class diagrams: an empirical study pp. 129–137(2001)34. Shannon, A.: Tidy Drawings of Trees. IEEE Transactions on Software Engineering SE-5 (5), 514–520 (1979). https://doi.org/10.1109/TSE.1979.23421235. Tamassia, R., Battista, G.D., Batini, C.: Automatic Graph Drawing and Read-ability of Diagrams. IEEE Transactions on Systems, Man and Cybernetics (1),61–79 (1988). https://doi.org/10.1109/21.8705536. Taylor, M., Rodgers, P.: Applying graphical design techniques to graph visuali-sation. Proceedings of the International Conference on Information Visualisation , 651–656 (2005). https://doi.org/10.1109/IV.2005.1937. Tullis, T.S.: Evaluation of Alphanumeric, Graphic, and ColorInformation Displays. Human Factors (5), 541–550 (1981).https://doi.org/10.1177/00187208810230050438. Vom Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., Cleven, A.:Reconstructing the giant: On the importance of rigour in documenting the liter-ature search process. 17th European Conference on Information Systems, ECIS2009 (2009)39. Ware, C., Purchase, H., Colpoys, L., McGill, M.: Cogni-tive measurements of graph aesthetics. Information Visualization ppendix Graph 1 Graph 2 Graph 3Graph 4 Graph 5 Graph 6Graph 7 Graph 8 Graph 9Graph 10 Graph 11 Graph 12
Fig. 4.
Elements for Group Araph 1 Graph 2 Graph 3Graph 4 Graph 5 Graph 6Graph 7 Graph 8 Graph 9Graph 10 Graph 11 Graph 12
Fig. 5.
Elements for Group Braph 1 Graph 2 Graph 3Graph 4 Graph 5 Graph 6Graph 7 Graph 8 Graph 9Graph 10 Graph 11 Graph 12