Daniel Toll

Evaluation of a static architectural conformance checking method in a line of computer games

We present an evaluation of a simple method to find architectural problems in a product line of computer games. The method uses dependencies (direct, indirect, or no) to automatically classify types in the implementation to high-level components in the product line architecture. We use a commercially available tool to analyse dependencies in the source code. The automatic classification of types is compared to a manual classification by the developer, and all mismatches are reported. To evaluate the method, we inspect the source code and look for a pre-defined set of architectural problems in all types. We compare the set of types that contained problems to the set of types where the manual and automatic classification disagreed to determine precision and recall. We also investigate what changes are needed to correct the found mismatches by either designing and implementing changes in the source code or refining the automatic classification. Our evaluation shows that the simple method is effective at detecting architectural problems in a product line of four games. The method is lightweight, customisable and easy to implement early in the development cycle.

A Study of the Effect of Data Normalization on Software and Information Quality Assessment

Indirect metrics in quality models define weighted integrations of direct metrics to provide higher-level quality indicators. This paper presents a case study that investigates to what degree quality models depend on statistical assumptions about the distribution of direct metrics values when these are integrated and aggregated. We vary the normalization used by the quality assessment efforts of three companies, while keeping quality models, metrics, metrics implementation and, hence, metrics values constant. We find that normalization has a considerable impact on the ranking of an artifact (such as a class). We also investigate how normalization affects the quality trend and find that normalizations have a considerable effect on quality trends. Based on these findings, we find it questionable to continue to aggregate different metrics in a quality model as we do today.

Evaluation of an architectural conformance checking software service

Static architectural conformance checking can be used to find architectural violations, cases where the implementation does not adhere to the architecture, and prevent architectural erosion. We implement a software service for automated conformance checking and investigate the effect this has on the number of architectural violations in software projects. The service is implemented using our heuristic-based approach to static architecture conformance checking of the Model-View-Controller pattern. The service is integrated in the source code management system of each project, so a report is generated every time the source code is modified. The service was evaluated in a field experiment that consisted of eight student projects. We found that the four projects that used the service produced significantly fewer violations compared to those that did not.

Evolution and evaluation of the model-view-controller architecture in games

In game software it is important to separate game play code from rendering code to ease transitions to new technologies or different platforms. The architectural pattern Model-View-Controller (MVC) is commonly used to achieve such separation. We investigate how the MVC architectural pattern is implemented in five game projects from a small development studio. We define a metrics-based quality model to assess software quality goals such as portability and rendering engine independence and perform an architectural analysis. The analysis reveals three different evolutions of the pattern. We also assess the quality and find that 1. The evolutions of the architecture differ in quality and 2. An architectural refactoring to a newer version of the architecture increases the software quality.

Why is Unit-testing in Computer Games Difficult?

In this paper, we investigate how unit testing can be used to verify computer game simulations with implicit and poorly defined mathematical models. The difficulty of calculation of expected values for such cases tends to lead to Fragile and Obscure tests. The effects of these test smells can be reduced if we improve the strategy we use to calculate expected values. We investigate four strategies and while each of these is insufficient, we outline how they can be combined into a single, sufficient strategy.

Visualizing Programming Session Timelines

Learning programming with tutor tools has grown in popularity. These tools present programming assignments and provide feedback in the form of test-cases and compilation errors. Our timeline visualization of data from one such tool allows us to tell a story about what files were accessed and for how long, in what order files were edited, grown or shrunk, what errors the student ran into, and how those errors were addressed. This can be done without a need to read and replay the entire programming session. In sum, the tool has been used to visualize logs from students that tried to solve programming assignments and we find interesting stories that can help us improve how we address new assignments.

I know it when I see it Perceptions of Code Quality: ITiCSE '17 Working Group Report

Context. Code quality is a key issue in software development. The ability to develop high quality software is therefore a key learning goal of computing programs. However, there are no universally accepted measures to assess the quality of code and current standards are considered weak. Furthermore, there are many facets to code quality. Defining and explaining the concept of code quality is therefore a challenge faced by many educators. Objectives. In this working group, we investigated code quality as perceived by students, educators, and professional developers, in particular, the differences in their views of code quality and which quality aspects they consider as more or less important. Furthermore, we investigated their sources for information about code quality and its assessment. Methods. We interviewed 34 students, educators and professional developers regarding their perceptions of code quality. For the interviews they brought along code from their own experience to discuss and exemplify code quality. Results. There was no common definition of code quality among or within these groups. Quality was mostly described in terms of indicators that could measure an aspect of code quality. Among these indicators, readability was named most frequently by all groups. The groups showed significant differences in the sources they use for learning about code quality with education ranked lowest in all groups. Conclusions. Code quality should be discussed more thoroughly in educational programs.

Detailed Recordings of Student Programming Sessions

Observation is important when we teach programming. It can help identify students that struggle, concepts that are not clearly presented during lectures, poor assignments, etc. However, as development tools become more widely available or courses move off-campus and online, we lose our ability to naturally observe students. Online programming environments provide an opportunity to record how students solve assignments and the data recorded allows for in-depth analysis. For example, file activities, mouse movements, text-selections, and text caret movements provide a lot of information on when a programmer collects information and what task is currently worked on. We developed CSQUIZ to allow us to observe students on our online courses through data analysis. Based on our experience with the tool in a course, we find recorded sessions a sufficient replacement for natural observations.

The challenge of teaching students the value of programming best practices

We investigate the benefits of our programming assignments in correlation to what the students learn and show in their programming solutions. The assignments are supposed to teach the students to use best practices related to program comprehension, but do the programming assignments clearly show the benefits of best practices? We performed an experiment that showed no significant result which suggests that the assignments did not emphasise the value of best practices. As lecturers, we understand that constructing assignments that match the sought after outcome in students learning is a complex task. The experiment provided valuable insights that we will use to improve the assignments to better mirror best practices.