Dennis K. Peters

Using test oracles generated from program documentation

The paper illustrates how software can be described precisely using LD-relations, how these descriptions can be presented in a readable manner using tabular notations, and one way such descriptions can be used to test programs. The authors describe an algorithm that can be used to generate a test oracle from program documentation, and present the results of using a tool based on it to help test part of a commercial network management application. The results demonstrate that these methods can be effective at detecting errors and greatly increase the speed and accuracy of test evaluation when compared with manual evaluation. Such oracles can be used for unit testing, in situ testing, constructing self-checking software, and ensuring consistency between code and documentation.

Requirements-based monitors for real-time systems

Before designing safety- or mission-critical real-time systems, a specification of the required behavior of the system should be produced and reviewed by domain experts. After the system has been implemented, it should be thoroughly tested to ensure that it behaves correctly. This is best done using a monitor, a system that observes the behavior of a target system and reports if that behavior is consistent with the requirements. Such a monitor can be used both as an oracle during testing and as a supervisor during operation. Monitors should be based on the documented requirements of the system. If the target system is required to monitor or control real-valued quantities, then the requirements, which are expressed in terms of the monitored and controlled quantities, will allow a range of behaviors to account for errors and imprecision in observation and control of these quantities. Even if the controlled variables are discrete valued, the requirements must specify the timing tolerance. Because of the limitations of the devices used by the monitor to observe the environmental quantities, there is unavoidable potential for false reports, both negative and positive, This paper discusses design of monitors for real-time systems, and examines the conditions under which a monitor will produce false reports. We describe the conclusions that can be drawn when using a monitor to observe system behavior.

Generating a test oracle from program documentation: work in progress

A fundamental assumption of software testing is that there is some mechanism, an oracle, that will determine whether or not the results of a test execution are correct. In practice this is often done by comparing the output, either automatically or manually, to some pre-calculated, presumably correct, output [17]. However, if the program is formally documented it is possible to use the specification to determine the success or failure of a test execution, as in [1], for example. This paper discusses ongoing work to produce a tool that will generate a test oracle from formal program documentation. In [9], [10] and [11] Parnas et al. advocate the use of a relational model for documenting the intended behaviour of programs. In this method, tabular expressions are used to improve readability so that formal documentation can replace conventional documentation. Relations are described by giving their characteristic predicate in terms of the values of concrete program variables. This documentation method has the advantage that the characteristic predicate can be used as the test oracle -- it simply must be evaluated for each test execution (input & output) to assign pass or fail. In contrast to [1], this paper discusses the testing of individual programs, not objects as used in [1]. Consequently, the method works with program documentation, written in terms of the concrete variables, and no representation function need be supplied. Documentation in this form, and the corresponding oracle, are illustrated by an example. Finally, some of the implications of generating test oracles from relational specifications are discussed.

Comparison of Algorithms for Wind Parameters Extraction From Shipborne X-Band Marine Radar Images

In this paper, curve-fitting and intensity-level-selection (ILS)-based algorithms for wind parameter extraction from shipborne X-band nautical radar images are investigated. First, to exclude the rain cases and low-backscatter images, a data quality control process is designed for both algorithms. An additional process is then introduced for the ILS-based method to improve the accuracy of wind measurements, including the recognition of blockages and islands in the temporally integrated radar images. For the low sea states, a dual-curve-fitting is proposed. These wind algorithms are tested using radar images and shipborne anemometer data collected on the east coast of Canada. It is shown that the dual-curve-fitting algorithm produces improvements in the mean differences between the radar and the anemometer results for wind direction and speed of about 5.7° and 0.3 m/s, respectively, under sea states with significant wave height lower than 2.30 m. Also, a harmonic function that is least-squares fitted to the selected range distances vector as a function of antenna look direction is applied. Compared with the original ILS-based algorithm, the modified procedure reduces the standard deviation for wind direction and speed by about 4° and 0.2 m/s, respectively. Finally, the performance of these two modified methods are compared.

Hyper-Real-Time Ice Simulation and Modeling Using GPGPU

This paper describes the design of an efficient parallel implementation of an ice simulator that simulates the behaviour of a ship operating in pack ice. The main idea of the method is to treat ice as a set of discrete objects with very simple properties, and to model the system mechanics mainly as a set of discrete contact and failure events. In this way it becomes possible to parallelize the problem, so that a very large number of ice floes can be modeled. This approach is called the Ice Event Mechanics Modeling (IEMM) method which builds a system solution from a large set of discrete events occurring between a large set of discrete objects. The simulator is developed using the NVIDIA Compute Unified Device Architecture (CUDA). This paper also describes the execution of experiments to evaluate the performance of the simulator and to validate the numerical modeling of ship operations in pack ice. Our results show speed up of 11 times, reducing simulation time for a large ice field (9,801 floes) from over 2 hours to about 12 minutes.

An IDE for software development using tabular expressions

We present preliminary work on an IDE for formal software development using tabular expressions as the basis for precise specifications and descriptions of software behaviour.

An Easily Extensible Toolset for Tabular Mathematical Expressions

We describe the design of an open ended set of tools for manipulating multi-dimensional tabular expressions. The heart of the toolset is a set of modules that makes it possible to add new tools to the toolset without having detailed knowledge of the other tools. This set of modules, the Tool Integration Framework, supports new and existing tools by providing abstract communication interfaces. The framework that we describe has proven to be a practical approach to building an extensible set of tools.

Specification-based test oracles with JUnit

Software testing is an important step to help ensure that the software is behaving correctly. An important component of the test process is a test oracle, which determines if the software behavior is correct or not. In this paper, we present tools that enhance an integrated development environment to give the user the ability to write the formal specifications in a readable manner and generate test oracles automatically. The generated test oracles integrate smoothly with test frameworks (e.g., JUnit) and hence they can be directly used to test the behavior of the program. This approach for testing has the advantage that the quality of testing can be high and very efficient.

The draw-bot: a project for teaching software engineering

The authors present a course project which was successfully used to teach software design principles to third year computer engineering students. The goal of the project is to program a robot to trace a shortest path through a maze. The students, organized in teams of five, have to follow the classical steps of software development and prepare interface, design and testing documents. Having a project that requires controlling a device to complete a clear task generates enthusiasm in the students and helps them to understand the principles taught in the course.

GPU-Event-Mechanics Evaluation of Ice Impact Load Statistics

The paper explores the use of a GPU-Event-Mechanics (GEM) simulation to assess local ice loads on a vessel operating in pack ice. The methodology uses an event mechanics concept implemented using massively parallel programming on a GPU enabled workstation. The simulation domain contains hundreds of discrete and interacting ice floes. A simple vessel is modeled as it navigates through the domain. Each ship-ice collision is modeled, as is every ice-ice contact. Each ship-ice collision event is logged, along with all relevant ice and ship data. Thousands of collisions are logged as the vessel transits many tens of kilometers of ice pack. The GEM methodology allows the simulations to be performed much faster than real time. The resulting impact load statistics are qualitatively evaluated and compared to published field data. The analysis provides insight into the nature of loads in pack ice. The work is part of a large research project at Memorial University called STePS2 (Sustainable Technology for Polar Ships and Structures). Introduction Ice class vessels are unique in a number of ways in comparison to non-ice class vessels. Hull strength, power, hull form and winterization aspects are all issues that raise special challenges in the design of ice class ships. This paper focuses on matters of local ice loads which pertain to hull strength in ice class vessels. More specifically, the paper examines the parametric causes of local ice loads and statistics that result as a ship transits through open pack ice. The issue of pack ice transit is of interest to those wishing to operate safely in such conditions. One key question is that of safe operational speeds. Consider the special case of open pack ice, where floes are relatively small, numerous and resting in calm water. A vessel moving through such an ice cover would experience a series of discrete collisions. As long as a vessel moved very slowly, the loads would be very low. In such a case the vessel could make safe and steady progress, even if it had a relatively low ice class. However, if the vessel attempted to operate more aggressively, impact speeds would increase and a higher ice class would be needed for safe operations. The investigation below provides some insight into the factors that influence the loads in this situation. These factors include hull form, speed, floe size and concentration, ice thickness, strength and edge shape. Most prior studies have tended to focus on ice thickness and strength as the primary determinants of load. This study shows that ice edge shape and mass, along with hull form and locations are also strong determinants of loads, and especially the load statistics. The simulations provide some interesting data, especially when compared to field trials data. A related focus for the study is to explore the use of the GPU-Event-Mechanics (GEM) simulation approach. The GEM approach represents the integration of a number of concepts. The physical space is described as a set of bodies. The movement (kinematics) of the bodies is tracked using simple equations of motion. Time is divided into relatively long ‘moments’, during which events occur. All variables in the simulation; forces, movements, fractures and other changes, are considered to be aspects of events. Some events are momentary, while others are continuing. Some events involve a single body and are termed solo events. Motion, for example, is treated as a solo event. Some events are two-body events. Impact is an example of a two-body event. The GEM approach lends itself to parallel implementation, which in this case is accomplished in a GPU environment. A GPU (Graphics Processing Unit) is a common element found in modern computer graphics cards. The GPU is primarily intended for making rapid calculations associated with the display. However, special software can access the GPU and enhance the computing power available to the user. See (Daley et.al. 2012) for further discussion of GPUs. The event models are the analytical solutions of specific scenarios. As a result, the events do not require solution (in the numerical sense) during the GEM simulation. The