Dennis Andersson

Empirical Analysis of System-Level Vulnerability Metrics through Actual Attacks

The Common Vulnerability Scoring System (CVSS) is a widely used and well-established standard for classifying the severity of security vulnerabilities. For instance, all vulnerabilities in the US National Vulnerability Database (NVD) are scored according to this method. As computer systems typically have multiple vulnerabilities, it is often desirable to aggregate the score of individual vulnerabilities to a system level. Several such metrics have been proposed, but their quality has not been studied. This paper presents a statistical analysis of how 18 security estimation metrics based on CVSS data correlate with the time-to-compromise of 34 successful attacks. The empirical data originates from an international cyber defense exercise involving over 100 participants and were collected by studying network traffic logs, attacker logs, observer logs, and network vulnerabilities. The results suggest that security modeling with CVSS data alone does not accurately portray the time-to-compromise of a system. However, results also show that metrics employing more CVSS data are more correlated with time-to-compromise. As a consequence, models that only use the weakest link (most severe vulnerability) to compose a metric are less promising than those that consider all vulnerabilities.

Architectures for cognitive radio testbeds and demonstrators — An overview

Wireless communication standards are developed at an ever-increasing rate of pace, and significant amounts of effort is put into research for new communication methods and concepts. On the physical layer, such topics include MIMO, cooperative communication, and error control coding, whereas research on the medium access layer includes link control, network topology, and cognitive radio. At the same time, implementations are moving from traditional fixed hardware architectures towards software, allowing more efficient development. Today, field-programmable gate arrays (FPGAs) and regular desktop computers are fast enough to handle complete baseband processing chains, and there are several platforms, both open-source and commercial, providing such solutions. The aims of this paper is to give an overview of five of the available platforms and their characteristics, and compare the features and performance measures of the different systems.

Measuring team effectiveness in cyber-defense exercises: a cross-disciplinary case study

In 2010, IT-security experts from northern European governments and organizations gathered to conduct the first of a series of NATO-led cyber-defense exercises in a pilot attempt of training cyber defense. To gain knowledge on how to assess team effectiveness in cyber-defense exercises, this case study investigates the role of behavioral assessment techniques as a complement to task-based performance measurement. The collected data resulted in a massive data set including system logs, observer reports, and surveys. Six different methods were compared for feasibility in assessing the teams’ performance, including automated availability check, exploratory sequential data analysis, and network intrusion detection system attack analysis. In addition, observer reports and surveys were used to collect aspects relating to team structures and processes, aiming to discover whether these aspects can explain differences in effectiveness. The cross-disciplinary approach and multiple metrics create possibilities to study not only the performance-related outcome of the exercise, but also why this result is obtained. The main conclusions found are (1) a combination of technical performance measurements and behavioral assessment techniques are needed to assess team effectiveness, and (2) cyber situation awareness is required not only for the defending teams, but also for the observers and the game control.

Enabling organizational learning from rescue operations

Purpose – The purpose of this paper is to explore the re-development process of the Swedish Rescue Services Incident Reporting System from an organizational learning perspective with the purpose to suggest what is needed to enable long-term learning from rescue operations. Design/methodology/approach – The study is carried out as a case study relying on interviews, participant observation and workshop methods. The study case is the Swedish Incident Reporting System. Findings – The objectives expressed by the central agency leading the studied process aimed at implementing double-loop learning objectives by revising the incident reports and to improve future operations accordingly. In practice this objective was lost along the way, with the agency focussing on cosmetic changes to the report such as terminology, attributes and labels. Meanwhile the local rescue services expressed different and concrete needs, requiring new system functionality, case/experience based learning, process improvements and organi...

Extending the attribute explorer to support professional team-sport analysis

Advances in interactive systems and the ability to manage increasing amounts of high-dimensional data provide new opportunities in numerous domains. Information visualization techniques are especially useful in situations where analysts seek patterns and information of interest in massive data sets. In this article, we propose an extension of the original Attribute Explorer (AE) technique by Spence and colleagues to take on the challenges presented in the domain of professional team-sport analysis. We describe the implementation of an extended AE and use football game-event data to highlight the new possibilities.

Approaches to team performance assessment: a comparison of self-assessment reports and behavioral observer scales

Human factors research popularly employs perception-based techniques to investigate team performance and its dependency to cognitive processes. Such studies frequently rely upon either observer-based or self-assessment techniques to collect data. In this study, we examined behavioral observer ratings and self-assessment ratings for measuring team performance in virtual teams, with team performance regarded as a combination of task outcome and team cognition. Juxtaposing self-assessments and observer ratings from a quasi-experiment comparing team performance rating techniques reveals that they indeed produce overall similar results, with both singling out teamwork effectiveness ratings as the strongest contributor to overall team performance. However, the comparisons show remarkably low correlation on individual questionnaire items. The most striking difference is that the team members’ self-assessments of workload are lower than the corresponding observer ratings. In particular, the self-assessments do not correlate at all with overall team performance, whereas the observers’ workload ratings are more consistent with contemporary research that suggests a strong correlation between workload and team performance, suggesting that observer-based techniques are more reliable than self-assessments for assessing workload. For other ratings, the results show that the two techniques are fairly equal, suggesting that the choice between methods to employ can be deferred to other considerations such as obtrusiveness, accessibility, and resource availability.

Computer-Aided Football Training: Exploiting Advances in Distributed Tactical Operations Research

Professional team-sport training involves demanding work. In this discussion paper we explore the possibilities and challenges involved in adapting an existing analysis approach from the domain of Distributed Tactical Operations to the domain of professional team sports. In particular, we outline how advanced computer-supported data collection and data presentation could contribute to a general football training system.

Tactical Reconnaissance Using Groups of Partly Autonomous UGVs

This paper investigates how one operator can control a multi-robot system for tactical reconnaissance using partly autonomous UGVs. Instead of controlling individual UGVs, the operator uses supervisory control to allocate partly autonomous UGVs into suitable groups and define areas for search. A state-of-the-art pursuit-evasion algorithm then performed the detailed control of available UGVs. The supervisory control was evaluated by allowing subjects to control either six or twelve UGVs for tactical reconnaissance along the route of advance for a convoy traveling through an urban environment with mobile threats. The results show that increasing the number of UGVs improve the subjects situation awareness, increase the number of threats that are detected, and reduce the number of hits on the convoy. More importantly, these benefits were achieved without any increase in mental workload. The results support the common belief in autonomous functions as an approach to reduce the operator-to-vehicle ratio in military applications.

Assessing Resilience and Agile Capability in Socio-technical Systems

This poster presents work-in-progress aiming to create a tool for assessing resilience and agility in socio-technical systems. The tool is organized in four sections; system goal and context, ability to detect deviating events, ability to cope with deviating events, and agile C2 capability. The intended area of use is naval military forces.