Thomas W. Edgar

Realizing scientific methods for cyber security

There is little doubt among cyber security researchers about the lack of rigor underlying much of the scientific literature. The issues are manifold and are well documented. Much of the problem lies with insufficient scientific methods. Cyber security exists at the frontier between the operations of machines and the behaviors and actions of users. While we inherit the challenges of computer and social sciences, we also must face a variety of new issues that are unique to cyber security. In this paper we discuss the challenges created by the need for rigorous cyber security science. We review the methods used by other sciences and discuss how they relate to cyber security. This paper is by no means comprehensive: its purpose is to foster discussion in the community on how we can improve rigor in cyber security science.

Towards an experimental testbed facility for cyber-physical security research

Cyber-Physical Systems (CPSs) are under great scrutiny due to large Smart Grid investments and recent high profile security vulnerabilities and attacks. Research into improved security technologies, communication models, and emergent behavior is necessary to protect these systems from sophisticated adversaries and new risks posed by the convergence of CPSs with IT equipment. However, cyber-physical security research is limited by the lack of access to universal cyber-physical testbed facilities that permit flexible, high-fidelity experiments. This paper presents a remotely-configurable and community-accessible testbed design that integrates elements from the virtual, simulated, and physical environments. Fusing data between the three environments enables the creation of realistic and scalable environments where new functionality and ideas can be exercised. This novel design will enable the research community to analyze and evaluate the security of current environments and design future, secure, cyber-physical technologies.

Evaluating transactive controls of integrated transmission and distribution systems using the Framework for Network Co-Simulation

With an ever-evolving power grid, concerns regarding how to maintain system stability, efficiency, and reliability remain constant because of increasing uncertainties and decreasing rotating inertia. To alleviate some of these concerns, demand response represents a viable solution and is virtually an untapped resource in the current power grid. This work describes a hierarchical control framework that allows coordination between distributed energy resources and demand response. This control framework is composed of two control layers: a coordination layer that ensures aggregations of resources are coordinated to achieve system objectives and a device layer that controls individual resources to assure the predetermined power profile is tracked in real time. Large-scale simulations are executed to study the hierarchical control, requiring advancements in simulation capabilities. Technical advancements necessary to investigate and answer control interaction questions, including the Framework for Network Co-Simulation platform and Arion modeling capability, are detailed. Insights into the interdependencies of controls across a complex system and how they must be tuned, as well as validation of the effectiveness of the proposed control framework, are yielded using a large-scale integrated transmission system model coupled with multiple distribution systems.

Applying the scientific method to cybersecurity research

The cyber environment has rapidly evolved from a curiosity to an essential component of the contemporary world. As the cyber environment has expanded and become more complex, so have the nature of adversaries and styles of attacks. Today, cyber incidents are an expected part of life. As a result, cybersecurity research emerged to address adversarial attacks interfering with or preventing normal cyber activities. Historical response to cybersecurity attacks is heavily skewed to tactical responses with an emphasis on rapid recovery. While threat mitigation is important and can be time critical, a knowledge gap exists with respect to developing the science of cybersecurity. Such a science will enable the development and testing of theories that lead to understanding the broad sweep of cyber threats and the ability to assess trade-offs in sustaining network missions while mitigating attacks. The Asymmetric Resilient Cybersecurity Initiative at Pacific Northwest National Laboratory is a multi-year, multi-million dollar investment to develop approaches for shifting the advantage to the defender and sustaining the operability of systems under attack. The initiative established a Science Council to focus attention on the research process for cybersecurity. The Council shares science practices, critiques research plans, and aids in documenting and reporting reproducible research results. The Council members represent ecology, economics, statistics, physics, computational chemistry, microbiology and genetics, and geochemistry. This paper reports the initial work of the Science Council to implement the scientific method in cybersecurity research. The second section describes the scientific method. The third section in this paper discusses scientific practices for cybersecurity research. Section four describes initial impacts of applying the science practices to cybersecurity research.

Chapter 5 – Descriptive Study

Descriptive studies focus in depth on a specific case of some system. This chapter will discuss descriptive study methods, such as case studies, surveys, and case reports, as well as providing guidance on which is the most suitable method to choose for any particular cyber security research. It also addresses data collection as it relates to observational study in general terms, e.g., the medium through which surveys/questionnaires are undertaken and how to develop specific questions in order to best obtain the data that you require. Data analysis is also discussed and the chapter ends by outlining the design and presentation of descriptive studies.

A hybrid Authentication and authorization process for control system networks

This paper presents a new authentication protocol for control systems that draws from Extensible Authentication Protocol and Kerberos. Traditional authentication schemes do not meet control system requirements of very high availability, failsafe operation, noninterruption of devices and networks, and resilience to loss of connectivity. Our hybrid protocol meets the requirements and provides device-to-device authentication both within a remote station and between remote stations and control centers.

Experiment as a service

The absence of scientific validation of results is one of the greatest obstacles in the field of cyber security. The lack of reproducible experimental environments and results is a main contributor to this problem. Scientific rigor requires that experiments be re-run to confirm their results. For the field of cyber security to progress, experiments must be run in a realistic and controllable environment. Significant advancement in cloud technology presents potential to alleviate these problems. We explore leveraging and modifying the OpenStack cloud platform to create a parameterized, repeatable, and shareable enterprise model in the context of a real experiment. Gaps and solutions are discussed to turn cloud technology into a scientific instrument.

Chapter 4 – Exploratory Study

This chapter discusses data collection as it relates to observational studies, focusing on exploratory studies—the collection, analysis, and interpretation of observations about known designs, systems, or models, or about abstract theories or subjects. The chapter introduces the different types of exploratory studies—ecological, longitudinal/cohort, cross-sectional, case-control—and cyber security examples of each are given. The form and use of gathered data is discussed along with the significance level. The chapter also explains analysis bias and introduces some of the most commonly used statistical tools for data analysis. The chapter ends by discussing the design and presentation of exploratory studies.

Introduction to Science

This chapter aims to introduce science and the way it has been used to help our understanding of the universe and everything in it, as well as to achieve societal and technological advancement. The philosophy of science, the body of knowledge of science, and the scientific process to discover knowledge will all be discussed. The chapter will provide an overview of the different branches of science, the different forms of scientific research, and the types of methods used. The chapter will discuss empirical evidence provided by scientific research methods and explain the hierarchy of evidence, as well as discussing why the scientific method requires that beliefs and preferences are subordinated to data and information. The continuum of discovery is introduced with a brief historical review of the investigations to understand the planetary motion of the solar system.

Chapter 6 – Machine Learning

This chapter aims to introduce machine learning—a field of study that uses computational algorithms to turn empirical data into usable models. Cyber security machine learning based models need to be able to represent a real-world system, infer system properties, and learn and adapt based on knowledge and observations. The chapter starts with an introduction of the concepts and techniques of machine learning, outlining the categories of machine learning—classification, clustering, regression, and anomaly detection. The chapter then explores the use of probabilistic models, such as Bayesian networks and hidden Markov models, as data driven classification/modeling strategies, with examples given.