Scott R. Graham

Fundamental Limits on Synchronizing Clocks Over Networks

We characterize what is feasible concerning clock synchronization in wireline or wireless networks. We consider a net work of n nodes, equipped with affine clocks relative to a designated clock that exchange packets subject to link delays. Determining all unknown parameters, i.e., skews and offsets of all the clocks as well as the delays of all the communication links, is impossible. All nodal skews, as well as all round-trip delays between every pair of nodes, can be determined correctly. Also, every transmitting node can predict precisely the time indicated by the receivers clock at which it receives the packet. However, the vector of unknown link delays and clock offsets can only be determined up to an (n - 1)-dimensional subspace, with each degree of freedom corresponding to the offset of one of the (n - 1) clocks. Invoking causality, that packets cannot be received before they are transmitted, the uncertainty set can be reduced to a polyhedron. We also investigate structured models for link delays as the sum of a transmitter-dependent delay, a receiver-dependent delay, and a known propagation delay, and identify conditions which permit a unique solution, and conditions under which the number of the residual degrees of freedom is independent of the network size. For receiver-receiver synchronization, where only receipt times are available, but no time-stamping is done by the sender, all nodal skews can still be determined, but delay differences between neighboring communication links with a common sender can only be characterized up to an affine transformation of the (n - 1) un known offsets. Moreover, causality does not help reduce the uncertainty set.

Time in general-purpose control systems: the Control Time Protocol and an experimental evaluation

No two clocks generally agree. Distributed control applications, however, require accurate timing information. This necessitates some form of message passing for interpretation of distributed clocks. We exhibit a fundamental indeterminacy in estimating time when delays are unequal in the two directions, necessitating an assumption of symmetry. We present the Control Time Protocol as an architecturally clean algorithm for providing timing information in the domain of distributed control. The protocol introduces no additional dependencies beyond those already present in the very control loops, thus enhancing the reliability of the systems. It has been implemented on the testbed in the Convergence Laboratory at the University of Illinois. Experimental results demonstrating how time stamping can be used to enhance stability and performance, as well as measurements of network latencies, are provided.

Abstractions, Architecture, Mechanisms, and a Middleware for Networked Control

We focus on the mechanism half of the policy-mechanism divide for networked control systems, and address the issue of what are the appropriate abstractions and architecture to facilitate their development and deployment. We propose an abstraction of ldquovirtual collocationrdquo and its realization by the software infrastructure of middleware. Control applications are to be developed as a collection of software components that communicate with each other through the middleware, called Etherware. The middleware handles the complexities of network operation, such as addressing, start-up, configuration and interfaces, by encapsulating application components in ldquoShellsrdquo which mediate component interactions with the rest of the system. The middleware also provides mechanisms to alleviate the effects of uncertain delays and packet losses over wireless channels, component failures, and distributed clocks. This is done through externalization of component state, with primitives to capture and reuse it for component restarts, upgrades, and migration, and through services such as clock synchronization. We further propose an accompanying use of local temporal autonomy for reliability, and describe the implementation as well as some experimental results over a traffic control testbed.

The convergence of control, communication, and computation

The convergence of communication and computation over the past two decades has given us the Internet. We believe that the next phase of the information technology revolution will be the convergence of control, communication, and computation. This will provide the ability for large numbers of sensors, actuators, and computational units, all interconnected wirelessly or over wires, to interact with the physical environment. We argue that in the proliferation of this “convergence,” a critical role will be played by the architecture. We describe an experimental Convergence Testbed at the University of Illinois, outline the architectural challenges, and our efforts in this direction.

Issues in the convergence of control with communication and computing: proliferation, architecture, design, services, and middleware

We anticipate that a possible next phase in the information technology revolution could be the convergence of control, i.e., sensing and actuation, with communication and computing. We address the broad set of issues that we believe to be important to the design, implementation, and proliferation of such systems. In particular, we expound on the topics of the architecture of such systems, methodologies for design, distributed time, services, and middleware. We describe our efforts in the Convergence Lab at the University of Illinois with respect to each of these topics.

A Trust System Architecture for SCADA Network Security

This paper discusses the use of a communications network security device, called a trust system, to enhance supervisory control and data-acquisition (SCADA) security. The major goal of the trust system is to increase security with minimal impact on existing utility communication systems. A previous paper focused on the technical operation of the trust system by augmenting routers to protect User Datagram Protocol (UDP)-based traffic. This paper concentrates on placing the trust system into a broader context, creates new trust system implementations to increase its flexibility, and demonstrates the trust system using TCP traffic. Specifically, the article expands on previous work in the following ways: 1) the article summarizes major threats against SCADA systems; 2) it discusses new trust system implementations, which allow the trust system to be used with a wider array of network-enabled equipment; 3) it discusses key SCADA security issues in the literature and shows how the trust system responds to such issues; 4) the paper shows the impact of the trust system when widely prevalent TCP/IP network communication is used; and 5) finally, the paper discusses a new hypothetical scenario to illustrate the protection that a trust system provides against insider threats.

Etherware: domainware for wireless control networks

The promise of middleware is to enable integration and evolution of complex systems dynamically. In demanding domains such as wireless control networks, fulfilling this promise while maintaining complete generality is extremely complicated. Understanding and exploiting the forcing functions of a domain helps manage this complexity by avoiding redundant generalizations. Domainware exploits this technique and adopts a simpler architecture to support more important nonfunctional requirements effectively. This paper presents Etherware, a domainware for wireless control networks. Capitalizing on our development of a fairly complex control system testbed, commonly supported yet redundant generalizations are identified and eliminated. The resulting architecture is simple, and can support a wide range of trade-offs that can be manipulated easily at run-time. This is illustrated by showing how the performance of control time protocol (CTP), an Etherware service, is optimized by the additional options available in Etherware

Distributed Fault-Tolerant Quality of Wireless Networks

A mobile ad hoc network (MANET) consists of a group of communicating hosts that form an arbitrary network topology by means of any of several wireless communication media. MANET communications represent a diversification in communication technology necessary to solve the stringent end-to-end requirements of QoS-based communication networks. Of the many challenges in this complex distributed system, the problem of routing based on a predefined set of customer preferences, critical to guaranteeing quality-of-service, is the focus of this research. Specifically, this paper modifies a cluster-based QoS routing algorithm for mobile ad hoc networks with the aim of providing fault tolerance, which is a critical feature in providing QoS in the link failure-prone environment of mobile networks. Performance of this new fault-tolerant cluster-based QoS wireless algorithm is evaluated according to failure recovery time, dropped packets, throughput, and sustained flow bandwidth via simulations involving node failure scenarios along QoS paths.

Service Continuity in Networked Control Using Etherware

Service continuity is the capability to provide persistent and reliable service, with graceful degradation in the presence of changes. This is particularly important for networked control systems interacting with the real world, as they have strict safety requirements and must tolerate changes such as component failures and upgrades while maintaining operational integrity. The authors describe how they address these issues in Etherware, their middleware for networked control.

A General Method of Measuring Interoperability and Describing Its Impact on Operational Effectiveness

A general method of measuring the interoperability of a heterogeneous set of systems, experiencing any type and number of interoperations, in the context of an operational process is given. Furthermore, for confrontational operations (friendly versus adversary), the method gives sufficient conditions for relating the interoperability measurement to operational effectiveness. Owing to the difficulty in creating a general method of interoperability measurement, developers of extant interoperability assessment methods have relied upon problem decomposition to produce methods of assessing the interoperability of specific types of systems experiencing distinct modes of interoperation. Unfortunately, this approach fractured the problem, effectively driving them further from the solution.Therefore, in this research, a holistic, fundamental, and flexible means of describing a general method of interoperability measurement was undertaken which models systems according to their interoperability-related features in the context of an operational process.An application of the method highlights the new concept of confrontational interoperability, demonstrates the relationship between interoperability and operational effectiveness in the context of a suppression of enemy air defenses (SEAD) scenario, and illustrates how an interoperability measurement can motivate system upgrades.