SDN4CoRE: A Simulation Model for Software-Defined Networking for Communication over Real-Time Ethernet
IIf you cite this paper, please use the original reference: T. H¨ackel, P. Meyer, F. Korf, and T. C. Schmidt. SDN4CoRE: A SimulationModel for Software-Defined Networking for Communication over Real-Time Ethernet. In:
Proceedings of the 6th InternationalOMNeT++ Community Summit . September, 2019, Easychair.
SDN4CoRE: A Simulation Model for Software-DefinedNetworking for Communication over Real-Time Ethernet
Timo H¨ackel, Philipp Meyer, Franz Korf, and Thomas C. Schmidt
Dept. Computer Science , Hamburg University of Applied Sciences , Germany { timo.haeckel, philipp.meyer, franz.korf, t.schmidt } @haw-hamburg.de Abstract
Ethernet has become the next standard for automotive and industrial automation net-works. Standard extensions such as IEEE 802.1Q Time-Sensitive Networking (TSN) havebeen proven to meet the real-time and robustness requirements of these environments. Aug-menting the TSN switching by Software-Defined Networking functions promises additionalbenefits: A programming option for TSN devices can add much value to the resilience,security, and adaptivity of the environment. Network simulation allows to model highlycomplex networks before assembly and is an essential process for the design and validationof future networks. Still, a simulation environment that supports programmable real-timenetworks is missing.This paper fills the gap by sharing our simulation model for Software-Defined Network-ing for Communication over Real-Time Ethernet (SDN4CoRE) and present initial resultsin modeling programmable real-time networks. In a case study, we show that SDN4CoREcan simulate complex programmable real-time networks and allows for testing and verifyingthe programming of real-time devices.
In recent years, Ethernet has emerged to become the next standard for automotive networks.Complementary protocols such as IEEE 802.1Q Time-Sensitive Networking (TSN) have provento meet the real-time and robustness requirements of these environments. On the other hand,the Software-Defined Networking (SDN) paradigm has revolutionized campus and data centernetworks. Separating the control from the data plane of network devices at a central controlunit with global network knowledge enables simple and fast-forwarding at devices, while high-level control applications can steer the entire network. SDN, though, cannot grant Quality-of-Service (QoS) guarantees to the forwarding plane. First results from integrating the SDNparadigm with TSN standards are promising [2]. A programming option for TSN devices canadd much value to the resilience, security, and adaptivity of the environment. Although theapplication of the combination of TSN and SDN is expected to expand with the introduction of5G and the Internet of Things, we focus on the use case in automotive networks in this work.Network simulation allows highly complex networks to be modeled before the assembly.This is an important technique for the design and validation of future networks and therefore a r X i v : . [ c s . N I] A ug DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt remains an active research topic. The discrete event simulation platform OMNeT++ suits wellas a simulation toolchain for automotive communication, as shown in prior work [10].In this paper, we present our simulation model for Software-Defined Networking for Com-munication over Real-Time Ethernet (SDN4CoRE) and describe the first results in modelingprogrammable real-time networks. SDN4CoRE is built on top of the INET framework anduses the CoRE4INET simulation models developed in previous work [7]. To make the networkdevices programmable, we implement client and server modules for the NetConf protocol andintegrate the OpenFlowOMNeTSuite [4] for OpenFlow protocol modules. In our case study, weshow how our model can be used to test and evaluate configuration mechanisms in real-timenetworks.The remainder of this paper is structured as follows. Section 2 provides background knowl-edge and related work. The concept of programming options for the different real-time Eth-ernet components is discussed in Section 3. In Section 4, we describe the components of theSDN4CoRE simulation models, followed by a case study in Section 5. Finally, Section 6 con-cludes this work with an outlook on future work.
Today, more and more software components get deployed in automotive networks, demandinga steady increase in communication bandwidth and timing guarantees. Ethernet has emergedas the next high-bandwidth communication technology for in-car networks. There have beenseveral attempts to introduce support for real-time requirements in Ethernet networks. Time-Triggered Ethernet (AS6802) provides a synchronous Time Division Multiple Access (TDMA)implementation, as well as rate-constrained traffic classes for Ethernet. Audio Video Bridg-ing (AVB) defines dynamic bandwidth reservation mechanisms for streams and traffic shapersto guarantee maximum latency. AVBs successor Time-Sensitive Networking (IEEE 802.1Q-2018 [3]) is a set of standards which are defined by the TSN task group of the IEEE. Thesestandards extend Ethernet to concurrently forward real-time and Cross-Traffic (CT). It sup-ports both synchronous TDMA traffic and asynchronous bandwidth reservation for streams.CoRE4INET [7] (Communication over Real-time Ethernet for INET) is a suite of real-timeEthernet simulation models developed over the past decade. Currently, CoRE4INET supportsthe AS6802 protocol suite, traffic shapers of Ethernet AVB, and implementations of IEEE802.1Q, as well as models to map IP traffic to real-time traffic classes.As communication requirements increase, so does the complexity and network adaptabil-ity requirements. The SDN paradigm promises to solve this problem by moving the controllogic out of the network devices into a central control unit [6]. Kreutz et al. discuss theparadigms and concepts of SDN in their comprehensive survey [5]. The network logic is splitinto three layers: (1) The data plane on which each switch forwards packets according to theflow rules, (2) the control plane on which each switch is connected to a logically (not necessarilyphysically) centralized controller that manages the forwarding logic, and (3) the managementplane on which network administrators manage the controller applications. The communica-tion between the SDN controller and the switches is specified in the OpenFlow standard of theONF [9]. The OpenFlowOmnetSuite – originally developed at the University of W¨urzburg –is an OMNeT++ simulation model that implements a concept of SDN with OpenFlow [4]. Itprovides the OpenFlow standard message types, implementations of forwarding devices, a SDNcontroller implementation, and interfaces for controller applications.In TSN, configuration of various components is enabled through the NetConf protocol(RFC 6241 [1]). It specifies a management architecture in which the managed unit contains a2
DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt
NetConf server, and the administrative unit connects through a NetConf client. On the server-side, configurations are stored in configuration data stores. NetConf provides Remote ProcedureCall (RPC) operations such as get-config and edit-config , which allow the client to request oredit the configuration data stores at the server. SDN4CoRE provides a base implementation ofthe NetConf protocol, including client, server, data stores, and operations.Rather little work has been done to combine the concepts of TSN and SDN. Nayak et al.mention the term ”Time-Sensitive Software-Defined Network” in 2016 for the first time andshow a concept of a programmable scheduler [8]. In previous work, we introduced a conceptof software-defined networks supporting time-sensitive in-vehicle communication [2] along witha switching methodology for TSN streams, the programming of the bandwidth reservation viathe Stream Reservation Protocol (SRP), and a detailed analysis.To the best of our knowledge, no simulation environment supports programmable real-timenetworks. Making the modules of CoRE4INET programmable allows using other availablesimulation models as well, such as CAN bus signals and gateways to model realistic in-vehicularcommunication [7].
Introducing a programming option for real-time Ethernet devices that improves resilience, se-curity, and adaptability of the environment requires changes on all three layers of the SDNconcept: Forwarding devices must provide a programming interface using open standards, con-trol functionality for real-time must be extracted from the switches and integrated into the SDNcontroller, and controller applications must be able to program and manage real-time devices.In real-time Ethernet and SDN, switches contain additional modules to extend the func-tionality of regular switching hardware, which is depicted in Figure 1. One of the additionalmodules introduced in some real-time communication is an ingress control module. In TSN thisis the “Per-Stream Filtering and Policing” module. It is used to filter incoming Ethernet framesfor controlling bandwidth and arrival times. A similar module is used at the egress of a switchand implements in particular priority queuing, real-time scheduling and traffic shaping. In TSNthis module is called “Enhancements for Scheduled Traffic” and specified in IEEE 802.1Qbv. Ituses a Gate Control List (GCL) to indicate, which 802.1Q priorities are allowed to pass througha particular port at a specific time. The scheduling information is stored in the “Schedule” tableand needs a precisely synchronized time at all network devices, which is managed by the “TimeSync” module. For stream-based bandwidth reservation, the “SR Table” module contains allregistered talkers and listeners for time-sensitive streams.In a SDN switch, the forwarding module of a standard Ethernet switch is replaced by aflow-based forwarding module that performs flow table lookups based on packet match rules.The SDN controller performs tasks such as topology discovery, MAC-Address learning, androute determination. The programming interface between the switch and the SDN controller isan open southbound API that implements standard protocols such as NetConf or OpenFlow.Additional network applications can be executed on top of the controller.In programmable real-time networks, flow-based operations require merging the modules ofthe real-time and the SDN switching components. To ensure that the real-time capabilitiesare not altered in any way, the ingress and egress control modules must remain unchanged bythe additional programming option. When data packets arrive, the ingress control managesthe timing and applies stream-based filters. After that, the packet is matched against the flowtable, and the discovered actions are executed. The packet then gets forwarded to the correctegress ports, where the egress control manages the timing and shaping of the outgoing traffic. If3
DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt
Ingress
Ports
Per-Stream
Filtering and Policing
Flow-based
Forwarding
Enhancements for
Scheduled Traffic
Egress
PortsSR Table
Flow TableSchedule
Time SyncTime Sync
Control Plane
SDN Controller
Forwarding Plane
Open Southbound API
Management Plane
Network Applications
Open Northbound API
Programming Interface
TSNLegacy switch SDN
Legend:
Figure 1:
Components of a programmable real-time switch. no corresponding forwarding rule exists, the packet is dropped by default, while most controllerapplications insert the default rule to forward the packet to the controller.SDN controller applications manage the forwarding and store information for the controlfunctionality of the SDN devices. This usually includes features such as neighbor discoveryor source MAC address learning. Because many real-time Ethernet protocols use multicastMAC addresses to transfer data to a group of receivers, the controller applications must beable to learn and control multicast MAC addresses. AVB and TSN use the SRP to announcethe sender (talker) and receivers (listener) of the multicast groups. Therefore, the controllerapplication must receive and understand the SRP packets. When a forwarding device receivesa SRP message, it is forwarded directly to the SDN controller via the OpenFlow protocol. Thecontroller application then registers the talker or listener in its SR table and sends the SRPmessage back to the switch via the OpenFlow protocol. The switch updates its SR table andforwards the message to the next hop. In this way, the ‘talker advertise’ and ‘listener ready’messages are spread across the network, and each switch goes through the same process untilthe messages reach the clients. When a new client subscribes to a stream with a ‘listener ready’message, a forwarding rule for the TSN stream is inserted in the switches flow table before the‘listener ready’ command is sent back to the switch and forwarded along the path. In a futurerelease, the modules handling the SRP protocol could be removed from the switches to simplifythem. The controller could insert a match rule for the SRP messages to receive them directly.Subsequently, the SRP tables could then be updated directly via the NetConf protocol.Besides the dynamic stream reservation, the controller must be able to control the scheduledtraffic. Therefore, the switch must provide a specific NetConf data store to control the schedul-ing tables. In this work, we implemented such a data store for the gate control of 802.1Qbv.To change the GCL, the controller sends an ‘edit config’ message containing updates or a fullconfiguration for the active GCL. The NetConf server module forwards those messages to thedata store which then updates the GCL in the corresponding port. In a future release, addi-tional modules may be added to the data store. For example, the controller could program the802.1Qci filters, gates, and meters. On the other hand, algorithms to calculate a new schedule4
DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt
Figure 2:
OMNeT++ Module of an OpenFlow and NetConf capable switch. for the entire network could be implemented to adapt the schedules dynamically.
The SDN4CoRE simulation module is based on the discrete event simulator OMNeT++( https://omnetpp.org/ ) and the INET framework ( https://inet.omnetpp.org/ ). It usesthe CoRE4INET [11] simulation model for real-time Ethernet communication and extends somemodules to be programmable. On the other hand, SDN4CoRE uses the OpenFlow protocolimplemented in the OpenFlowOMNeTSuite [4]. We forked and extended this suite to harmo-nize with our models. For example, we introduced interface modules to enhance flexibility andmodularity and created a new flow table structure.Figure 2 shows the general module structure of a forwarding device that is capable ofOpenFlow and NetConf. The eth0 port connects the switch to the controller. The NetConfserver module is connected to this port via TCP, but can also be connected to other protocols,e.g. for security reasons. It handles the NetConf protocol and extracts RPCs to forward themto the data store manager, which in turn applies the command on the proper data store andcreates the response. The OpenFlow protocol client is implemented in the relay unit and alsoconnected via TCP. Besides, the relay unit performs the flow table lookup in the flow tablesand forwards incoming data packets to the correct Ethernet port. For the etherMAC interfaces,we provide several real-time Ethernet port implementations that control the ingress and egresstimings. Some of them require the Time Synchronization and Scheduler modules, which alsocontribute a realistic device clock. The stream reservation table and protocol modules areneeded for AVB/TSN streams. The switch implementation also provides the option to importor export a launch configuration for all modules and tables.The SDN controller module is based on a standard host provided by the INET frameworkso that other applications can run smoothly. The OpenFlow controller module from the Open-FlowOMNeTSuite and the NetConf client module connect to the forwarding devices via TCP.The NetConf client module handles connectivity to the forwarding devices and forwards RPCrequests and replies to the corresponding NetConf application or server. We include a baseimplementation for NetConf and OpenFlow applications as well as specific implementation forreal-time controllers. With this, it is possible to create controller apps that use NetConf andOpenFlow simultaneously. The controller also provides the ability to import or export a launchconfiguration. 5
DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt
SDN ControllerSwitch 1 (S1) SinkSwitch 2 (S2)Host 1Host 2Host 3Host 4Priority: 1
Cycle: uniform (0.2ms-0.8ms)Framesize: 1522 Byte
Priority: 2
Cycle: 1ms
Framesize: 1522 Byte
Priority: 6
Cycle: 1msFramesize: 122 BytePriority: 7Cycle: 1ms (Starts at 4s)Framesize: 122 Byte Gate Control List (t=0s):C,C,C,C,C,C,o,o: 0.015 mso,o,o,o,o,o,C,C: 0.860 ms
C,C,C,C,C,C,C,C: 0.125 ms
Gate Control List (t=0s):C,C,C,C,C,C,o,o: 0.015 mso,o,o,o,o,o,C,C: 0.860 ms
C,C,C,C,C,C,C,C: 0.125 ms
Figure 3:
Network topology and device configurations at startup.
Our results in previous work show that SDN control overhead can be added to TSN streamswithout a delay penalty, provided protocols are mapped properly [2]. In this work, we analyzethe programmability of the GCL in switches supporting IEEE 802.1Qbv. Since this articlefocuses on programming rather than calculating correct schedules, we have disabled clock jitterand time synchronization for all devices to simplify the simulation and calculation of schedules.The network topology of our case study consists of four hosts sending messages to one sinkvia two switches in the presence of one SDN controller as shown in Figure 3. Each host sendsone periodic IEEE 802.1Q message, so the traffic shaping is controlled by the switches in thenetwork. These messages have a fixed priority, frame size, and cycle. As an exception, host1 sends low priority messages with high frequency that varies according to gaussian normaldistribution. To introduce a change in high priority traffic, host 4 starts transmitting frameswith the highest priority after 4 seconds. The two switches are NetConf and OpenFlow capable.The output on each switch port is controlled by an 802.1Qbv module, which contains gates foreach 802.1Q priority that can either be closed (C) or opened (O) at a certain point in time.This information is stored in the GCL that can be programmed at run time using the NetConfprotocol. In this evaluation we use three phases for this list: A time slot T red , closing all gatesto ensure that the transmission of previous frames has ended before switching to the greenphase; A time window T green , for high priority traffic with priorities 6-7; And a window T yellow ,for all low priority traffic of priorities 0-5.The cycle time of the schedule is equivalent to the largest cycle time of the high prioritymessages, which is 1 ms in our scenario. We calculated the phases and added a safety margin: T red = T MaxF rametx + T margin = 1 , Byte
M bit/s + 5 µs = 121 , µs + 5 µs ≈ µs (1) T green = T Host tx + T margin = 122 Byte
M bit/s + 5 µs = 9 . µs + 5 µs ≈ µs (2) T yellow = T cycle − T green − T red = 1 ms − µs − µs = 860 µs (3)Figure 4 is a graph showing the changes in end-to-end latency during 10 s simulation time.These changes are introduced by updates to the configuration of the GCL depicted in Table 1and changes in the traffic pattern.At the beginning of the simulation all devices are configured as shown in Figure 3. The highpriority packets from host 3 to the sink have a constant delay of 1.03 ms (see Figure 4). In the6 DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt .
51 Simulation time [s] E nd - t o - e nd l a t e n c y [ m s ] Host 3 → Sink (PCP 6)Host 4 → Sink (PCP 7)
Figure 4:
The end-to-end latency of high prior-ity data flows from the host to the sink. Host 4starts sending after 4 s simulation time. t( s ) Dev Gate Control List ( µ s )0-2 S1 G: 15 ; Y: 860 ; R: 125S2 G: 15 ; Y: 860 ; R: 125
S1 R: 10 ; G: 15 ; Y: 860 ; R: 115S2 R: 20 ; G: 15 ; Y: 860 ; R: 105
S1 R: 10 ; G: 15 ; Y: 860 ; R: 115S2 R: 20 ; G: 15 ; Y: 860 ; R: 105
S1 R: 10 ; G: 30 ; Y: 845 ; R: 115S2 R: 20 ; G: 15 ; Y: 860 ; R: 105
S1 R: 10 ; G: 30 ; Y: 845 ; R: 115S2 R: 20 ; G: 30 ; Y: 845 ; R: 105
Table 1:
Changes to the 802.1Qbv Gate ControlList of the switches (S1 and S2) with the threephases green (G), yellow (Y) and red (R). simulation, we can see that this delay is introduced at the second switch, where the packetsmiss their time slot because of the transmission time of 20 µs over two links. To eliminate thisdelay, the controller updates the GCL of both forwarding devices at 2 s simulation time. Thered phase is split into two parts, one at the beginning of the cycle and one at the end. Thered phase at the beginning is exactly the transmission time of the high priority packet to thedevice. For switch 1 this is the transmission delay of 10 µs from host 3 to switch 1. For switch 2this phase has to be shifted another 10 µs for the transmission from switch 1 to switch 2. Thisupdate reduces the end-to-end latency to the expected value of 30 µs .At 4 s simulation time, host 4 starts sending frames with the same size and cycle but a higherpriority than those of host 3. Therefore, the transmission time of high priority frames at theswitches doubles. The packets of host 4 have the expected end-to-end latency of 30 µs . Thoseof host 3 stay in time for their slot at switch 1 because of the safety margin of 5 µs . At switch2, however, they miss their time slot and need to wait for one cycle. To solve this time conflict,the controller updates the GCLs at 6 s simulation time, but only for switch 1 to simulate afailure in the configuration of switch 2. Since the update was unsuccessful on switch 2, the end-to-end latency remains unchanged. Although the configuration of switch 2 was unsuccessful,the update of the configuration of switch 1 was still performed. In very sensitive environmentsthis can lead to dangerous behavior of the network. Additional mechanisms are required toensure that changes to the schedule are executed at the same time and are consistent across thenetwork. At 8 s simulation time, the controller performs the update of switch 2. This reducesthe end-to-end latency of host 3 as expected.This initial evaluation shows how the programming of schedules in the network can in-crease the flexibility and performance of the system. On the other hand, it indicates thatre-configuration of time slots should be used with care during runtime. Configuring multipledevices might lead to an inconsistent state of the network and loss or delay of critical messages. In this paper, we shared our simulation model SDN4CoRE and described in detail how tomake real-time communication programmable with NetConf and OpenFlow. In a case study,we demonstrated the use of our model for testing and evaluating configuration mechanisms in7
DN4CoRE Simulation Model H¨ackel, Meyer, Korf and Schmidt real-time networks. An initial assessment indicates that re-configuration of time slots should beused with care during runtime, and additional mechanisms are required to ensure a consistentstate across the network.In future work, we plan to further extend the programmability of the CoRE simulationmodels, for example, with the recently published implementation of IEEE 802.1Qci modules.The evaluation of control plane concepts for automotive networks is one of our next tasks.The SDN4CoRE simulation model, including its original fork of the OpenFlowOMNeTSuite,as well as all other simulation models and analyses tools, are published as open-source on ourGitHub page at https://github.com/CoRE-RG/ . Acknowledgments
This work is funded by the Federal Ministry of Education and Research of Germany (BMBF)within the SecVI project.
References [1] Rob Enns, Martin Bj¨orklund, Andy Bierman, and J¨urgen Sch¨onw¨alder. Network ConfigurationProtocol (NETCONF). RFC 6241, IETF, June 2011.[2] Timo H¨ackel, Philipp Meyer, Franz Korf, and Thomas C. Schmidt. Software-Defined NetworksSupporting Time-Sensitive In-Vehicular Communication. In
Proc. of the IEEE 89th VehicularTechnology Conference: VTC2019-Spring , Piscataway, NJ, USA, April 2019. IEEE Press.[3] Institute of Electrical and Electronics Engineers. IEEE Standard for Local and Metropolitan AreaNetwork–Bridges and Bridged Networks. Standard, IEEE, July 2018.[4] Dominik Klein and Michael Jarschel. An OpenFlow extension for the OMNeT++ INET frame-work. In
Proceedings of the 6th International ICST Conference on Simulation Tools and Tech-niques , pages 322–329. ICST (Institute for Computer Sciences, Social-Informatics and Telecom-munications Engineering), 2013.[5] D. Kreutz, F. M. V. Ramos, P. E. Verssimo, C. E. Rothenberg, S. Azodolmolky, and S. Uhlig.Software-Defined Networking: A Comprehensive Survey.
Proceedings of the IEEE , 103(1):14–76,Jan 2015.[6] Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rex-ford, Scott Shenker, and Jonathan Turner. OpenFlow: enabling innovation in campus networks.
ACM SIGCOMM Computer Communication Review , 38(2):69–74, 2008.[7] Philipp Meyer, Franz Korf, Till Steinbach, and Thomas C Schmidt. Simulation of mixed criticalin-vehicular networks. In
Recent Advances in Network Simulation , pages 317–345. Springer, 2019.[8] Naresh Ganesh Nayak, Frank D¨urr, and Kurt Rothermel. Time-sensitive Software-defined Network(TSSDN) for Real-time Applications. In
Proceedings of the 24th International Conference on Real-Time Networks and Systems , RTNS ’16, pages 193–202, New York, NY, USA, 2016. ACM.[9] Open Networking Foundation. OpenFlow Switch Specification. Standard ONF TS-025, ONF,2015.[10] Till Steinbach, Hermand Dieumo Kenfack, Franz Korf, and Thomas C. Schmidt. An Extensionof the OMNeT++ INET Framework for Simulating Real-time Ethernet with High Accuracy. In
SIMUTools 2011 – 4th International OMNeT++ Workshop , pages 375–382, New York, USA,March 21-25 2011. ACM DL.[11] Till Steinbach, Philipp Meyer, Stefan Buschmann, and Franz Korf. Extending omnet++ towardsa platform for the design of future in-vehicle network architectures. In Anna Foerster, Vladim´ırVesely, Antonio Virdis, and Michael Kirsche, editors,
Proceedings of the 3rd OMNeT++ Commu-nity Summit, Brno, Czech Republic, September 15, 2016 . ArXiv e-prints, September 2016.. ArXiv e-prints, September 2016.