Transport Services: A Modern API for an Adaptive Internet Transport Layer
Michael Welzl, Safiqul Islam, Michael Gundersen, Andreas Fischer
11 Michael Welzl, Safiqul Islam, Michael Gundersen, Andreas Fischer: ”Transport Services:A Modern API for an Adaptive Internet Transport Layer”, accepted for publication, IEEECommunications Magazine, April 2021.
Version accepted for publication in IEEE Communications Magazine. ©2021 IEEE. Personal use of this material is permitted. Permission fromIEEE must be obtained for all other uses, in any current or future media,including reprinting/republishing this material for advertising or promo-tional purposes, creating new collective works, for resale or redistributionto servers or lists, or reuse of any copyrighted component of this work in other works. a r X i v : . [ c s . N I] F e b Transport Services: A Modern API foran Adaptive Internet Transport Layer
Michael Welzl, Safiqul Islam, Michael Gundersen, and Andreas Fischer
Abstract —Transport services (TAPS) is a working group of theInternet’s standardization body, the Internet Engineering TaskForce (IETF). TAPS defines a new recommended API for theInternet’s transport layer. This API gives access to a wide varietyof services from various protocols, and it is protocol-independent:the transport layer becomes adaptive, and applications are nolonger statically bound to a particular protocol and/or networkinterface. We give an overview of the TAPS API, and wedemonstrate its flexibility and ease of use with an example usinga Python-based open-source implementation. I NTRODUCTION
Previously, it was common to regard the Berkeley SoftwareDistribution (BSD) Socket interface (also known as “Berkeleysockets”) as the standard API for the transport layer. Sincethe 1980’s, it has been the best known way to access the twocommonly available transport protocols: TCP and UDP. Nowa-days, however, more protocols and mechanisms are available,offering a much richer set of services — Multipath TCP(MPTCP) can intelligently utilize multiple network paths [1];QUIC can multiplex independent data streams to avoid head-of-line blocking delay (among many other things) [2]; LowExtra Delay Background Transport (LEDBAT) is a congestioncontrol mechanism that enables “background” communicationwhich gets out of the way of “foreground” traffic [3].Today, these protocols and mechanisms are implementedand used by industry giants: MPTCP is implemented in iOS,and used by Apple in support of their applications “Siri”and “Apple Music” [4]; QUIC is implemented and used inGoogle’s Chrome browser [2]; a variant of LEDBAT is imple-mented and used by Microsoft for Operating System updates,telemetry, and error reporting [5]. Such in-house developmentof both the protocols and their use cases is very labour-intensive (and hence costly), leaving an important questionunanswered: how can “smaller” users, such as small andmedium-sized enterprises (SMEs), access these new servicesand benefit from them? Enter TAPS: The Transport Services working group (TAPSWG) of the Internet Engineering Task Force (IETF) intends toeliminate the static compile-time binding between applicationsand transport protocols, allowing each one of these elementsto evolve independently — and in doing so, it can put an
Michael Welzl and Safiqul Islam are with the Department of Informatics,University of Oslo. E-mail: { michawe, safiquli } @ifi.uio.no.Michael Gundersen is with Bekk. E-mail: [email protected] Fischer is with the Fakult¨at Angewandte Informatik, TechnischeHochschule Deggendorf. E-mail: andreas.fi[email protected]. MPTCP is now also supported by Network.Framework, which we willdiscuss later.
Application
Protocol implementation; choosing + configuring a protocol; testing for availability, falling back; interface choice; QoS; ...
Transport Layer
Protocol implementation; choosing + configuring a protocol; testing for availability, falling back; interface choice; QoS; ...
Application Transport Layer
Traditional transport layer usage Transport layer usage with TAPS
Static, simple Dynamic, a bit more complex
Fig. 1.
TAPS relocates the code to implement a protocol and/or chooseand configure protocols, test for availability, etc., into the transportlayer. end to the situation of growing unfairness between “big” and“small” developers. The basic idea, as shown in Fig. 1, is tomove functionality out of the applications, into a commontransport layer implementation (which resides in an OperatingSystem or a library), and to enable access to these functionsvia a new API. This has the potential to empower “smallerplayers” with a much richer set of services than ever before,but without the need to invest a huge amount of manpowerinto the development of a custom-made protocol.By offering a modern API that follows an event-drivenprogramming model, TAPS is also an effort to define a newstandardized interface to the transport layer for programmersthat is easy to use. Compared to BSD sockets, which representan old fashioned, heavily C-influenced low-level programmingstyle, this should lower the entry barrier for network program-mers seeking more services than common higher-level APIssuch as an HTTP-based REST interface can offer.We will now give an overview of the new Transport Servicesconcept and how it affects the way network code is written;then, we will discuss available implementations of TAPS-conforming transport systems. Finally, we will use a Pythonexample from the open-source implementation “NEATPy” toshow the flexibility and ease of use of this new API.T
RANSPORT S ERVICES (TAPS) O
VERVIEW
As Fig. 1 illustrates, providing a flexible transport layer withinterchangeable protocols and a run-time choice of networkinterfaces and protocol configurations requires a somewhatsophisticated machinery that dynamically and intelligentlyutilizes the available infrastructure. This machinery needs totake care of: • Protocol racing:
Peer and path support for protocolsneeds to be actively tested. Intelligent caching strategies should be used to limit such probing to save time andreduce server overhead. • API-protocol mapping: finding the matching functionalityto support API calls can be a simple matter of callingfunction 𝑋 when request 𝑌 is made, but it can also entaila more complicated use of underlying protocols. • System policy management: an application may expressa wish to use a certain network interface — yet, forexample, smart phones commonly give the user system-wide control over the choice of the WiFi and cellularnetwork interfaces. System controls are normally ex-pected to overrule per-application preferences. Interfacechoice is only one example of a system policy that mayneed to interact with an application preference; clearly,a richer API that offers applications a wealth of networkmechanisms to choose and configure must meaningfullyinteract with the underlying system’s policy manager.This article focuses on the API. Thus, we refer to relatedwork for further details on “under the hood” functionality. Ref-erence [6] gives an overview of the internals of the “NEATPy”implementation that we will discuss later, and general TAPSimplementation guidance can be found in [7].
Motivation
We must first understand why there is a need for an APIchange at all. For example, using the Stream Control Trans-mission Protocol (SCTP), it is possible to transparently mapTCP connections between the same end hosts onto streamsof a single association (SCTP’s term for a connection) [8].This allows applications to benefit from a new protocol featurewithout changing the API (multi-streaming, which is availablein SCTP and QUIC, reduces the signaling overhead and allowsmultiple data streams to jointly be controlled by a singlecongestion control instance). There are, however, limits to thegains that can be obtained in such a way — some protocolmechanisms must be exposed in an API.
Head-of-line blocking example:
Consider an online multi-player game that needs to reliably transfer position updates.The game may not care about the order of these updates,but they are latency-critical. Now, let us assume that thisapplication uses TCP, and that every application data chunkfits inside exactly one TCP segment (this may be an unre-alistic simplification for position updates in a game, whichare typically very small, but the same arguments hold if aTCP segment contains multiple application data chunks). Thissituation is shown in Fig. 2: here, four application data chunksare transmitted as TCP segments 1-4, and segment 2 arriveslate (e.g., it was lost and retransmitted). Since TCP deliversdata to the application as a consecutive byte stream, chunks3 and 4 cannot be handed over; they have to wait in the TCPreceiver buffer until segment 2 arrives.Clearly, our game application could better be supported bya transport protocol that can hand over messages out of order— but, most importantly, the protocol would need to be toldabout the size of messages, the requirement of reliable delivery,and the possibility to accept messages out-of-order . A drop-in TCP replacement below the BSD socket API could never
Chunk 2 Chunk 3 Chunk 4 Chunk 1
TCP Segment
3 1
App. chunk
Fig. 2.
Application data chunks arriving at the TCP receiver in thewrong order. TCP segment 2, carrying chunk 2, arrives late, delayingthe application’s reception of chunks 3 and 4. hand over chunks 3 and 4: this would not be in line with theinterface’s expected behavior, and the application might fail.However, falling back to TCP (in case a different protocolis not available) would work: if an API allows out-of-orderdelivery, yet TCP is used below, then ordering will be ensuredat the cost of efficiency. Ordered delivery is never incorrect,it may just be slower — and, in line with the Internet’s “besteffort” service model, efficiency is not guaranteed .This example has shown us that a better transport layer must offer some services beyond the well-known two: reliable bytestream (TCP) and unreliable datagram (UDP). These servicesmust be reflected in all APIs in order to be usable: if, say, thesocket API is extended to support this functionality, yet anapplication uses a communication library on top which onlyoffers a consecutive byte stream, then, again, there is no way tobenefit from unordered reliable message delivery. This meansthat upgrading the transport layer is not “merely” an APIchange — it is a new way of thinking about communication.
API Overview
Using BSD sockets requires working in a C-oriented low-level programming style of the 1980’s (a socket has to beactively polled for incoming traffic, error codes are returnedas integer values, etc.). This has contributed to a shift towardsusing other communication libraries or middle-ware systemsthat are built on top of BSD sockets. The services that suchupper layers can expose are, however, inevitably limited by theunderlying services (TCP and UDP). Hence, when changingthe transport layer one should take the opportunity for a much-needed modernization of the interface.Accordingly, the design of TAPS follows a more modernparadigm of network communication. It is fully asynchronous,event-driven, and easy to use: rather than distinguishing be-tween a “stream” and “datagram” communication model, inTAPS, all data are transferred in the form of messages, andall communication is connection oriented. A “connection” isdefined as “shared state of two or more end systems thatpersists across messages that are transmitted between theseend systems”; under the hood, a TAPS connection may berealized by UDP datagrams or TCP connections.
Control flow:
Communication begins with making decisionsabout the remote end to communicate with, specified in anyway that is suitable (e.g., a DNS name, or “any”, whenlistening), as well as transport properties and security parame-ters. Then, a “preconnection” is created. All of this shouldbe done as early as possible, to give the transport systemthe necessary information to efficiently race protocols. Most
Preconnection Connection
Closed
Close()Abort()
Listener
Message
Receive()Send()InitiateWithSend()Initiate()Rendezvous () Connection Received Connection Ready
Pre-Establishment Established Termination
Listen()NewPreconnection()
Fig. 3.
Lifetime of a connection provided by a TAPS transport system(redrawn from [9]). transport properties to be used at this stage have a “preference”qualifier, with possible values require , prefer , ignore , avoid or prohibit . Require and prohibit should be used with care, asthey limit the system’s flexibility. Transport properties conveyservice requests, such as the use of a protocol that supportsreliability, possibly configurable per message, and being ableto use “0-RTT” session establishment (i.e., sending the firstmessage without a preceding handshake).Event handlers must be registered before connecting orlistening. Similarly, if they are used, framers must be addedto the preconnection at this time. Framers are functions thatan application can define to translate data into application-layer messages; these functions will intercept all send() andreceive() calls on the connection. In this way, an applicationcan define its own message delineation (typically a protocolheader — e.g., the HTTP header, in case of an HTTP/TAPSimplementation) that will allow the transport system to handlemessages even when the underlying protocol is TCP.Then, a connection is established by calling either the“Initiate” or “InitiateWithSend” primitive associated with thepreconnection (or “Listen”, in case of a server). The semanticsof “Initiate” are slightly different from the traditional “con-nect” and “accept”: calling “Initiate” is not guaranteed toinvoke a “ConnectionReceived” event (the TAPS equivalentof “accept”) at the peer — for example, in case of UDP,“ConnectionReceived” occurs when the first message arrives.Data are always transferred as messages. Each messagemay have associated properties to express requirements suchas a lifetime, reliability, ordering, etc. Connections also haveproperties that can be changed while they are active — forexample, a capacity profile, which can influence the value ofthe DiffServ CodePoint (DSCP) in the IP header.On the sender side, it is possible to execute some levelof control over the send buffer because a “sent” event isfired when a send action has completed (i.e., the messagehas been passed to the underlying protocol stack), and these“sent” events can be used to steer data transmission — forexample, allowing only one message to be buffered at a timeby issuing one “send” per “sent”. Applications that focus ontraffic that is not latency critical may simply ignore these“sent” events. On the receiver side, it is necessary to avoidbeing overwhelmed by too many quickly firing “received”events. This is achieved via the “receive” call, which queuesthe reception of a message; the system guarantees a one-to-onemapping between “receive” calls and “received” events. Closing a connection is also not guaranteed to invoke anevent at the peer: in case of TCP, it will, but in case ofUDP, it will not. Half-closed connections (as with TCP) arenot supported because not all protocols support them (e.g.,SCTP does not), and supporting half-closed connections wouldtherefore prohibit the use of these protocols. Figure 3 gives ahigh-level overview of the control flow (connection lifetime)that we have just described.
Cloning:
For new connections that are established to analready connected peer, it is recommended to use the “Clone”primitive with the ongoing connection. A successful “Clone”call will yield a new connection that is “entangled” withthe existing one; changing a property on one of them willaffect the other. This continues similarly for all connectionsof a group that is formed by calling “Clone” multiple times.Using “Clone” allows the transport system to represent a newconnection as a new stream of an existing underlying transportconnection or association if the protocol supports this.A TAPS transport system can fall back to TCP in case nonewer protocol is supported by the peer and the path. Thisenables one-sided deployment of new protocols. “Clone” maytherefore yield a new TCP connection; to avoid surprises, thesystem will then ensure that changing a connection propertyaffects all the connections in a group. Specifying a capacityprofile, or allowing un ordered or un reliable message deliverymay not have an effect. None of these fall-backs endangercorrectness — using TCP instead of a desired better protocolmerely sacrifices performance.I MPLEMENTATIONS
There are currently three known implementations of atransport system conforming to the IETF TAPS specification:PyTAPS [10], Network.Framework [11], and NEATPy [12].Table I shows a comparison of the three implementations andtheir respective protocol and feature support.PyTAPS is a prototype implementation of a transport sys-tem, using the specification of the abstract interface by theIETF TAPS working group. PyTAPS supports TCP, UDP, andthe use of TLS over TCP. It is written in Python and uses asyn-cio, a Python Standard Library for writing concurrent code.It presents an asynchronous transport system with an eventloop that operates on tasks. Concurrent execution is basedon Python co-operative routines (coroutines). A coroutine isan asynchronous block of code with the ability to “yield”,that is, pause its execution and give control to the event loopscheduler at any time during its execution, and maintain itsinternal state. PyTAPS uses co-routines to define all API andcallback functions.Network.framework is Apple’s reference implementation ofa TAPS system. This implementation is available in bothObjective-C and Swift, and it is used to transport applicationdata across Apple’s many platforms. However, Apple’s im-plementation does not currently specify abstract requirementsneeded for the transportation of data, which could be used forthe selection of a protocol that satisfies certain requirements.Instead, the user can indicate preferences tied to a specificprotocol. For example, UDP is modeled as a class called
Support for... PyTAPS Network.Framework NEATPy
TCP/IP (cid:51) (cid:51) (cid:51)
UDP/IP (cid:51) (cid:51) (cid:51)
SCTP/IP (cid:55) (cid:55) (cid:51)
STCP/UDP/IP (cid:55) (cid:55) (cid:51)
TLS/TCP/IP (cid:51) (cid:51) (cid:51)
DTLS/UDP/IP (cid:55) (cid:51) (cid:51)
DTLS/SCTP/IP (cid:55) (cid:55) (cid:51)
DTLS/STCP/UDP/IP (cid:55) (cid:55) (cid:51)
MPTCP (cid:55) (cid:51) (cid:51)
Protocol selection by selection properties (cid:51) (cid:55) (cid:51)
Transport protocol racing (cid:51) (cid:55) (cid:51)
Message framers (cid:51) (cid:51) (cid:51)
Message context / Properties (cid:55) (cid:51) (cid:51)
Cloning (cid:55) (cid:55) (cid:51)
Rendezvous (cid:55) (cid:51) (cid:55)
Connection properties (cid:55) (cid:51) (cid:51)
TABLE I T HE THREE KNOWN
TAPS
IMPLEMENTATIONS : SUPPORTED PROTOCOL STACKS AND KEY
TAPS
FEATURES . NWProtocolUDP , which has the option preferNoChecksum .Naturally, being Apple’s common network interface in pro-duction use, it offers many services beyond a common TAPSAPI. Examples include the possibility for developers to gethighly detailed connection metrics and the ability to createconnections using WebSockets.NEATPy presents a Python-based TAPS API, realized withthe help of language bindings, utilizing the protocol machineryof its underlying core system “NEAT”. NEAT (A New, Evo-lutive API and Transport-Layer Architecture for the Internet),which is described in detail in [6], was the first open-sourceimplementation of a TAPS system, written in C. It was anoutput of the European research project with the same name.Development work on NEAT finished with the project’s end,in 2018; in contrast, NEATPy’s development persisted untilmid-2020, bringing NEAT’s core functionality in line with anup-to-date TAPS system. The NEAT API differs from the mostrecent TAPS specification in several ways. For example, whileNEAT already allowed to specify selection and connectionproperties, it did not offer message properties—instead, someper-message functions were available as parameters of thesend() call. Instead of the five preference levels of TAPS,NEAT only supported qualifying properties as “required” or“desired”. Also, framers were not supported in NEAT—it wasentirely up to the application to parse messages from anincoming block of data.NEATPy can benefit from NEAT’s policy component inthe NEAT user module, which maps properties to policies.These policies do not only enable protocol racing between thecandidate protocols but also provide a fallback mechanism incase a selection of a protocol fails. NEATPy has more featuresand supports more protocol stacks (including SCTP with andwithout UDP encapsulation, and MPTCP; the latter requiresinstalling the reference MPTCP implementation from [13]) than PyTAPS, but this comes at the cost of more overheadand slower overall execution. NEATPy can run on variousoperating systems, and it will make use of different capabilitiesdepending on what the underlying operating system offers. Ourtests used NEATPy on Linux and FreeBSD.For developers, the choice between the three different im-plementations should be relatively easy: Network.Frameworkis an obvious choice for Apple systems, where it is tied to theprogramming languages Objective-C and Swift. Else, if theintention is to use and possibly extend a lightweight system,PyTAPS should be preferred to NEATPy. The latter seemsa good choice for the more experimentally minded, givingaccess to a wealth of features via a somewhat heavyweight un-derlying library. To date, SCTP is only supported by NEATPy,and QUIC is not presently supported by any of the three TAPSimplementations, but it could be added.TAPS IN A CTION
To show how a TAPS system operates, we present a codeexample of NEATPy and discuss the performance achieved ina local test using an emulated network environment.
Code: A Client-Server Example
Listing 1 shows a TAPS server and client, implementedusing NEATPy. The server is written to be as simple aspossible, while we use the client to highlight a little bit moreof the typical TAPS functionality. This code is runnable andcomplete except for some import statements at the top,which are omitted for brevity.The TAPS server listens to incoming connections and sim-ply prints and returns any messages that it gets. A precon-nection object is created, and two arguments are passed toit: a local endpoint (this specifies a port number where the def simple_receive_handler(connection, message, context, is_end, error): data = message.data.decode() print (f"Got message with length {len(message.data)}: {data}") connection.send(data.encode("utf-8"), None) def new_connection_received(connection: Connection): connection.receive(simple_receive_handler) if __name__ == "__main__": local_specifier = LocalEndpoint() local_specifier.with_port(5000) tp = TransportProperties() preconnection = Preconnection(local_endpoint=local_specifier, transport_properties=tp) new_listener = preconnection.listen() new_listener.HANDLE_CONNECTION_RECEIVED = new_connection_received preconnection.start() class FiveBytesFramer(Framer): def start(self, connection): pass def stop(self, connection): pass def new_sent_message(self, connection, message_data, message_context, sent_handler,is_end_of_message): connection.message_framer.send(connection, ’HEADER’.encode("utf-8") + message_data, message_context, sent_handler, is_end_of_message) def handle_received_data(self, connection): header, context, is_end = connection.message_framer.parse(connection, 6, 6) connection.message_framer.advance_receive_cursor(connection, 6) connection.message_framer.deliver_and_advance_receive_cursor(connection, context, 5, True) def clone_error_handler(con:Connection): print ("Clone failed!") def receive_handler(con:Connection, msg, context, end, error): print (f"Got message with length {len(msg.data)}: {msg.data.decode()}") def ready_handler1(connection: Connection): connection.receive(receive_handler) connection.send(("FIVE!").encode("utf-8"), None) connection2 = connection.clone(clone_error_handler) connection2.HANDLE_STATE_READY = ready_handler2 def ready_handler2(connection: Connection): connection.receive(receive_handler) connection.send(("HelloWorld").encode("utf-8"), None) if __name__ == "__main__": ep = RemoteEndpoint() ep.with_address("127.0.0.1") ep.with_port(5000) tp = TransportProperties() tp.require(SelectionProperties.RELIABILITY) tp.prohibit(SelectionProperties.PRESERVE_MSG_BOUNDARIES) preconnection = Preconnection(remote_endpoint=ep, transport_properties=tp) preconnection.add_framer(FiveBytesFramer()) connection1 = preconnection.initiate() connection1.HANDLE_STATE_READY = ready_handler1 preconnection.start() Listing 1.
A TAPS client and server example using a framer and two entangled connections in NEATPy, available from [12]. server will listen) and a transport properties object (this setsa preference level for a couple of selection properties). Sinceno properties are configured in the transport properties object,this server will listen on all available protocols that supportreliability (enabling reliability is a default property choice, asspecified in [14]).Then, we call listen() to accept any incoming connectionsfrom remote endpoints. The server uses two event handlers.The first event handler, “new connection received”, is regis-tered with the member
HANDLE_CONNECTION_RECEIVED of the listener class whenever a new connection is established,and the second event handler is registered inside the thefirst event handler when queuing a receive event. The secondevent handler receives the message, converts its bytes to text,prints the text to the screen and sends the data back. Havingconfigured the preconnection, registered the event handlers,and called “listen”, we call the preconnection’s start() methodin order to start the transport system.The client also creates a preconnection object, to whichit passes a remote endpoint object (specifying the re-mote address and port) and transport properties. In thiscase, not only do we ask for reliable data transfer, butwe prohibit the preservation of message boundaries, whichpractically enforces TCP—indeed, without this requirement,NEATPy communicated via SCTP in our test, and adding“tp.ignore(SelectionProperties.PRESERVE ORDER)” wouldgive us the behavior that we discussed earlier (Fig. 2).Prohibiting message boundary preservation may be a strangerequest to make, but it allows us to test if a message framerworks correctly even when the underlying transport protocoltreats all data as a byte tream. To see this, we add an objectof our
FiveBytesFramer class to the preconnection. Thisframer adds a textual header containing the word
HEADER toall messages (in the method new_sent_message ), whichare supposed to contain only five bytes of data. Upon receiving( handle_received_data ), this header is removed, andthe five bytes are handed over to the data reception handler.The framer inherits from the abstract Framer class, whichrequires defining the “start” and “stop” methods; these allow toimplement initialization and finalization activities, before/afterany data are written or read. We leave them empty as we donot need such functionality in our example.Back in the main function, we register the first eventhandler with
HANDLE_STATE_READY when a connection isestablished, and we call the start() function to start thetransport system. This invokes ready_handler1 as soon asthe connection is ready to accept data. There, one receive eventis queued, a message containing the data
FIVE! is sent, anda new connection is created via clone . Since we use TCP,this just produces another TCP connection, but with SCTP(in FreeBSD only, as support for this type of connection-stream mapping has not been implemented for Linux in theNEAT core), the new connection ( connection2 ) would bea new stream of the already existing SCTP association to theserver. The new connection’s handler for the ready event alsoqueues a single receive event and transmits a message, thistime containing more than five bytes of data:
HelloWorld .Both connections use the same receive handler, which only
Time (s) F l o w s i z e ( M B ) NEATPyPyTAPS
Fig. 4.
Flow completion time of a long (15 MB) and short (1 MB) flowwith NEATPy and PyTAPS: the transfer time of a short flow withNEATPy, joining after 10 seconds, is significantly reduced becauseit benefits from the SCTP association’s large congestion window. prints out the received data together with its length.Running this code produces the following output on theserver side:
Got message with length 11: HEADERFIVE!Got message with length 16:HEADERHelloWorld
This output contains the header because the server does notimplement a framer and simply prints out the raw message infull. On the client side, the output looks as follows:
Got message with length 5: FIVE!Got message with length 5: Hello
As we can see, the framer has removed the header uponreception, and the second message was correctly truncated toa length of five bytes.
Performance
To demonstrate the benefit of protocol independent, portablecode, we ran two simple experiments between a client anda server on a single physical host, using two instances of“VirtualBox” with FreeBSD OSes for a sender and a receiver,respectively. The two VirtualBox instances were logicallyinterconnected on our Mac OS host system, and we seta maximum rate of 5 Mbit/s and introduced a propagationdelay of 30 ms using dummynet/ipfw. We opted for FreeBSDbecause the second experiment uses multistreaming, which forNEATPy is only available with FreeBSD.The first experiment is a simple “hello world” style test,where we merely transferred a single 12-byte message withPyTAPS and NEATPy, and found PyTAPS to be faster: thetransfer took 0.202 seconds with NEATPy and 0.173 secondswith PyTAPS (this is the average duration of 10 tests, witha standard deviation of 4 percent). This is not surprising: Py-TAPS is an altogether much more lightweight implementation,and the reduced overhead plays out positively here.The next experiment aims to show the benefit of a mech-anism in a protocol that is available in NEATPy but not inPyTAPS: SCTP’s multi-streaming. We used two connections,a long file transfer that is joined by a short file transfer afterten seconds, exploiting ‘clone’ in case of NEATPy.Figure 4 shows the result: multi-streaming yields a signifi-cant improvement in the short flow’s completion time (FCT)because, being just a new stream of an ongoing SCTP associ-ation, it can immediately take advantage of the association’s already-grown congestion window. The FCT of the shortflow with NEATPy is reduced by 54 percent in comparisonwith the short flow with PyTAPS, where the two TAPSconnections become two TCP connections, without supportfor multistreaming. We repeated this test ten times with onelong flow (15 MB) starting at t=0 s, and one short flow (1 MB)starting at t=10 s, and show the average FCT. The standarddeviation was between 0,49 percent and 1.53 percent.PyTAPS and NEATPy expose a very similar API (not 100percent equal because they each have their own language-specific ways to implement the abstract interface specifiedin [14]). Thus, the code used in these two tests was essentiallythe same, with only minor syntactical changes. This meansthat (almost) the same program ran faster on the lighter-weight implementation when we did not utilize the protocolfeature “multi-streaming”, and it ran faster on the heavierimplementation when we did use that feature. This is theflexibility that TAPS aims to attain: code can be portable, yetit can benefit from underlying protocol features that go beyondplain TCP and UDP. C
ONCLUSION
This article presented and discussed TAPS as a modernand flexible transport layer replacement for the legacy BSDSocket API. At the time of writing, the Transport ServicesWorking Group is close to finishing its three core documents:the architectural overview [9], API [14] and implementationguidance [7]. We discussed three implementations of thisnovel API and demonstrated its flexibility with code samplesemploying NEATpy. This flexibility allows experimenters toeasily switch between implementations with only minor mod-ifications to the code, while being able to exploit features ofnovel transport protocols that go well beyond TCP’s reliablebyte stream on one hand, and UDP’s unreliable datagramtransmission on the other.TAPS implementations greatly facilitate the comparison ofdifferent transport protocols. Support for new protocols such asQUIC, or novel configurable extensions to existing protocolscould be added to the modular open-source code of NEAT—and with it, NEATPy—relatively easily. This should make itan attractive tool for the research community.A
CKNOWLEDGMENTS
This work has been supported by the Research Councilof Norway under its “Toppforsk” programme through the“OCARINA” project (grant agreement no. 250684).R
EFERENCES[1] C. Raiciu et al. , “How hard can it be? designing and implementing adeployable multipath TCP,” in . San Jose, CA:USENIX Association, Apr. 2012, pp. 399–412.[2] A. Langley et al. , “The QUIC transport protocol: Design and internet-scale deployment,” in
SIGCOMM ’17 . New York, NY, USA: ACM,2017, p. 183-196.[3] D. Ros and M. Welzl, “Less-than-best-effort service: A survey of end-to-end approaches,”
IEEE Comm. Surveys Tutorials, , vol. 15, no. 2, 2013.[4] J. Mehta and E. Kinnear, “Boost performance and security with modernnetworking,” in
Proc. WWDC 2020, date accessed: 2020-12-21 . Apple,Inc., 2020. [Online]. Available: https://developer.apple.com/videos/play/wwdc2020/10111/ [5] P. Balasubramanian, “LEDBAT++: low priority TCP congestioncontrol in windows,” in
Proc. IETF-100, date accessed:2020-12-21 et al. , “NEAT: a platform- and protocol-independent internettransport api,”
IEEE Communications Magazine , vol. 55, no. 6, 2017.[7] A. Brunstrom et al. , “Implementing Interfaces to Transport Services,”IETF, Internet-Draft draft-ietf-taps-impl-08, 2020, work in progress.[8] F. Weinrank and M. T¨uxen, “Transparent flow mapping for neat,” in