An Online Learning Based Path Selection for Multipath Video Telephony Service in Overlay
AAn Online Learning Based Path Selection for Multipath VideoTelephony Service in Overlay ∗ Songyang Zhang
School of Computer Science and Engineering,Northeastern University, [email protected]
Weimin Lei
School of Computer Science and Engineering,Northeastern University, [email protected]
ABSTRACT
Even real time video telephony services have been pervasively ap-plied, providing satisfactory quality of experience to users is still achallenge task especially in wireless networks. Multipath transmis-sion is a promising solution to improve video quality by aggregatingbandwidth. In existing multipath transmission solutions, senderconcurrently splits traffic on default routing paths and has no flexi-bility to select paths. When default paths fall into severe congestionand the available bandwidth decreases, sender has to decrease videoquality by reducing resolution or encoding bitrate. Deploying relayservers in current infrastructure to form overlay network providespath diversity. An online learning approach based on multi-armedbandits is applied for path selection to harvest maximum profit.Further, a congestion control algorithm adapted from BBR is im-plemented to probe bandwidth and to avoid link congestion. Tomaintain throughput stability and fairness, a smaller probe up gainvalue is used and the cycle length in bandwidth probe phase israndomized. To reduce delay, the inflight packets will be reducedto match with the estimated bandwidth delay product in the probedown phase. Experiments are conducted to verify the effectivenessthe proposed solution to improve throughput and quality in videocommunication service.
KEYWORDS congestion control, real time communication, video telephony, over-lay network, path selection
ACM Reference format:
Songyang Zhang and Weimin Lei. 2016. An Online Learning Based Path Se-lection for Multipath Video Telephony Service in Overlay. In
Proceedings ofACM Conference, Washington, DC, USA, July 2017 (Conference’17),
11 pages.DOI: 10.1145/nnnnnnn.nnnnnnn
According to a report [1], video traffic will occupy more than 80% ofall the traffic by the year 2022. The popular video telephony servicespush further increase of the video traffic. For WeChat alone, 410million audio and video calls happened per day [2]. Providing ∗ Produces the permission block, and copyright informationPermission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected].
Conference’17, Washington, DC, USA © 2016 ACM. 978-x-xxxx-xxxx-x/YY/MM...$15.00DOI: 10.1145/nnnnnnn.nnnnnnn satisfactory of quality of experience for video streaming is stillhighly challenge [3], especially for the real time video telephonyservices [4]. The best effort packets delivery mode of Internetdoes not provide any guarantee on quality of service. Bandwidthfluctuation, increased delay and packet loss would cause blurryimages, video rendering process stalling, mosaic and skipped videoframes. Such effects are quite annoying for users.To improve user engagement [5, 6], ensuring high quality ofexperience should be taken as priority for video service providers.Extensive solutions have been proposed to improve video stream-ing quality in current networks. In DASH (Dynamic AdaptiveStreaming over HTTP) system, a same duration of video chunk isencoded with different quality. Clients make decision on differentvideo chunks based on estimated bandwidth and buffered videolength. The goal is to gain maximum video quality while reduc-ing video play stall event. A playout buffer is deployed to absorbinstantaneous throughput variation.The interactive video telephony services have stringent delayrequirement and are allergic to bandwidth fluctuation. For con-versational audio, ITU-T G.114 [7] recommends for less than 150milliseconds one-way delay for excellent quality of experience, butdelays between 150 and 400 milliseconds are still acceptable. Un-like DASH system, which applies a long buffer to insist bandwidthfluctuation, the video telephony applications usually deploy a con-gestion control algorithm at application layer to adapt the bitrateof video encoder to match with available bandwidth. For example,Rebera [8] is a congestion control algorithm designed for WeChatInternational and GCC [9] is the default congestion control algo-rithm deployed in WebRTC . There are also other mechanisms e.g.,negative acknowledgement (NACK), forward error correction (FEC)to combat packet loss for possible quality improvement in videotelephony services. Apart from these mentioned mechanisms, thereare other possibilities to improve quality for video telephony. It iscommon for mobile devices equipped with multi-homed interfaces.The multipath transmission scheme can be exploited to improvevideo steaming quality by scheduling traffic over heterogeneousnetworks (4G and WiFi). Building overlay network [10] on existingInternet infrastructure is another option. With the developmentof cloud service, the geographically distributed datacenters areconnected with managed backbone links, which provides an alter-native to deploy relay servers to provide high performance service.TURN (Traversal Using Relays around NAT) servers are necessaryto relay media traffic for real time video communication when di-rect communication between participating clients is not possible.These relay servers can be used to improve video call quality whendefault path falls into poor condition. By analyzing a dataset of https://webrtc.org/ a r X i v : . [ c s . N I] M a r
30 million calls from Skype, VIA [11] revisits the classic overlaynetworking techniques to reroute traffic for better call quality whenthe access link is not the bottleneck. An overlay network SD-RTN(software defined real time network) has been built by Agora.io in datacenters to provide intelligent routing service for video calls.Centralized servers could monitor the quality of overlay paths andrecommend routings to users. It is claimed the SD-RTN can providelower transmission delay and lower packet loss rate.In this work, we combine overlay routing with multipath trans-mission scheme to improve quality of experience for video tele-phony service. In existing multipath transmission solutions [12, 13],sender has no flexibility to select paths. The video transmissionquality is suffering when default paths fall into congestion andthe available bandwidth is drastically reduced. Overlay networkprovides path diversity. An example is shown in Figure 1. With thehelp of relay nodes, the clients can choose routing path dynamicallyfor each sub-flow to gain maximum profit.There are some issues to be solved under such transmissionscenario. How to select paths from several candidate paths with dif-ferent path metrics to achieve maximum benefit? Without sendingpackets to a specific path, the path metrics remains unknown. Eventhe best path is chosen at present, it may become suboptimal aftera short time. In such situation, insisting a chosen path will missanother higher quality path. To solve such dilemma, the multi-armbandit (MAB) approach is applied for path selection, which is anefficient method to learn in uncertain environment. Tradeoff ismade between exploiting the path with the best predicted perfor-mance and exploring alternatives for possible improvement. Thesecond thing is to implement a congestion control algorithm forvideo transmission service. An algorithm inspired form BBR [14]to better adapt for real time video traffic is applied. The availablebandwidth estimated by congestion control algorithm providesreference for path selection algorithm.The rest of this paper is organized as follows. In section 2, re-lated works on multipath transmission and overlay network arebriefly reviewed. The proposed multipath transmission solutionis described in detail in section 3. Experiments are conducted andresults are presented in Section 4. Section 5 is conclusion. The bandwidth for streaming regular high definition video (720p, 30frames per second) ranges from 2.5Mbps to 4Mbps. More bandwidth(3.5Mbps to 5Mbps) is required if streaming with higher frame rate(720p, 60fps). People have ever increasing demand on video quality.The video resolution has evolved from 480p to 720p to 1080p andeven 4K ultra high definition video is preparing to stream overInternet. Even the internet infrastructure has made fast advancein recent years, the bandwidth for video streaming is still limited.For example, the Verizon 4G LTE wireless broadband can providedownload speed between 5 and 12 Mbps and upload speed between2 and 5 Mbps [15]. Due to dynamic of traffic load at the bottleneck,it would be hard to reach the maximum access upload rate forvideo telephony traffic. Especially when the call sessions traversedifferent ASes (Autonomous system), different countries, and thelast mile links are usually not the bottleneck. In [16], the authors test upload rates in TD-LTE networks. 42% users achieve ratesbelow 2Mbps. The sessions with average throughput below 400kbps accounting for 40% from Taobao-Live network measurementtraces [4].Hence, it is an attractive feature to send packets simultaneouslyby use of multiple interfaces to gain higher throughput. Currentproposed multipath transmission protocols CMP-SCTP [12] andMPTCP [13] are mainly designed for bulk data transfer. There areseveral works focused on congestion control and packet schedulingin MPTCP context. Current proposed congestion control algorithms(LIA [17], OLIA [18], wVegas [19]) for MPTCP couple all subflowstogether to achieve friendliness if the subflows of multipath sessiontraverse a same bottleneck. The scheduling algorithms DEMS [20],STMS [21], and QAware [22] are mainly proposed to alleviate pack-ets arriving out of order problem in MPTCP. Due to varying pathproperty, packets may arrive out of order at receiver, which mayget buffer resource fully occupied and would cause head of lineblocking (HOL) [23]. In kernel net stack, the buffer is pre-allocated.Once the buffer at receiver is fully occupied, the subsequent ar-riving packets will be dropped. HOL blocking leads suboptimalthroughput in MPTCP since the rate back off after packet loss isnot caused by congestion.It is inconvenient to apply MPTCP to enable multipath trans-mission since it requires OS kernel support at both sides. Someworks explore multipath video streaming at application layer by ini-tializing separate TCP connections. MSPlayer [24] requests videodifferent chunks from two different servers in DASH system byaccessing WiFi and cellular networks. The bandwidth on each pathis estimated by harmonic mean to mitigate the effect large outliersdue to network throughput variation. Such multipath transmissionstrategy reduces start up delay and provides higher video quality. In[25], the video frames are encoded into different layers by scalablevideo coding (SVC) technologies. Layer selection is formed as anoptimization problem based on throughput predication.These multipath transmission protocols that strictly guaranteein order delivery and reliable transmission by implementing Auto-matic Repeat reQuest (ARQ) are not appropriate for delay sensitivevideo telephony traffic. Once a packet is lost, the subsequent re-ceived packet would not be submitted to the upper layer until thelost packet is recovered by retransmission. Considerable latencyis thus introduced. Hence, the video telephony traffic is usuallystreaming over UDP and this work is no exception. MPRTP [26]is designed as an extension to RTP protocol for real time videocommunication.The implementation of overlay network to provide better serviceis not a new idea. In [27], overlay helpers are used to optimize globallive video streaming. Experiments on real test beds indicate videotransmission in overlay network can reduce scheduling delay andimprove throughput. In [28], a solution on relay nodes placementin current network infrastructure to satisfy the full throughputregion is proposed. The conclusion is that only a small fraction ofcontrollable nodes is sufficient to achieve maximum throughput. The multipath transmission system for video telephony in overlaynetwork is illustrated in Figure 1. R denotes relay server. Sender P P packet sendmanagervideo encoder scheduler buffer Frame DeliverReceiver bufferSender packet receivemanager
R1R2
CC pacer ratecontroller P raw frames queue packet sendmanagerCC pacer encodedframes path manager subflow1 subflow2 subflow1subflow2 packet receivemanager Figure 1: Multipath Video Transmission of Multihomed Client in Overlay Network distributes multimedia packets over different subflows. Each sub-flow can dynamically choose path (default path or overlay pathformed by overlay nodes) for sending packets according to pathquality. The relay nodes can be recommend by a centralized serverbased on historical statistics. In this work, we mainly focus on pathselection to maximize video transmission quality, assuming relayshave already been available for endpoints. The modules in Figure 1at sender are responsible for packets transmission, path selectionand bitrate adjustment on video encoder.
For each path, there is a packet send manager implemented at senderside and a corresponding packet receive manager at receiver side.Only two pairs are shown in Figure 1 due to space limitation. Theestimated bandwidth and round trip delay can be got in packet sendmanager. At receiver sider, acknowledgement packet is sent to itspeer at intervals when multimedia packets arrive. The congestioncontroller (CC) estimates available bandwidth based on feedbackpackets.
The congestioncontrol algorithm implemented in CC module is a modificationfrom BBR [14] to adapt it for video telephony traffic transmissionto maintain rate stability of video encoder. BBR is claimed as a con-gestion based rate control algorithm. Its goal is to get close to theoptimal control point, in which flows achieve the maximum avail-able bandwidth, minimum transmission delay and lowest packetloss rate. In BBR, when a sent packet is acknowledged, a roundtriptime sample and the inflight packet length ( ∆ delivered ) when thesent packet departs from sender can be got. A bandwidth estima-tion sample is calculated in Equation 1. The packet sending rate is pacinд rate in Equation 2, which is the product of pacinд дain andmaximum estimated throughput ( bw es ) in 10 rounds. bw = ∆ delivered ∆ t (1) pacinд rate = bw es ∗ pacinд дain (2)The four control states StartUp, Drain, ProbeBW, and ProbeRTTin BBR are shown in Figure 2. ProbeBWStartUp ProbeBWProbeRTTDrain2/ln2 ln2/2pacing_gain
Figure 2: Control states in BBR
In ProbeBW state, there are different pacinд дain values [1.25,0.75, 1, 1, 1, 1, 1, 1] in 8 RTTs. The probe up phase with 1.25gain is to increase inflight packets to probe extra bandwidth, andprobe down phase with 0.75 gain is to get rid of excess queue.The congestion window is set as 2*BDP in ProbeBW to guaranteeenough packets can be sent during probe up phase. If the minimalRTT is not sampled again within 10 seconds, the link seems fallinginto congestion, and it will enter into ProbeRTT state, and thecongestion window is set as 4*MSS to get inflight packets totallydrained from links.Even BBR achieves excellent performance in term of throughputfor bulk data transmission in TCP, it also suffers from several draw-backs. Firstly, bandwidth allocation fairness can not be guaranteedin bottleneck link with shallow buffer. When multi flows share abottleneck, flows overestimate bandwidth and overload the link.High packet loss rate is introduced. Secondly, BBR suffers from RTTunfairness issue and flavors towards flows with longer RTT. Asindicated in [29], these strategic receivers could easily manipulatesuch drawback by sending delayed acknowledged packets to stealbandwidth.To implement BBR for real time video communication is an at-tempt in this work. In video telephony applications, video framesare captured at fixed intervals and the output bitrate of video en-coder is stable in a short time span. In such situation, there maybe not enough packets available in probe up phase with 1.25 gainfor bandwidth probe. To maintain rate stability and avoid excessqueueing delay, the pacing gain is 1.1 in probe up phase and thepacing gain is 0.85 in probe down phase. In our previous work[30], a delay constraint BBR (Delay-BBR) algorithm is proposedin order to achieve lower transmission delay. Once delay signal xceeds a predefined threshold, Delay-BBR will actively reducesending rate to drain inflight packets. In recent tests, we found itinherits the RTT unfairness property from BBR. In old version ofBBR, the probe down phase holds for essentially RTT min . In newlypatch, the probe down phase holds the drain state until the inflightpackets matching with estimated BDP. As verified in [31], the lowerqueue delay can be achieved and RTT unfairness issue is alleviatedby this modification, but it results in rate fluctuation when flowscompeting for bandwidth. We found to randomize the gain cyclelength from 2 to 8 can solve rate fluctuation problem. Such idea isapplied in this work. This improved congestion control algorithmfor video telephony traffic is named as RTC-BBR.In RTC-BBR, the procedure to update pacing gain in ProbeBW isdescribed in Algorithm 1.
CYCLE RAN D is 7 and kGainCycleLen is 8. inf liдht denotes the length of sent packets that have not beenacknowledged. When there is packet loss event, the probe up phaseexists earlier (line 14 in Algorithm 1).
Algorithm 1
UpdateGainCyclePhase
Input: the timestamp (now), inf liдht , has loss elapsed ← now − cycle mstamp if elapsed > cycle len ∗ RTT min then cycle mstamp ← now cycle len ← kGainCycleLen − rand () % CYCLE RAN D pacinд дain ← . return end if if pacinд дain == then return end if if pacinд дain < . and inf liдht ≤ BDP then pacinд дain ← end if if elapsed > RTT min and ( inf liдht > . ∗ BDP or has loss ) then pacinд дain ← . end if The connection will evenly send packetsinto network according to pacinд rate . Traditionally, packets areseding in burst mode, which may lead packets queued at interme-diate routers and introduce extra delay. When a packet A withlength L a is sent out at time a sent ts , pacer calculates the nexttime (sent ts) that another packet is allowed to be sent out. sent ts = a sent ts + L a pacinд rate (3) An encoded video frame willbe packetized into segments with length smaller than maximumsegment size. Apart from video payload, necessary information istagged into header: frame index, frame captured timestamp, totalsegments of a frame and segment index. These segments are storedin buffer. The scheduler decides which subflow to send a specificpacket in buffer.The connection will maintain a queue in each subflow to recordtotal length ( Q ) and packet offset for these scheduled packets. The time to send a new packet is depended on the pacer. When thetime comes, the connection will find the packet in buffer with offsetinformation and delivery it out.The goal of the implemented packets distribution algorithm inmultipath context is to minimize packet expected arriving latency.The latency is composed of queue delay and transmission delay inEquation (4). The information on smoothed roundtrip time SRTT s and available bandwidth bw ses in subflow s can be got from packetsend manager. When a new RTT sample is avaialble, SRTT isupdated according to Equation (5), and delta is set as 0.85. Whennew packetized segments comes, it will be scheduled to the subflowwith minimal delay in Equation (4).For packet belonging to a key frame, only when acknowledgmentis received, it is evicted from buffer. For non-key frame packets,the sent packets will be cached at buffer at most 400 milliseconds.The lost packet that can be found in buffer will be retransmittedimmediately over path with minimal transmission delay. λ s = SRTT s + Q s bw ses (4) SRTT = ( − δ ) × SRTT + δ × RTT (5)
There are k i available paths for sub-flow i . The default path isincluded and the other k i − P = { P , P } for subflow1 and path set P = { P , P } is forsubflow2. Transitional multipath transmission solutions only takethe advantage of aggregating bandwidth. The path diversity pro-vided by relays in overlay gives sender flexibility to choose routingpaths to gain maximum profit. For example, when transmissionquality of the default path P22 deteriorates (reduced bandwidth orinsufferable transmission delay), subflow1 can switch to overlaypath P21 for possible improvement.The rate distortion model in Equation (6) is introduced in [32]. R e is the output bitrate of video encoder. D , θ and R are parametersrelated to video sequence and codec. It clearly shows that lowerencoded image distortion can be achieved by increasing R e . Butfor real time video transmission, R e should be matched with thereference rate R set by Rate Controller. Hence, it is a role that pathmanager module can paly to choose high quality paths to reducevideo distortion. D e = θR e − R + D (6)In each time slot, the path manager will choose a routing pathfor each sub-flow for packets transmission. Q ( i , j , t ) denotes thetransmission quality of path j for subflow i at time slot t . a ( i , j , t ) indicates whether path j is chosen at time slot t . The goal is tomaximize video quality in the long run as Equation 7 shows.max (cid:205) Tt = (cid:205) i (cid:205) j Q ( i , j , t ) ∗ a ( i , j , t ) T subject to : a ( i , j , t ) ∈ { , } (7)Due to lack of oracle perspective, to choose paths always providemaximum quality of experience to users is a hard task. Withoutsending packets to a specific path, the path metrics remains un-known. Even the best path is chosen at present, it may become uboptimal after a short time. The multi-armed bandit model isfit for such task. Because traffic load on each path is highly dy-namic, path selection belongs to restless bandit problems. Eachpath can be regarded as an arm of a multiple slots machine. Oncean arm ( j ) is pulled in time slot t , the path metrics (available band-width and delay) can be revealed from feedback packets and reward( X ( j , t ) ) is generated. Always choosing an arm with maximumpartially known properties may miss another arm with higher re-ward. Hence, tradeoff is made between exploration and exploitation.Exploration takes the risk by trying alternative choices to collec-tion information on arms and exploitation just takes the best armempirically with observed information.The upper confidence bound (UCB) policy is applied here, whichwas analyzed in detail in [33]. In UCB, the upper bound rewardof a specific arm is estimated by previous rewards and plus someuncertainty as Equation (8) shows. I ( j , t ) = X ( j , t ) + U ( j , t ) (8) X ( j , t ) = (cid:205) ts = X ( j , s ) ( a ( j , s )) N ( j , t ) (9) U ( j , t ) = B (cid:115) loд ( T ) N ( j , t ) (10) X ( k , t ) is the mean of previous rewards. N ( j , t ) counts the timesthat arm j has been pulled from start. U ( j , t ) is the upper confi-dence bound of rewards, which describes uncertainty of an arm.A common form for U ( j , t ) is given in Equation 10. Here, B is atunable factor.In each time slot, player selects the arm with maximum I k , t .Equation (8) reflects well the tradeoff between exploration andexploitation. Based on this equation, player decides whether tocontinue with the current arm or try alternatives. If the times topull an arm are less, the bias on the estimated average mean is large.The term U ( k , t ) encourages player to pull less selected arms tocollect more samples to eliminate mean estimation bias.These involved parameters are explained here. Estimated band-width is used as reward for path manager to choose paths for sender.As indicated in [34], the averaging methods in Equation 9 is ap-propriate for static bandit problems, in which reward probabilitiesremain unchanged over time. But available bandwidth on eachpath is nonstationary. When a path provides higher throughput forsome time, similar throughput can be maintained in large possibil-ity hereafter. It is reasonable to give more weight to recent rewards.The exponential smoothing filter is applied for such purpose andthe term on empirical mean reward is redefined in Equation (11). α is empirically set as 0.9. bw is the newest estimated bandwidthfrom congestion controller when a path is exploited. C denotes thepaths to choose in each time slot, which is equal to the number ofsubflows. ˆ Bw ( j ) = ( − α ) × ˆ Bw ( j ) + α × bw (11) U ( j , t ) = Bw ( j ) (cid:115) loд ( C × T ) N ( j , t ) (12) a ( i , j , t ) = arg max ˆ Bw ( i , j ) + Bw ( j ) (cid:115) loд ( C × T ) N ( i , j , t ) (13) The upper confidence bound term is given in Equation (12). Therule for path manager module to decide which path to use forsubflow i is given in Equation (13). Bw ( j ) is the maximum observedbandwidth during a monitor interval (kObservedTime=10 seconds).It can be interpreted that client expects to achieve the previousthroughput by re-selecting this path. If the path is not exploitedwithin the monitor interval, Bw ( j ) is assigned with the maximumbandwidth sample observed in currently. It encourages client tochoose this path again to collect bandwidth sample. When thepath is chosen, Bw ( j ) is updated with the new maximum estimatedbandwidth during the path usage slot. The detail to update Bw ( j ) isshown in Algorithm 2. Before each decision slot, the path managerwill call Algorithm 3 to delete obsolete bandwidth samples.The detail on path selection is show in Algorithm 4. When apath with the maximum reward (line 13) is chosen for a subflow, itspulled times counter N will be updated (line 23). At session initialphase, all available paths will be exploited to collect throughputand path delay information. The initial value of N is 1. Algorithm 2
OnNewBandwidthSample
Input: bw , now Push back ( bw , now ) to bwSamples if samples == then Bw ← bw maxBw ← bw ˆ Bw = bw else ˆ Bw = ( − α ) × ˆ Bw + α × bw end if if bw > maxBw then maxBw ← bw end if DeleteObsoleteSamples(now) samples ← samples + Each captured video frame is firstly delivered to raw frame queuewaiting to be processed by encoder. The packets of encoded imageswill be put into buffer.Rate controller adjusts the output bitrate of video encoder withreference rate ( R = (cid:205) s ∈ S bw s ). bw s is got from congestion con-trollers of currently exploited routing path in sublfow s . The refer-ence bitrate of encoder is reconfigured every 50 milliseconds.We observed in experiments that video encoder can not imme-diately generate bitrate matching with the target rate. Especiallywhen available bitrate is decreased, encoder takes about 1 secondto output encoded images approximating the target bitrate, whichis also revealed in [4]. In such situation, the length of buffer willincrease. In order to assure low frame delivery delay, it is commonto drop some raw frames when delay exceeds certain threshold. Amethod to reduce bitrate by reducing frame rate.The delay ( d s ) before packets of an captured frame can be deliv-ered at sender is composed by three parts: delay ( d q ) at raw framequeue, encoding delay ( d en ) and buffer delay ( d b ). When a rawframe is sent to encoder, a sample of encoding delay d en ( i ) is got lgorithm 3 DeleteObsoleteSamples
Input: now while bwSamples .size()¿1 do sample ← bwSamples . f ront () if now − sample . time > kObservedTime then bwSamples . pop f ront () else break end if end while bw ← for each sample ∈ bwSamples do if sample . bw > bw then bw ← sample . bw end if end for Bw ← bw if bwSamples . size () == then Bw ← maxBw end if after the frame is encoded. The exponential smooth filter is appliedagain in Equation (14) to update the estimation on image encodingdelay.When a raw frame is dequeued, it will be dropped if the delay inEquation (15) exceeds 400 milliseconds. Here, λ min = min s ∈ S λ s .ˆ d en = ( − α ) × ˆ d en + α × d en ( i ) (14) d = d q + ˆ d en + λ min (15) For low latency consideration, packets in RTC applications areusually sent over UDP with partially reliability. To evaluate theperformance of the proposed multipath transmission solution inthis work, a transmission protocol is implemented. It is referencedfrom QUIC and only three frames (STREAM, STOP WAITINGand ACK) are applied. Each sent packet is allocated with a uniquepacket number and the ACK frame is sent to its peer to notifyreceived packets information. Receiver sorts received packets byoffset number extracted from STREAM frame. The STOP WAITINGframe notifies the peer to stop waiting these packets with packetnumber under the notified number. Such designation leaves senderflexibility to decide whether to retransmit lost STREAM framesaccording to packet importance and delay. Each retransmittedpacket is allocated a new packet number. All experiments involved in this part are running on ns3.26. Thecollected simulation data in this work is publicly available at To test the performance of RTC-BBR algorithm, a dumbbell topol-ogy in Figure 3 is built on ns3. Apart from BBR, Cubic [35] and https://pan.baidu.com/s/1Y3ZTQxBFpyjCmjTuYCKNzA:yty3 L0 L2L3 L1 n0n1 n2 n3 n4n5 L4 Figure 3: Dumbbell topologyTable 1: Link configuration on L1
Case Bandwidth Propagation Delay Queue Length1 3Mbps 50ms 3Mbps*100ms2 3Mbps 50ms 3Mbps*150ms3 3Mbps 50ms 3Mbps*200ms4 5Mbps 50ms 5Mbps*100ms5 5Mbps 50ms 5Mbps*150ms6 5Mbps 50ms 5Mbps*200ms7 6Mbps 50ms 6Mbps*150ms8 6Mbps 50ms 6Mbps*200ms9 8Mbps 50ms 8Mbps*150ms10 8Mbps 50ms 8Mbps*200ms11 10Mbps 50ms 10Mbps*150ms12 10Mbps 50ms 10Mbps*200ms
50 100 150 200 250 300 1 2 3 4 5 6 7 8 9 10 11 12 a v e r a g e o n e w a y d e l a y / m s index BBRRTC-BBRCubicGCC Figure 4: Average one way transmission delay -0.05 0 0.05 0.1 0.15 1 2 3 4 5 6 7 8 9 10 11 12 p a c k e t l o ss r a t e index BBRRTC-BBRCubicGCC Figure 5: Packet loss rate r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (a) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (b) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (c) Figure 6: Rate dynamics of GCC flows. (a) C1. (b) C5. (c) C11. r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (a) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (b) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (c) Figure 7: Rate dynamics of BBR flows. (a) C1. (b) C5. (c) C11. r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (a) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (b) r a t e / k bp s time/s fl ow1 fl ow2 fl ow3 (c) Figure 8: Rate dynamics of RTC-BBR flows. (a) C1. (b) C5. (c) C11.
GCC are also evaluated to work as benchmarks. The configurationon link L lgorithm 4 Path Selection Algorithm
Input: T , time slots counter, sub f lows , stores f lowid of all subflows, paths inf o , stores ( id , f lowid , Bw , ˆ Bw , N ) on all availablepaths Outpt: exploited path , the chosen path for subflow C = len ( sub f lows ) P = len ( paths inf o ) for i ∈ [ , C ) do f lowid ← sub f lows [ i ] . id X max ← path id ← − for j ∈ [ , P ) do if paths inf o [ j ] . f lowid == f lowid then id ← paths inf o [ j ] . id ˆ Bw ← paths inf o [ j ] . ˆ Bw Bw ← paths inf o [ j ] . Bw N ← paths inf o [ j ] . N X ← ˆ Bw + Bw (cid:113) loд ( C × T ) N if X > X max then X max ← X path id = id end if end if end for sub f lows [ i ] . exploited path = path id for j ∈ [ , P ) do if paths info[j].id==path id then paths inf o [ j ] . N ← paths inf o [ j ] . N + end if end for end for T ← T + Table 2: Configuration of links to test rtt unfairness
Case L0 L1 L2 L3 L4(BW, OWD, Q)1 (10, 10, 200) (4, 10, 200) (10, 10, 200) (10, 20, 200) (10, 30, 200)2 (10, 10, 200) (4, 10, 200) (10, 10, 200) (10, 10, 200) (10, 30, 200)3 (10, 20, 200) (4, 10, 200) (10, 10, 200) (10, 10, 200) (10, 30, 200)
Table 3: Calculated results in RTT unfairness simulation
Algo Case 1 2 3( x , x , R)BBR (634, 3155, 4.98) (837, 2972, 3.55) (1338, 2471, 1.85)RTC-BBR (1792, 2026, 1.13) (1835, 1988, 1.08) (1921, 1880, 1.02)
500 1000 1500 2000 2500 3000 3500 0 50 100 150 200 250 300 r a t e / k bp s time/s fl ow1 fl ow2 Figure 9: RTT unfairness issue of BBR in case 3
500 1000 1500 2000 2500 3000 3500 0 50 100 150 200 250 300 r a t e / k bp s time/s fl ow1 fl ow2 Figure 10: RTT fairness improvement of RTC-BBR in case 3 bandwidth can get the bandwidth resource highly utilized, whichis an attractive feature of BBR-like algorithms.As indicated in [29, 37], BBR flavors towards flows with longerround trip delay. The dumbbell topology is applied to verify whetherRTC-BBR algorithm suffers from the same RTT unfairness problem.The configuration of all links is shown in Table 2. Q is in unit ofmilliseconds and the buffer length of each link will be configure as BW ∗ Q . Three experiments are designed. In each case, two flowsare running at the same time and are constrained under a samecongestion control algorithm(BBR or RTC-BBR). flow1 starts fromn0 to destination n4 (path1) and flow2 sends packets through path2(n1 to n5). The running time of each experiment lasts 300 seconds. L3 L6L2 n1 n3 n2 n4r1 r2 L4 L5 Sender Receiver
Figure 11: Mulitpath topology C D F mean (Mbps) Figure 12: Cumulative average rate distribution of traceddataset
The average throughput is calculated as Equation 16. bytes is thelength of all received packets at receiver. The throughput ratiodefined in Equation 17 of flow2 and flow2 is calculated. The resultsare given in Table 3 and The average rates of the two RTC-BBRflows are quite close in each test. BBR flow2 can approach higherthroughput in three tests. The rate dynamics of case 3 are given inFigure 9 and 10. RTC-BBR algorithm can obviously alleviate theRTT unfairness issue. x = bytesduration (16) R = x x (17) To evaluate the performance of the proposed path selection algo-rithm, two subflows are applied from sender to receiver. Two pathsare available for each subflow. The experiment topology is shownin Figure 11. For subflow1, L1 the default path and the other path(L2 and L3) is overlay path provided by relay node. The availablebandwidth in each path is randomly configured with throughputtraces collected from real networks. The traced throughput datasetis publicly available at [38]. The bandwidth samples are collected C D F mean (Mbps)defaultoursoracle Figure 13: Cumulative distribution of average throughput r a t e / M bp s Figure 14: Error bar on average rate of all tests
40 40.5 41 41.5 42 42.5 43 0 20 40 60 80 100 120 140 p s n r index defaultoursoracle Figure 15: Average PSNR from mobile devices which access Internet from LTE or WiFi. 100traces are included in this dataset and Figure 12 gives the cumula-tive average throughput distribution. Since the trace dataset lackof transmission delay information, the transmission delay in eachpath is configured with delay value that is uniformly distributedbetween 50 milliseconds and 100 milliseconds.Total 150 experiments are running and each simulation processlasts 400 seconds. In each test, four throughput traces are randomlychosen for this four paths. X264 is used to encode video frames. e tested YUV video is KristenAndSara (600 frames, 1280x720,60fps) and is downloaded from . Even the original video frames arecaptured in 60 frames per second, the video sequences are deliveredto raw frame queue in 30 fps in simulation, which is enough forRTC application. Path manager makes decision to choose path foreach subflow on every 1 second. When the video frames file isread to its end, its offset will be randomly chosen to align with thebeginning of a frame. The available bandwidth on each routingpath is traced.The proposed path selection algorithm in multipath transmissioncontext is compared with two other schemes “default” and “oracle”.Here, “default” means sender exploits the default routing path ineach its subflow for packet transmission and “oracle” means senderalways chooses the routing path with maximum throughput foreach subflow in each decision slot. The cumulative average band-width distribution of all 150 experiments is given in Figure 13. Theerror bar on average throughput of all experiments in each pathselection algorithm is given in Figure 14. The results clearly indicatethat the proposed path selection algorithm can gain higher ratein most experiments than fixed routing path transmission scheme.The “oracle” transmission scheme can gain the maximum profit insimulation, which is an ideal path selection mechanism. In simula-tion based on trace dataset, the future throughput of a path is totallyknown. Such algorithm is not applicable in real environment. Aswe argued previously, sender could not know exactly the reachablebandwidth of a routing path before sending packets into it.At receiver side, when received packets can be reassembled intoa complete frame, it will be handled over to decoder. Peak signalto noise ratio (PSNR) is calculated to evaluate picture quality afterframe decoding. The average PSNR value is given in Figure 15.The frames involved to calculate the average PSNR in each test areabout 10000. Since the proposed path selection algorithm can gainhigher throughput than the default routing transmission scheme,the encoder can allocate more bit on each encoded video frames,the sender with proposed path selection algorithm can gain highervideo quality as verified in Figure 15. In this work, we combine the multipath transmission scheme withpath selection for possible improvement for video telephony traf-fic. In real networks, the path quality is highly dynamic accordingto traffic load in the bottleneck link. Deploying servers in geo-graphically distributed datacenters to form overlay network is apromising solution to improve video call quality. With the helpof relay nodes in overlay network, sender can choose good pathsfrom several candidate paths to gain maximum benefit. An onlinelearning approach based on multi-armed bandit model is applied toselect paths for subflows. Based on experiments with throughputtrace collected from real networks, sender with the proposed pathselection algorithm can gain higher throughput and improve videoquality than the method to insist the default path transmissionduring the whole session. https://media.xiph.org/video/derf/ https://blog.csdn.net/abcSunl/article/details/53841953 As real time conversational video communication in Internethas gained quite popularity in recent years, implementing a con-gestion control algorithm is necessary to avoid link congestion andto promote bandwidth allocation fairness. A congestion controlalgorithm RTC-BBR is implemented on the multipath transmis-sion framework, which is adapted from BBR for real time videotraffic. Experiments are conducted to evaluate the performanceof the proposed congestion control algorithm. Other congestioncontrol algorithms (Cubic, BBR and GCC) are also tested to workas baselines. Results show it can achieve lower queue delay andalleviate RTT unfairness issue in BBR algorithm. Most importantly,the improved version shows less drastic throughput variation. Suchproperty could maintain the stability of video encoder.
REFERENCES doi:10.1109/INFOCOM.2014.6848080 .[4] A. Zhou, H. Zhang, G. Su, L. Wu, R. Ma, Z. Meng, X. Zhang, X. Xie, H. Ma, X. Chen,Learning to coordinate video codec with transport protocol for mobile videotelephony, in: The 25th Annual International Conference on Mobile Computingand Networking, MobiCom ’19, ACM, New York, NY, USA, 2019, pp. 29:1–29:16. doi:10.1145/3300061.3345430 .URL http://doi.acm.org/10.1145/3300061.3345430[5] F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. Joseph, A. Ganjam, J. Zhan, H. Zhang,Understanding the impact of video quality on user engagement, SIGCOMMComput. Commun. Rev. 41 (4) (2011) 362–373. doi:10.1145/2043164.2018478 .URL http://doi.acm.org/10.1145/2043164.2018478[6] S. S. Krishnan, R. K. Sitaraman, Video stream quality impacts viewer behavior:Inferring causality using quasi-experimental designs, IEEE/ACM Transactionson Networking 21 (6) (2013) 2001–2014. doi:10.1109/TNET.2013.2281542 doi:10.1145/2910017.2910608 .URL http://doi.acm.org/10.1145/2910017.2910608[9] G. Carlucci, L. De Cicco, S. Holmer, S. Mascolo, G. Carlucci, L. De Cicco, S. Holmer,S. Mascolo, Congestion control for web real-time communication, IEEE/ACMTrans. Netw. 25 (5) (2017) 2629–2642. doi:10.1109/TNET.2017.2703615 .[10] D. Andersen, H. Balakrishnan, F. Kaashoek, R. Morris, Resilient overlay networks,SIGOPS Oper. Syst. Rev. 35 (5) (2001) 131–145. doi:10.1145/502059.502048 .URL http://doi.acm.org/10.1145/502059.502048[11] J. Jiang, R. Das, G. Ananthanarayanan, P. A. Chou, V. Padmanabhan, V. Sekar,E. Dominique, M. Goliszewski, D. Kukoleca, R. Vafin, H. Zhang, Via: Improvinginternet telephony call quality using predictive relay selection, in: Proceedingsof the 2016 ACM SIGCOMM Conference, SIGCOMM ’16, ACM, New York, NY,USA, 2016, pp. 286–299. doi:10.1145/2934872.2934907 .URL http://doi.acm.org/10.1145/2934872.2934907[12] J. R. Iyengar, P. D. Amer, R. Stewart, Concurrent multipath transfer using sctpmultihoming over independent end-to-end paths, IEEE/ACM Transactions onNetworking 14 (5) (2006) 951–964. doi:10.1109/TNET.2006.882843 .[13] A. Ford, C. Raiciu, M. J. Handley, O. Bonaventure, TCP Extensions for Multi-path Operation with Multiple Addresses, RFC 6824 (Jan. 2013). doi:10.17487/RFC6824 doi:10.1109/TVT.2016.2581020 . doi:10.1109/TNET.2013.2274462 .URL http://dx.doi.org/10.1109/TNET.2013.2274462[19] Y. Cao, M. Xu, X. Fu, Delay-based congestion control for multipath tcp, in: 201220th IEEE International Conference on Network Protocols (ICNP), 2012, pp. 1–10. doi:10.1109/ICNP.2012.6459978 .[20] Y. E. Guo, A. Nikravesh, Z. M. Mao, F. Qian, S. Sen, Accelerating multipathtransport through balanced subflow completion, in: Proceedings of the 23rdAnnual International Conference on Mobile Computing and Networking, ACM,2017, pp. 141–153.[21] H. Shi, Y. Cui, X. Wang, Y. Hu, M. Dai, F. Wang, K. Zheng, Stms: Improvingmptcp throughput under heterogeneous networks, in: 2018 USENIX AnnualTechnical Conference, 2018, pp. 719–730.[22] T. Shreedhar, N. Mohan, S. K. Kaul, J. Kangasharju, Qaware: A cross-layerapproach to mptcp scheduling, in: 2018 IFIP Networking Conference (IFIP Net-working) and Workshops, 2018, pp. 1–9. doi:10.23919/IFIPNetworking.2018.8696843 .[23] S. Ferlin, ¨O. Alay, O. Mehani, R. Boreli, Blest: Blocking estimation-basedmptcp scheduler for heterogeneous networks, in: 2016 IFIP Networking Con-ference (IFIP Networking) and Workshops, 2016, pp. 431–439. doi:10.1109/IFIPNetworking.2016.7497206 .[24] Y. Chen, D. Towsley, R. Khalili, Msplayer: Multi-source and multi-path videostreaming, IEEE Journal on Selected Areas in Communications 34 (8) (2016)2198–2206. doi:10.1109/JSAC.2016.2577322 .[25] A. Elgabli, K. Liu, V. Aggarwal, Optimized preference-aware multi-path videostreaming with scalable video coding, IEEE Transactions on Mobile Computing(2018) 1–1 doi:10.1109/TMC.2018.2889039 .[26] V. Singh, S. Ahsan, J. Ott, Mprtp: Multipath considerations for real-time media(2013) 190–201 doi:10.1145/2483977.2484002 .URL http://doi.acm.org/10.1145/2483977.2484002[27] D. Ren, Y. Xu, S.-H. G. Chan, Beyond 1mbps global overlay live streaming: Thecase of proxy helpers, ACM Trans. Multimedia Comput. Commun. Appl. 11 (2)(2015) 26:1–26:22. doi:10.1145/2652485 .URL http://doi.acm.org/10.1145/2652485[28] N. M. Jones, G. S. Paschos, B. Shrader, E. Modiano, An overlay architecture forthroughput optimal multipath routing, IEEE/ACM Transactions on Networking25 (5) (2017) 2615–2628. doi:10.1109/TNET.2017.2703867 .[29] S. Ma, J. Jiang, W. Wang, B. Li, Fairness of congestion-based congestion control:Experimental evaluation and analysis (2017). arXiv:arXiv:1706.09115 .[30] S. Zhang, W. Lei, W. Zhang, Y. Guan, H. Li, Congestion control and packetscheduling for multipath real time video streaming, IEEE Access 7 (2019) 59758–59770. doi:10.1109/ACCESS.2019.2913902 .[31] S. Zhang, An evaluation of bbr and its variants (2019). arXiv:arXiv:1909.03673 .[32] K. Stuhlmuller, N. Farber, M. Link, B. Girod, Analysis of video transmission overlossy channels, IEEE Journal on Selected Areas in Communications 18 (6) (2000)1012–1032. doi:10.1109/49.848253 .[33] P. Auer, N. Cesa-Bianchi, P. Fischer, Finite-time analysis of the multiarmedbandit problem, Machine Learning 47 (2) (2002) 235–256. doi:10.1023/A:1013689704352 .URL https://doi.org/10.1023/A:1013689704352[34] R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, MIT press,2018.[35] S. Ha, I. Rhee, L. Xu, Cubic: a new tcp-friendly high-speed tcp variant, ACMSIGOPS operating systems review 42 (5) (2008) 64–74.[36] J. Gettys, K. Nichols, Bufferbloat: Dark buffers in the internet, Queue 9 (11) (2011)1–15.[37] M. Hock, R. Bless, M. Zitterbart, Experimental evaluation of bbr congestioncontrol, in: 2017 IEEE 25th International Conference on Network Protocols(ICNP), 2017, pp. 1–10. doi:10.1109/ICNP.2017.8117540 .[38] Live video streaming challenge, https://github.com/NGnetLab/Live-Video-Streaming-Challenge, accessed October 14, 2019..[38] Live video streaming challenge, https://github.com/NGnetLab/Live-Video-Streaming-Challenge, accessed October 14, 2019.