Maxim Claeys

QoE-Driven Rate Adaptation Heuristic for Fair Adaptive Video Streaming

HTTP Adaptive Streaming (HAS) is quickly becoming the de facto standard for video streaming services. In HAS, each video is temporally segmented and stored in different quality levels. Rate adaptation heuristics, deployed at the video player, allow the most appropriate level to be dynamically requested, based on the current network conditions. It has been shown that today’s heuristics underperform when multiple clients consume video at the same time, due to fairness issues among clients. Concretely, this means that different clients negatively influence each other as they compete for shared network resources. In this article, we propose a novel rate adaptation algorithm called FINEAS (Fair In-Network Enhanced Adaptive Streaming), capable of increasing clients’ Quality of Experience (QoE) and achieving fairness in a multiclient setting. A key element of this approach is an in-network system of coordination proxies in charge of facilitating fair resource sharing among clients. The strength of this approach is threefold. First, fairness is achieved without explicit communication among clients and thus no significant overhead is introduced into the network. Second, the system of coordination proxies is transparent to the clients, that is, the clients do not need to be aware of its presence. Third, the HAS principle is maintained, as the in-network components only provide the clients with new information and suggestions, while the rate adaptation decision remains the sole responsibility of the clients themselves. We evaluate this novel approach through simulations, under highly variable bandwidth conditions and in several multiclient scenarios. We show how the proposed approach can improve fairness up to 80% compared to state-of-the-art HAS heuristics in a scenario with three networks, each containing 30 clients streaming video at the same time.

Design and Evaluation of a Self-Learning HTTP Adaptive Video Streaming Client

HTTP Adaptive Streaming (HAS) is becoming the de facto standard for Over-The-Top (OTT)-based video streaming services such as YouTube and Netflix. By splitting a video into multiple segments of a couple of seconds and encoding each of these at multiple quality levels, HAS allows a video client to dynamically adapt the requested quality during the playout to react to network changes. However, state-of-the-art quality selection heuristics are deterministic and tailored to specific network configurations. Therefore, they are unable to cope with a vast range of highly dynamic network settings. In this letter, a novel Reinforcement Learning (RL)-based HAS client is presented and evaluated. The self-learning HAS client dynamically adapts its behaviour by interacting with the environment to optimize the Quality of Experience (QoE), the quality as perceived by the end-user. The proposed client has been thoroughly evaluated using a network-based simulator and is shown to outperform traditional HAS clients by up to 13% in a mobile network environment.

Design and optimisation of a FAQ-learning-based HTTP adaptive streaming client

In recent years, HTTP (Hypertext Transfer Protocol) adaptive streaming (HAS) has become the de facto standard for adaptive video streaming services. A HAS video consists of multiple segments, encoded at multiple quality levels. State-of-the-art HAS clients employ deterministic heuristics to dynamically adapt the requested quality level based on the perceived network conditions. Current HAS client heuristics are, however, hardwired to fit specific network configurations, making them less flexible to fit a vast range of settings. In this article, a (frequency adjusted) Q-learning HAS client is proposed. In contrast to existing heuristics, the proposed HAS client dynamically learns the optimal behaviour corresponding to the current network environment in order to optimise the quality of experience. Furthermore, the client has been optimised both in terms of global performance and convergence speed. Thorough evaluations show that the proposed client can outperform deterministic algorithms by 11–18% in terms of mean opinion score in a wide range of network configurations.

Design and evaluation of learning algorithms for dynamic resource management in virtual networks

Network virtualisation is considerably gaining attention as a solution to ossification of the Internet. However, the success of network virtualisation will depend in part on how efficiently the virtual networks utilise substrate network resources. In this paper, we propose a machine learning-based approach to virtual network resource management. We propose to model the substrate network as a decentralised system and introduce a learning algorithm in each substrate node and substrate link, providing self-organization capabilities. We propose a multiagent learning algorithm that carries out the substrate network resource management in a coordinated and decentralised way. The task of these agents is to use evaluative feedback to learn an optimal policy so as to dynamically allocate network resources to virtual nodes and links. The agents ensure that while the virtual networks have the resources they need at any given time, only the required resources are reserved for this purpose. Simulations show that our dynamic approach significantly improves the virtual network acceptance ratio and the maximum number of accepted virtual network requests at any time while ensuring that virtual network quality of service requirements such as packet drop rate and virtual link delay are not affected.

A probabilistic ontology-based platform for self-learning context-aware healthcare applications

Context-aware platforms consist of dynamic algorithms that take the context information into account to adapt the behavior of the applications. The relevant context information is modeled in a context model. Recently, a trend has emerged towards capturing the context in an ontology, which formally models the concepts within a certain domain, their relations and properties. Although much research has been done on the subject, the adoption of context-aware services in healthcare is lagging behind what could be expected. The main complaint made by users is that they had to signicantly alter workow patterns to accommodate the system. When new technology is introduced, the behavior of the users changes to adapt to it. Moreover, small dierences in user requirements often occur between dierent

Optimizing scalable video delivery through OpenFlow layer-based routing

In recent years, HTTP Adaptive Streaming (HAS) is becoming the de facto standard for video delivery over the best effort Internet. In HAS, the video consists out of multiple temporal segments encoded at different quality rates. In this way, HAS allows to dynamically adapt the quality level to the perceived network conditions. Using Scalable Video Coding (SVC), the redundancy between these representations can be eliminated, increasing the efficiency of server and caching infrastructure. Software Defined Networking (SDN) allows the dynamic adjustment of forwarding tables to reroute different flows. Using a combination of the layered characteristics of SVC and the dynamic routing of flows, the delivery of video can be optimized. In this paper, an algorithm is presented to dynamically calculate the optimal delivery paths for the different video layers. This enables guaranteeing a reliable and continuous video playout. Using this approach the number of video freezes can be reduced with 72% compared to shortest path routing.

A multi-agent Q-Learning-based framework for achieving fairness in HTTP Adaptive Streaming

HTTP Adaptive Streaming (HAS) is quickly becoming the de facto standard for Over-The-Top video streaming. In HAS, each video is temporally segmented and stored in different quality levels. Quality selection heuristics, deployed at the video player, allow dynamically requesting the most appropriate quality level based on the current network conditions. Todays heuristics are deterministic and static, and thus not able to perform well under highly dynamic network conditions. Moreover, in a multi-client scenario, issues concerning fairness among clients arise, meaning that different clients negatively influence each other as they compete for the same bandwidth. In this article, we propose a Reinforcement Learning-based quality selection algorithm able to achieve fairness in a multi-client setting. A key element of this approach is a coordination proxy in charge of facilitating the coordination among clients. The strength of this approach is three-fold. First, the algorithm is able to learn and adapt its policy depending on network conditions, unlike current HAS heuristics. Second, fairness is achieved without explicit communication among agents and thus no significant overhead is introduced into the network. Third, no modifications to the standard HAS architecture are required. By evaluating this novel approach through simulations, under mutable network conditions and in several multi-client scenarios, we are able to show how the proposed approach can improve system fairness up to 60% compared to current HAS heuristics.

A learning-based algorithm for improved bandwidth-awareness of adaptive streaming clients

HTTP Adaptive Streaming (HAS) is becoming the de-facto standard for Over-The-Top video streaming. A HAS video consists of multiple segments, encoded at multiple quality levels. Allowing the client to select the quality level for every segment, a smoother playback and a higher Quality of Experience (QoE) can be perceived. Although results are promising, current quality selection heuristics are generally hard coded. Fixed parameter values are used to provide an acceptable QoE under all circumstances, resulting in suboptimal solutions. Furthermore, many commercial HAS implementations focus on a video-on-demand scenario, where a large buffer size is used to avoid play-out freezes. When the focus is on a live TV scenario however, a low buffer size is typically preferred, as the video play-out delay should be as low as possible. Hard coded implementations using a fixed buffer size are not capable of dealing with both scenarios. In this paper, the concept of reinforcement learning is introduced at client side, allowing to adaptively change the parameter configuration for existing rate adaptation heuristics. Bandwidth characteristics are taken into account in the decision process, thus allowing to improve the clients bandwidth-awareness. Focus in this paper is on actively reducing the average buffer filling, evaluating results for two heuristics: the Microsoft IIS Smooth Streaming heuristic and the QoE-driven Rate Adaptation Heuristic for Adaptive video Streaming by Petrangeli et al. We show that using the proposed learning-based approach, the average buffer filling can be reduced by 8.3% compared to state of the art, while achieving a comparable level of QoE.

Deadline-based approach for improving delivery of SVC-based HTTP Adaptive Streaming content

HTTP Adaptive Streaming (HAS) has several advantages compared to traditional streaming protocols, such as easy traversal of firewalls and reuse of widely deployed HTTP infrastructure. HAS content is temporally segmented, and encoded at different quality representations, allowing the video player to autonomously adapt to network conditions by adapting play-out quality between subsequent segment downloads. However, to guarantee continuous playback, current-generation HAS protocols require a large play-out buffer. This makes them ill-suited for live television, as it significantly increases the live signal delay. This paper proposes a novel HAS solution for live streaming services. A HAS video player was designed that can cope with buffers as small as 2 seconds. This obviously requires the player to more rapidly react to bandwidth changes, which was achieved by using the Scalable Video Coding (SVC) extension of the H.264 Advanced Video Coding (AVC) video codec. Moreover, an intelligent network proxy was developed that guarantees the delivery of the SVC base quality layer using Differentiated Services (DiffServ). Furthermore, a more dynamic deadline-based approach is proposed which allows the client itself to decide which segments should be prioritized based on the risk of running into a buffer starvation. This enables more efficient use of the prioritized channel, leading to less freezes and increased quality and stability. The combination of these technologies allows the video player to align its quality adaptation decisions to the available bandwidth more efficiently and completely avoid buffer starvations. The small buffer size also reduces the total live signal delay from multiple dozens to only a few seconds.

Scalable Cache Management for ISP-Operated Content Delivery Services

Content delivery networks (CDNs) have been the prevalent method for the efficient delivery of content across the Internet. Management operations performed by CDNs are usually applied only based on limited information about Internet Service Provider (ISP) networks, which can have a negative impact on the utilization of ISP resources. To overcome these issues, previous research efforts have been investigating ISP-operated content delivery services, by which an ISP can deploy its own in-network caching infrastructure and implement its own cache management strategies. In this paper, we extend our previous work on ISP-operated content distribution and develop a novel scalable and efficient distributed approach to control the placement of content in the available caching points. The proposed approach relies on parallelizing the decision-making process and the use of network partitioning to cluster the distributed decision-making points, which enables fast reconfiguration and limits the volume of information required to take reconfiguration decisions. We evaluate the performance of our approach based on a wide range of parameters. The results demonstrate that the proposed solution can outperform previous approaches in terms of management overhead and complexity while offering similar network and caching performance.