Ricky K. P. Mok

QDASH: a QoE-aware DASH system

Dynamic Adaptation Streaming over HTTP (DASH) enhances the Quality of Experience (QoE) for users by automatically switching quality levels according to network conditions. Various adaptation schemes have been proposed to select the most suitable quality level during video playback. Adaptation schemes are currently based on the measured TCP throughput received by the video player. Although video buffer can mitigate throughput fluctuations, it does not take into account the effect of the transition of quality levels on the QoE. In this paper, we propose a QoE-aware DASH system (or QDASH) to improve the user-perceived quality of video watching. We integrate available bandwidth measurement into the video data probes with a measurement proxy architecture. We have found that our available bandwidth measurement method facilitates the selection of video quality levels. Moreover, we assess the QoE of the quality transitions by carrying out subjective experiments. Our results show that users prefer a gradual quality change between the best and worst quality levels, instead of an abrupt switching. Hence, we propose a QoE-aware quality adaptation algorithm for DASH based on our findings. Finally, we integrate both network measurement and the QoE-aware quality adaptation into a comprehensive DASH system.

Inferring the QoE of HTTP video streaming from user-viewing activities

HTTP video streaming, employed by most of the video-sharing websites, allows users to control the video playback using, for example, pausing and switching the bit rate. These user-viewing activities can be used to mitigate the temporal structure impairments of the video quality. On the other hand, other activities, such as mouse movement, do not help reduce the impairment level. In this paper, we have performed subjective experiments to analyze user-viewing activities and correlate them with network path performance and user quality of experience. The results show that network measurement alone may miss important information about user dissatisfaction with the video quality. Moreover, video impairments can trigger user-viewing activities, notably pausing and reducing the screen size. By including the pause events into the prediction model, we can increase its explanatory power.

Evolutionary multimodal optimization using the principle of locality

The principle of locality is one of the most widely used concepts in designing computing systems. To explore the principle in evolutionary computation, crowding differential evolution is incorporated with locality for multimodal optimization. Instead of generating trial vectors randomly, the first method proposed takes advantage of spatial locality to generate trial vectors. Temporal locality is also adopted to help generate offspring in the second method proposed. Temporal and spatial locality are then applied together in the third method proposed. Numerical experiments are conducted to compare the proposed methods with the state-of-the-art methods on benchmark functions. Experimental analysis is undertaken to observe the effect of locality and the synergy between temporal locality and spatial locality. Further experiments are also conducted on two application problems. One is the varied-line-spacing holographic grating design problem, while the other is the protein structure prediction problem. The numerical results demonstrate the effectiveness of the methods proposed.

On the accuracy of smartphone-based mobile network measurement

As most of mobile apps rely on network connections for their operations, measuring and understanding the performance of mobile networks is becoming very important for end users and operators. Despite the availability of many measurement apps, their measurement accuracy has not received sufficient scrutiny. In this paper, we appraise the accuracy of smartphone-based network performance measurement using the Android platform and the network round-trip time as the metric. We use a multiple-sniffer testbed to overcome the challenge of obtaining a complete trace for acquiring the required timestamps. Our experiment results show that the RTTs measured by the apps are all inflated, ranging from a few milliseconds (ms) to tens of milliseconds. Moreover, the 95% confidence interval can be as high as 2.4ms. A finer-grained analysis reveals that the delay inflation can be introduced both in the Dalvik VM (DVM) and below the Linux kernel. The in-DVM overhead can be mitigated but the other cannot be. Finally, we propose and implement a native app which uses HTTP messages for network measurement, and the delay inflation can be kept under 5ms for almost all cases.

Appraising the delay accuracy in browser-based network measurement

Conducting network measurement in a web browser (e.g., speedtest and Netalyzr) enables end users to understand their network and application performance. However, very little is known about the (in)accuracy of the various methods used in these tools. In this paper, we evaluate the accuracy of ten HTTP-based and TCP socket-based methods for measuring the round-trip time (RTT) with the five most popular browsers on Linux and Windows. Our measurement results show that the delay overheads incurred in most of the HTTP-based methods are too large to ignore. Moreover, the overheads incurred by some methods (such as Flash GET and POST) vary significantly across different browsers and systems, making it very difficult to calibrate. The socket-based methods, on the other hand, incur much smaller overhead.Another interesting and important finding is that Date.getTime(), a typical timing API in Java, does not provide the millisecond resolution assumed by many measurement tools on some OSes (e.g., Windows 7). This results in a serious under-estimation of RTT. On the other hand, some tools over-estimate the RTT by including the TCP handshaking phase.

TRIO: measuring asymmetric capacity with three minimum round-trip times

Measuring network path capacity is an important capability to many Internet applications. But despite over ten years of effort, the capacity measurement problem is far from being completely solved. This paper addresses the problem of measuring network paths of asymmetric capacity without requiring the remote nodes control or overwhelming the bottleneck link. We first show through analysis and measurement that the current packet-dispersion methods, due to the packet size limitations, can only measure up to a certain degree of capacity asymmetry. Second, we propose TRIO that removes the limitation by using round-trip times (RTTs). TRIO cleverly exploits two types of probes to obtain three minimum RTTs to compute bothforward and reverse capacities, and another minimum RTT for measurement validation. We validate TRIOs accuracy and versatility on a testbed and the Internet, and develop a system to measure path capacity from the server or user side.

Detecting Low-Quality Workers in QoE Crowdtesting: A Worker Behavior-Based Approach

QoE crowdtesting is increasingly popular among researchers to conduct subjective assessments of network services. Experimenters can easily access a huge pool of human subjects through crowdsourcing platforms. Without any supervision, low-quality workers, however, can threaten the reliability of the assessments. One of the approaches in classifying the quality of workers is to analyze their behavior during the experiments, such as mouse cursor trajectory. However, existing works analyze the trajectory coarsely, which cannot fully extract the imbedded information. In this paper, we propose a novel method to detect low-quality workers in QoE crowdtesting by analyzing the worker behavior. Our approach is to construct a predictive model by using supervised learning algorithms. A quality score is computed by applying existing anti-cheating techniques and human inspections to label the workers. We define a set of ten worker behavior metrics, which quantifies different types of worker behavior, including finer-grained cursor trajectory analysis. A multiclass Naïve Bayes classifier is applied to train a model to predict the quality of workers from the metrics. We have conducted video QoE assessments on Amazon Mechanical Turk and CrowdFlower to collect the worker behavior. Our results show that the error rates of the model trained from four metrics are equal or less than 30%. We further find that combining the predictions from the four different 5-point Likert scale rating methods can improve the success rate in detecting low-quality workers to around 80%. Finally, our method is 16.5% and 42.9% better in precision and recall than CrowdMOS.

Improving the Packet Send-Time Accuracy in Embedded Devices

A number of projects deploy Linux-based embedded systems to carry out large-scale active network measurement and network experiments. Due to resource constrains and the increase of network speed, obtaining sound measurement results from these low-end devices is very challenging. In this paper, we present a novel network primitive, OMware, to improve the packet send-time accuracy by enabling the measurement application to pre-dispatch the packet content and its schedule into the kernel. By this pre-dispatch approach, OMware can also reduce the overheads in timestamp retrievals and sleeping, and the interference from other application processes.

Revealing the Load-Balancing Behavior of YouTube Traffic on Interdomain Links

For the last decade, YouTube has consistently been a dominant source of traffic on the Internet. To improve the quality of experience (QoE) for YouTube users, broadband access providers and Google apply techniques to load balance the extraordinary volume of web requests and traffic. We use traceroute-based measurement methods to infer these techniques for assigning YouTube requests to specific Google video content caches, including the interconnection links between the access providers and Google. We then use a year of measurements (mid-2016 to mid-2017) collected from SamKnows probes hosted by broadband customers spanning a major ISP in the U.S. and three ISPs in Europe. We investigate two possible causes of different interdomain link usage behavior. We also compare the YouTube video cache hostnames and IPs observed by the probes, and find that the selection of video cache has little impact on BGP selection of interdomain links.

Demystifying and Puncturing the Inflated Delay in Smartphone-based WiFi Network Measurement

Using network measurement apps has become a very effective approach to crowdsourcing WiFi network performance data. However, these apps usually measure the user-level performance metrics instead of the network-level performance which is important for diagnosing performance problems. In this paper we report for the first time that a major source of measurement noises comes from the periodical SDIO (Secure Digital Input Output) bus sleep inside the phone. The additional latency introduced by SDIO and Power Saving Mode can inflate and unstablize network delay measurement significantly. We carefully design and implement a scheme to wake up the phone for delay measurement by sending just enough warm-up and background traffic. Our evaluation results show that the overall median delay overheads can be kept within 3ms, regardless of the actual network delay.