Jonathan Dunne

Monitoring VoIP call quality using improved simplified E-model

ITU-T recommendation G.107 introduced the E-model, a repeatable way to assess if a network is prepared to carry a VoIP call or not. Various studies show that the E-model is complex with many factors to be used in monitoring purposes. Consequently, simplified versions of the E-model have been proposed to simplify the calculations and focus on the most important factors required for monitoring the call quality. In this paper, we propose simple correction to a simplified E-model; we show how to calculate the correction coefficients for 4 common codecs (G.711, G.723.1, G.726 and G.729A) and then we show that its predictions better match PESQ scores by implementing it in a monitoring application.

Accuracy analysis on call quality assessments in voice over IP

Voice over IP (VoIP) now has tremendous influence on the telecommunication market with its flexibility and price advantage. Users of VoIP expect call quality to be as good as, if not better than the traditional Public Switched Telephone Network (PSTN). However in VoIP, factors that are related to the IP transport network such as packet loss, delay, bandwidth, jitter, and voice encoding (codec) all affect call quality. Call quality assessment in VoIP systems is mainly conducted with off-line tests using the Perceptual Evaluation of Speech Quality (PESQ) [1] methodology. Another method that can be utilised is an on-line approach using the E-Model, which can be used in real time. However, these two methods have limits and inaccuracy, and often do not give the same results. Call quality assessment is often used to adjust system and codec parameters. Therefore, given inaccurate results, the system would decrease the adjustment efficiency or even inadvertently decrease call quality. The primary contribution of this paper is a comparison between the accuracy of PESQ and the E-Model investigated by conducting an extensive set of experiments in a real enterprise network using a widely deployed Voice over IP (VoIP) product. Experiments were conducted under varying controlled network conditions. The results show that under various conditions, loss rates, codecs and across a range of languages that there can be significant differences between the call quality measurement obtained when using the E-model versus a PESQ analysis.

Improved E-model for monitoring quality of multi-party VoIP communications

Maintaining good Quality-of-Experience (QoE) is crucial for Voice-over-IP (VoIP) applications, particularly those operating across the public Internet. Accurate online estimation of QoE as perceived by end users allows VoIP applications take steps to improve QoE when it falls below acceptable levels. ITU-T recommendation G.107 introduced the E-model, which provides a means to assess QoE levels for two-party VoIP sessions. In this paper we provide an analysis of the accuracy of the E-model for multi-party VoIP sessions when all audio is processed by a centralised focus node. We analyse the impact of what we term the “Focus Transcoding Effect (FTE),” the “Focus Forwarding Effect (FFE),” and the number of end-points participating in the session. Through comparison to QoE metrics produced by the offline PESQ method for three common audio codecs, we show that the standard E-model does not provide accurate QoE assessment for multi-party VoIP sessions. We then introduce an improved Emodel for these codecs for multi-party VoIP sessions. We describe the implementation of the improved E-model in a QoE monitoring application, showing that it produces results similar to actual PESQ scores.

A call quality assessment and analysis framework for video telephony applications in enterprise networks

We propose a generalised testing framework to evaluate the end-user perceptual quality of video over IP solutions in enterprise networks. The framework automates testing of communications applications to produce Quality-of-Experience (QoE) results in terms of industry defined standard metrics like PSNR, SSIM and video MOS. Furthermore, it aids in network planning by relating performance indices to network characteristics. Experiments are carried out in an emulated network environment to realise and test the application under varied network impairments. We describe a sample realization of framework using a suite of open source tools and application client extensions. We present a comparative analysis involving performance of two different popular enterprise VVoIP applications which explains the usage of framework as a quality verification and analysis tool. We also present a brief accuracy analysis of Opinion Model using Video Quality Metric (VQM).

Obscured by the cloud: A resource allocation framework to model cloud outage events

Abstract As Small Medium Enterprises (SMEs) adopt Cloud technologies to provide high value customer offerings, uptime is considered important. Cloud outages represent a challenge to SMEs and micro teams to maintain a services platform. If a Cloud platform suffers from downtime this can have a negative effect on business revenue. Additionally, outages can divert resources from product development/delivery tasks to reactive remediation. These challenges are immediate for SMEs or micro teams with a small levels of resources. In this paper we present a framework that can model the arrival of Cloud outage events. This framework can be used by DevOps teams to manage their scarce pool of resources to resolve outages, thereby minimising impact to service delivery. We analysed over 300 Cloud outage events from an enterprise data set. We modelled the inter-arrival and service times of each outage event and found a Pareto and a lognormal distribution to be a suitable fit. We used this result to produce a special case of the G/G/1 queue system to predict busy times of DevOps personnel. We also investigated dependence between overlapping outage events. Our predictive queuing model compared favourably with observed data, 72% precision was achieved using one million simulations.

Social testing: A framework to support adoption of continuous delivery by small medium enterprises

Continuous delivery (CD) represents a challenge for software test teams, because of the continuous introduction of new features and feedback from customers. We consider testing in a framework where users are encouraged to report defects through social or other incentive schemes. Using an enterprise dataset, we address the question of which types of defects can best be found in the field, allowing in-house test resources to be refocused. Validation of these touch points ultimately interweaves both customer and business needs. The proposed framework is one which can help small to medium software businesses, which typically have limited resources to test and release software via CD.

A new adaptive redundancy control algorithm for VoIP applications

Packet loss is one of the most important factors in degrading Voice over IP (VoIP) perceived call quality. Forward Error Correction (FEC) is a powerful technique for transmitting audio streams over the IP network to decrease the effect of packet loss. Although these method reduces the effect of packet loss, it increases the bandwidth and delay in order to recover from the lost packets. In this paper, we propose a new adaptive FEC mechanism for voice calls based on the generated codewords from a Reed-Solomon (RS) encoder. This mechanism chooses the optimum RS code from a family of codes to improve the conversational call quality. Our proposed mechanism is able to switch between different codes during the call to account for the variation of the network conditions including packet loss and delay. We have deduced the proposed algorithm by performing subjective mean opinion score (MOS) testing based on an interactive assessment tests. We show that our adaptive algorithm outperforms fixed RS codes under highly varying network conditions.

Social dogfood: A framework to minimise clouc field defects through crowd sourced testing

Delivering software for the Cloud represents a challenge for both micro teams and Small Medium Enterprises (SMEs), in part due to the rapid release methods adopted and the numerous ways in which software defects can be detected. We study field defect detection rates in a framework where these rates are used to refocus in-house test resources. Using an enterprise dataset, we address the question of what types of defects are found in the field and how soon after a system goes live defects are detected. Our framework can aid both micro teams and SMEs to minimise the number of defects found in the field by maximising internal usage through ‘Dogfood’ programs and by leveraging crowdsourced test methodologies.

Are you being served: A framework to manage Cloud outage repair times for Small Medium Enterprises

Hosting software applications in a Cloud based infrastructure represents challenges for Small Medium Enterprises (SMEs), due to the variety of ways in which production outages can occur. We consider repair times for outage events in a framework where these downtimes are used to re-focus Systems Operations resources. Using an enterprise dataset, we address the question of how outage events are distributed and what relationship these events have with different types of failures that can occur in a cloud data centre. The proposed framework can aid SMEs to maintain a highly available On-Demand service infrastructure, with limited resources.