How Do You Measure “Quality of Experience” in a Mobile Environment?

Mobile Network Operators (MNOs) have to adapt to a rapidly changing market. The explosion of smartphones(~1 Billion sold in 2014), the applications accessible through mobile networks and the advent of Over-the-Top (OTT) Services are clearly putting pressure on the traditional Service Providers to differentiate themselves, keep the customer base loyal and, most importantly, recover revenues that are currently flowing toward the OTT providers (or to other MNO competitors).

In the past, MNOs were just focused on the bandwidth that was available for mobile users: just think back to the evolution of network technologies for mobile data services. They went from 2G to 4G, passing through 2.5G and 3G. In that case, the only issue was increasing bandwidth, with no real focus on the services running on it (let me say that the only advertised service was a generic Internet access). They based this action on the assumption “more bandwidth = more subscribers”.

Later, OTT providers appeared, relegating the MNO to the role of dumb pipes providers, creating an environment where MNOs have invested significant amounts of money to increase network capacity, only to have their subscribers access services provided (and billed by) by other entities! So how can MNOs reestablish differentiate themselves with subscribers? One of the best ways to do this is by regaining ownership of the services, emphasizing what they can control on the network, that is, the quality of the offering. But, in order to do that, the MNOs need to have a full visibility regarding who is accessing the network, through which devices, from where, doing what and most important of all, what is the quality they are experiencing.

Why QoS isn’t enough

Various standard bodies and industry organizations have defined in the past how to measure the so called Quality of Service (QoS), meaning “the ability of a network to provide a service with an assured service level”. QoS was defined by means of a set of technical metrics (a.k.a. Key Performance Indicators or KPI) such as Packet Loss, Delay, Throughput, Answer-to-Seizure Ratio, Call Setup Time and so on. It should be noted that QoS was a valid approach in fixed line environments to derive the experience perceived by the subscribers due to the concept of “if the network is good, the service is good”.

Then, to better understand of the quality of voice delivery from a subscriber’s point of view (their perceived experience), the standard bodies and industry organizations developed the Mean Opinion Score (MOS), which provided a subjective measurement of the quality of a call. This methodology has evolved over the years to accommodate the transition from circuit-switch services to packet-switched networks carrying voice.

With the introduction of the data services over mobile networks, QoS has proven to be a valid way to check the health of the physical network and of the interconnection links between nodes, but lacks the ability to represent the objective perception of the quality from the subscriber point of view. While MOS was effective for voice, there is now a gap as it pertains to data services.

The Challenge: Smarter Scoring of QoE

MNOs have no shortage of service assurance solutions that generate hundreds or thousands of KPIs covering specific network domains. However, they are lacking an easy way to obtain a holistic view of the experience provided to their subscribers, because such solutions are totally uncorrelated and designed to score the service and/or the infrastructure performance, but only from the network point of view.

What is needed today is the definition of a computational model that goes beyond the QoS paradigm and is capable of objectively assessing the QoE, taking into account (and correlating) many parameters including:

Network dependent factors

Examples include application response time or the time a web page takes to be displayed on a device.

User expectations

“If I pay more, I expect more”.

Note this is required to provide context to the analysis and will vary based on the subscribers contract. Therefore the subscriber’s profile should be an input to the algorithm.

Context of use

The type of user can be categorized as Private, Business, M2M…

Application-specific factors

If I am simply browsing the Internet my QoE must be scored differently than when I am watching a video.

The Future of the QoE

Empirix is actively working to define and integrate a fully configurable, objective model for scoring QoE that will be applicable to any service or any category of subscribers, into its IntelliSight Analytics platform. The goal is to provide MNOs with an objective means to measure their customers’ experience, enabling service providers to quickly and proactively address issues and increase customer satisfaction. Is your company thinking about how to improve its visibility on the Quality of Experience offered to your subscribers? Let me know your view of QoE in the comments below. And stay tuned for upcoming posts that will discuss how to improve the quality of the services you offer.

More on Quality of Experience (QoE)

The ROI of CX Testing and Monitoring
Testing and Monitoring

QoE with your businesses’ communication systems determines success Your customers’ satisfaction and loyalty hinges on the quality of their experiences. When there’s a breakdown in a company’s communication environment, customers…