Javier Sánchez Pérez (RTVE)
Since the dawn of audiovisual broadcasting there has been the need to monitor the quality of the content produced and broadcast. Until now, the complexity of this task has required human intervention, with technicians viewing and analysing screens with the content on a subjective basis.
Developments in the audiovisual market have seen an unprecedented increase in the quantity and complexity of audiovisual media. As a result, the task of measuring quality has become a problem, as the number of content channels from producers, platforms, integrators, etc., has increased exponentially.
Despite this, quality continues to be checked (selectively and on certain channels at certain times) by human observation.
Quality of Experience
Quality of service (QoS) refers to parameters that can be measured objectively. Measurement depends on the availability of the channel and the received signal. Parameters include coverage, continuity of service functionality, reliability, compatibility of devices, and functions involved in service availability.
By contrast, quality of experience (QoE) is the overall quality perceived by the end viewer and so it is a wholly subjective assessment (Figure 1). QoE represents the viewer’s perception of how good video looks, how good audio sounds, how well combined audiovisual content is perceived or how well interaction with an audiovisual service works.
Figure 1. Degradation factors and perceived distortions that can arise in the end-to-end content distribution chain
Good QoS means that video content is objectively satisfactory, but it doesn’t necessarily follow that the viewer’s subjective QoE experience will also be positive.
Mean opinion score
Video quality is usually evaluated subjectively using the mean opinion score (MOS) parameter. The International Telecommunication Union has a recommendation (ITU-R BT.500- 11) that sets out criteria for subjective assessment (Figure 2). This method relies on the availability of human technicians to view thousands of channels of content. But it is this human involvement and the special facilities necessary to display content that make the process extremely expensive.
Video–MOS, a technology start-up based in Madrid, has developed a solution to this conundrum by using pioneering technology that combines video analytics and artificial intelligence (AI). Extensive testing in a real- world environment has shown that it is now possible to remove the need for human involvement in MOS measurement, replacing it with automated tools. These tools can estimate the QoE without reference (i.e., there is no need to have the original pre-broadcast content), performing real-time monitoring of the MOS based on psycho-visual impact and not the difference between images.
Automated measurement
The use of these tools was tested over a period of six months by RTVE and Video-MOS within the framework of Impulsa Visión Empresas, RTVE’s sandbox. In these tests, monitoring environments were deployed for three practical situations: monitoring of the distribution chain, monitoring of the programming grid, and analysis of the configuration of the entire Television Española DVB-T multiplex.
There are many areas in which the solution adds value, including:
- Monitoring. It is important to measure the quality perceived by users in order to optimize the network. Technology is needed to collect and systematize the measurements, manage the network and monitor the status of the network in a regular, simple and controlled way.
- Quality control and management. An online measurement of perceptual quality can be used for control and administration purposes. This information can be fed back to the production and broadcasting infrastructure, so that immediate action can be taken to maintain QoE.
- Management of resources. For premium content or prime-time slots, it is important to measure in real time the perceived quality and estimate how it can be affected during broadcast.
- Monitoring and control of the quality conditions. These are usually incorporated in the contract of the service provider.
- New Developments. Perceived quality measurements are a necessary tool for the evaluation and development of new systems or algorithms. Having a measurement of the perceived quality that can be performed automatically avoids having to perform long and expensive subjective tests in the design of new de/encoders, video enhancement algorithms, etc.
The use of a metric as simple as the MOS that is easily understood by technical and non-technical staff, can be used for the entire chain, and is easy to deploy, are clear advantages over the current solutions available. Together, these advantages provide significant efficiency gains in the production and dissemination of audiovisual content.
Results
The Video-MOS tool enables direct monitoring of QoE, differentiating between technical and artistic distortions and allowing complete analysis of the impact on the content as seen by the end users, letting know the impact on some of your programmes (Figure 3).
Figure 3. Summary of the results of the automated QoE analysis of RTVE’s La 1 HD service
Real-world testing has demonstrated the considerable benefits that this tool provides in the early detection of negative factors that affect the viewer experience, in addition to the ability to analyse the impact on quality control in relation to different configurations of the distribution network.
This article was first published in issue 51 of tech-i magazine.