Statistical scheduling of offline comparative subjective evaluations for real-time multimedia

Authors:
Batu Sat;Benjamin W. Wah
Affiliations:
Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL;Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL
Venue:
IEEE Transactions on Multimedia - Special issue on quality-driven cross-layer design for multimedia communications
Year:
2009

Citing 4
Cited 1

Playout scheduling and loss-concealments in voip for optimizing conversational voice communication quality

Proceedings of the 15th international conference on Multimedia
The Design of a Multi-party VoIP Conferencing System over the Internet

ISM '07 Proceedings of the Ninth IEEE International Symposium on Multimedia
Statistical Testing of Off-line Comparative Subjective Evaluations for Optimizing Perceptual Conversational Quality in VoIP

ISM '08 Proceedings of the 2008 Tenth IEEE International Symposium on Multimedia
Voice quality prediction models and their application in VoIP networks

IEEE Transactions on Multimedia

Exploiting just-noticeable difference of delays for improving quality of experience in video conferencing

Proceedings of the 4th ACM Multimedia Systems Conference

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we study the statistical scheduling of offline subjective tests for evaluating alternative control schemes in real-time multimedia applications. These applications are characterized by multiple counteracting objective quality metrics (such as delay and signal quality) that can be affected by various control schemes. However, the trade-offs among these metrics with respect to the subjective preferences of users are not defined. As a result, it is difficult to select the proper control schemes that lead to the best subjective quality at run time. Since subjective tests are expensive to conduct and the number of possible control schemes and run-time conditions is prohibitively large, it is important that a minimum number of such tests be conducted offline, and that the results learned can be generalized to unseen conditions with statistical confidence. To this end, we study in this paper efficient algorithms for scheduling a sequence of subjective tests, while leaving the generalization of limited offline subjective tests to guide the operation of the control schemes at run time to a future paper. Using Monte Carlo simulations, we verify the robustness of our algorithms in terms of their accuracy and efficiency.