Proceedings of the IEEE First International Workshop on Quality of Multimedia Experience (QoMEX 2009), pp. 133-138, San Diego, July 29-31, 2009.

Gradient Ascent Paired-Comparison Subjective Quality Testing

doi: 10.1109/QOMEX.2009.5246964

Stephen D. Voran; Andrew A. Catellier

Abstract: (This paper won the QoMEX 2009 Best Paper Award.) Subjective testing is the most direct means of assessing audio, video, and multimedia quality as experienced by users and maximizing the information gathered while minimizing the number of trials is an important goal. We propose gradient ascent subjective testing (GAST) as an efficient way to locate optimizing sets of coding or transmission parameter values. GAST combines gradient ascent optimization techniques with paired-comparison subjective test trials to efficiently locate parameter values that maximize perceived quality. We used GAST to search a two-dimensional parameter space for the known region of maximal audio quality as proof-of-concept. That point was accurately located and we estimate that conventional testing would have required at least 27 times as many trials to generate the same results.


Keywords: protocols; switches; audio recording; error correction codes; humans; Quality assessment; Redundancy; Robustness; Software quality; Software testing

For technical information concerning this report, contact:

Stephen D. Voran
Institute for Telecommunication Sciences
(303) 497-3839

To request a reprint of this report, contact:

Lilli Segre, Publications Officer
Institute for Telecommunication Sciences
(303) 497-3572

Disclaimer: Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.

Back to Search Results