June 2026 | Journal Article
Fix Your Netflix Experiment
Cite This Publication
Lucjan Janowski et al., “Fix Your Netflix Experiment.” This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
Lucjan Janowski 
, Margaret H. Pinson , Dominika Wanat , Kamil Koniuch , Katrien De Moor , and Mark D. Gross
Abstract:
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
This paper examines the hypothesis that asking people about video quality changes their behavior. We conducted an Absolute Category Rating (ACR) scale test as a baseline. We also conducted an experiment in which subjects chose a movie to watch and pressed a button whenever the quality disturbed their watching experience. This experiment design could lead to improved understanding of bitrate ladders. The action-focused experiment, whereby most subjects tolerated only very high quality video, revealed an extremely large diversity of button pressing behaviors. Our analyses indicate that each subject has their own unique lower threshold for the video quality that they tolerate. We observed two subgroups of subjects, one demanding and the other relaxed. The button pressing threshold of the demanding subgroup maps to between good and excellent on the ACR scale; that of the relaxed subgroup maps to between fair and good. An area for future work is an algorithm that differentiates between demanding and relaxed subjects
Keywords: video quality; subjective test; quality of experience (QoE); user experience (UX)
For technical information concerning this report, contact:
Lucjan Janowski
AGH University of Science and Technology, Kraków, Poland
For funding information concerning this report, click this link.
Funding Information
Performing Agency
U.S. Department of Commerce
National Telecommunications and Information Administration
Institute for Telecommunication Sciences
325 Broadway
Boulder, CO 80305
https://ror.org/00mj5bc69
Disclaimer:
Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.
For questions or information on this or any other NTIA scientific publication, contact the ITS Publications Office at ITSinfo@ntia.gov or 303-497-3572.