On the Number of Subjects Needed for 360° Video Quality Experiments: An SOS Based Analysis

Subjective 360° video quality experiments provide the ground truth for developing related systems such that acceptable quality of experience is offered to the users. To obtain statistically meaningful insights from the data recorded in quality experiments, engaging a sufficiently large number of subjects is essential. In contrast to conventional videos, little has been reported on statistical measures that may guide the choice of the number of subjects required for 360° video quality experiments. In this paper, the standard deviation of opinion score (SOS) hypothesis, measuring the diversity among the opinion scores given by a panel of subjects, is used to reveal the number of subjects needed. It is shown that the number of subjects needed varies depending on a threshold on SOS characteristics in terms of diversity among opinion scores.