Trust but Verify: Crowdsourced Mobile Network Measurements and Statistical Validity Measures

Network operators, regulators, and big data companies use crowdsourced measurements to study the performance of mobile networks on a large scale. Such a type of measurement is defined as the collection and processing of data measured by the crowd, here the crowd of mobile subscribers. Crowdsourced network measurements make it relatively easy and inexpensive to obtain large amounts of network data that also reflect the quality actually received by the end user. However, this measurement method also involves some uncertainties, since, for example, it is not possible to precisely control when, where and with which devices measurements are taken. Thus, there is a tradeoff between the reliability of the individual measurement and the scope of the measurements. Therefore, how data of this type is analyzed is particularly important in order to obtain valid results. To address this issue, our paper defines concepts and guidelines for analyzing the validity of crowdsourced mobile network measurements. In particular, we address precision, for example the number of measurements needed to make valid statements, and also representativeness, for example the spatial and temporal distribution of the data. In addition to the formal definition of these two aspects, we illustrate the issue and possible evaluation approaches with the help of an extensive example data set. This data set consists of more than 11.7M crowdsourced mobile measurements from all over France, measured by a commercial mobile data provider. In the end, we provide an evaluation guideline and two possible use cases.