Laser: A Living Analytics Experimentation System For Large-Scale Online Controlled Experiments

Tracking user browsing data and measuring the effectiveness of website design and web services are important to businesses that want to attract the consumers today who spend much more time online than before. Instead of using randomized controlled experiments, the existing approach simply tracks user browsing behaviors before and after a change is made to website design or web services, and evaluate the differences. To address the effects caused by hidden factors (e.g. promotion activities on the website) and to give fair comparison of different website designs, we propose the LASER system, a unified experimentation platform that enables randomized online controlled experiments to be easily conducted with minimal human effort and modifications to the experimented websites. More importantly, the LASER system manages the various aspects of online controlled experiments, namely the selection of participants into groups, exposure of different user interface features or recommendation algorithms to these groups, measuring their responses, and summarizing the results in the visual manner.