The Effectiveness of Security Interventions on GitHub.

Since 2017, GitHub has been the first online open source platform to show security warnings to its users. It has since introduced further security interventions to help developers improve the security of their open source software. In this study, we investigate and compare the effects of these interventions. We perform time series analysis of security-altering commits to infer the causal effects of the interventions. Our analysis shows that while all of GitHub's security interventions have a significant positive effect on security, they differ greatly in their effect size. By comparing the design of each intervention, we identify the building blocks that worked well and those that did not. We also provide recommendations on how practitioners can improve the design of their interventions to enhance their effectiveness.