Pains, Gains and PLCs: Ten Lessons from Building an Industrial Control Systems Testbed for Security Research.

Recent years have seen a number of cyber attacks targeting Industrial Control Systems (ICSs). Reports detailing the findings from such attacks vary in detail. Handson experimental research is, therefore, required to better understand and explore security challenges in ICSs. However, real-world production systems are often offlimits due to the potential impact such research could have on operational processes and, in turn, safety. On the other hand, software-based simulations cannot always reflect all the potential device/system states due to over-simplified assumptions when modelling the hardware in question. As a result, laboratory-based ICS testbeds have become a key tool for research on ICS security. Development of such a testbed is a costly, labour - and timeintensive activity that must balance a range of design considerations, e.g., diversity of hardware and software platforms against scalability and complexity. Yet there is little coverage in existing literature on such design considerations, their implications and how to avoid typical pitfalls. Each group of researchers embarks on this journey from scratch, learning through a painful process of trial and error. In this paper we address this gap by reflecting on over 3 years of experience of building an extensive ICS testbed with a range of devices (e.g., PLCs, HMIs, RTUs) and software. We discuss the architecture of our testbed and reflect on our experience of addressing issues of diversity, scalability and complexity and design choices to manage trade-offs amongst these properties.