Enabling End-to-End Simulation for Host Networking Evaluation using SimBricks

Full system "end-to-end" measurements in physical testbeds are the gold standard for evaluation of network systems but are fraught with challenges. Adequate testbeds are often not available, as projects target next generation devices, propose new hardware, or require larger scale. Further, evaluations in testbeds limit what we can observe without affecting system behavior, are frequently hard to reproduce, and are only available to groups with sufficient funding. Yet, we lack an accepted alternative, leaving us with ad-hoc non-end-to-end evaluations that do not form a solid basis for future work. We argue that full system simulations enable comparable end-to-end evaluation and are the next best alternative when a physical testbed is not available. To this end, we present SimBricks, a modular full system simulation framework for network systems. SimBricks combines multiple existing simulators for individual components, including processor and memory, NIC, and network, into full virtual testbeds running unmodified software. The architecture combines well-defined component interfaces for extensibility with new simulators, efficient communication channels for local and distributed simulators, and a novel efficient synchronization protocol for accurate timing across different simulators. We demonstrate that SimBricks simulations reproduce key findings from prior work in congestion control, NIC architecture, and in-network computing, and show scalability to 1000 simulated hosts.