Improving Mmdb Distributed Transactional Concurrency

Main Memory Database Systems (MMDBs) have been studied since the 80s [3,4], when memory was quite costly ($1500 per MByte in 1984). We can now buy memory for about $10 per GByte. An advantage of MMDBs is that serial execution of a non-distributed transaction on a uniprocessor from start to finish saves the work of disk I/O, locking, latching and deadlock handling [7]. The 2013 Bulletin on Data Engineering [11] had eight articles on recent MMDBs and only three mentioned distributed transactions. Implementing fast, serializable, distributed transactions on an MMDB is difficult, since communication delays typically leave some CPUs idle and reduce total throughput.We began a project in Fall 2011 to improve distributed transactional performance of the open-source MMDB VoltDB system [14], which was based on an earlier academic prototype MMDB H-Store [1]. We developed a low-overhead concurrency method that executes consecutive Prepares with delayed Commits on each node (CPU) and takes Write locks but not Read locks to detect conflicts. We developed an Ordered Escrow Method, a variant of Escrow [14] to greatly speed up transactions with incremental updates. We named our VoltDB modification CVoltDB (C for Concurrency) and proved it supports replica consistency and serializability. Full TPC-B and TPC-C benchmarks demonstrate greatly improved performance due to new features in CVoltDB.