Implementation of a virtual time synchronizer for distributed databases

Multi-Version Concurrency Control (MVCC) is a database management technique that enables concurrent access to a database while maintaining consistency and isolation between transactions. By maintaining multiple versions of records, MVCC allows readers to access older versions of data while writers update the current version, ensuring that each transaction gets a consistent snapshot of the data without being blocked by others. This reduces the need for locks, allowing higher concurrency and better performance, especially in read-heavy environments. Each transaction is assigned a timestamp to determine which version of the data it can access, and older versions are eventually cleaned up through garbage collection. While MVCC provides high throughput and scalability, it introduces storage overhead due to the need to store multiple versions and can become complex as the number of versions grows. Additionally, MVCC can face issues like phantom reads and write skew, especially in transactions with complex interactions. Despite these challenges, MVCC is widely used in distributed and relational database systems, such as PostgreSQL and MySQL, to provide efficient concurrency control and ensure consistency without the need for heavy locking mechanisms. MVCC is particularly effective in environments where read and write operations are frequent, as it minimizes contention between transactions. By enabling transactions to work with a snapshot of the data, it avoids conflicts that would otherwise arise from simultaneous read and write operations. However, one downside is that the system must manage and track the various versions of each record, which can result in additional overhead for both storage and garbage collection. In some systems, when the number of versions becomes large, it may impact the system’s overall performance. It is especially beneficial in systems where many transactions read the same data concurrently while only a few perform updates. As database systems continue to evolve, improvements in MVCC implementations, such as optimized garbage collection, are helping to mitigate its limitations and enhance scalability. In conclusion, MVCC strikes a balance between concurrency and consistency, making it an ideal choice for certain high-performance, multi-user applications. This Paper addresses the storage overhead of MVCC with optimistic concurrency control.

Main-memory multicore transactional systems have achieved excellent performance using single-version optimistic concurrency control (OCC), especially on uncontended workloads. Nevertheless, systems based on other concurrency control protocols, such as hybrid OCC/ locking and variations on multiversion concurrency control (MVCC), are reported to outperform the best OCC systems, especially with increasing contention. This paper shows that implementation choices unrelated to concurrency control can explain some of these performance differences. Our evaluation shows the strengths and weaknesses of OCC, MVCC, and TicToc concurrency control under varying workloads and contention levels, and the importance of several implementation choices called basis factors. Given sensible basis factor choices, OCC performance does not collapse on high-contention TPC-C. We also present two optimization techniques, deferred updates and timestamp splitting, that can dramatically improve the high-contention performance of both OCC and MVCC. These techniques are known, but we apply them in a new context and highlight their potency: when combined, they lead to performance gains of \(4.74\times \) for MVCC and \(5.01\times \) for OCC in a TPC-C workload.

Optimistic concurrency control, or OCC, can achieve excellent performance on uncontended workloads for main-memory transactional databases. Contention causes OCC's performance to degrade, however, and recent concurrency control designs, such as hybrid OCC/locking systems and variations on multiversion concurrency control (MVCC), have claimed to outperform the best OCC systems. We evaluate several concurrency control designs under varying contention and varying workloads, including TPCC, and find that implementation choices unrelated to concurrency control may explain much of OCC's previously-reported degradation. When these implementation choices are made sensibly, OCC performance does not collapse on high-contention TPC-C. We also present two optimization techniques, commit-time updates and timestamp splitting , that can dramatically improve the high-contention performance of both OCC and MVCC. Though these techniques are known, we apply them in a new context and highlight their potency: when combined, they lead to performance gains of 3.4X for MVCC and 3.6X for OCC in a TPC-C workload.

Distributed storage systems run transactions across machines to ensure serializability in single-service architecture. Traditional protocols for distributed transactions are based on two-phase commit (2PC) or multi-version concurrency control (MVCC). 2PC serializes transactions as soon as they conflict and MVCC resorts to aborts, leaving many opportunities for con-currency on the distributed storage system. In the micro-service architecture, service nodes are deployed in heterogeneous distributed systems. 2PC and MVCC are struggling to break through the performance limitations of their existence. This paper presents 2PC+, a novel concurrency control protocol for distributed transactions that outperforms 2PC allowing more concurrency in micro-service architecture. 2PC+ is based on the traditional 2PC and MVCC, combined with transaction thread synchronization blocking optimization algorithm SAOLA. Then, we verify the algorithm SAOLA using the temporal logic of action TLA+ language. Finally, we compare 2PC+ to 2PC applying a case of Ctrip MSECP. When concurrent threads of the service call reach a certain threshold, 2PC+’s RT performance is improved by 2.87 times to 3.77 times compared with 2PC, and 2PC+’s TPS performance is 323.7% to 514.4% higher than 2PC.

A serializable concurrency control mechanism ensures consistency for OLTP systems at the expense of a reduced transaction throughput. A DBMS, therefore usually offers the possibility to allocate lower isolation levels for some transactions when it is safe to do so. However, such trading of consistency for efficiency does not come with any safety guarantees. In this article, we study the mixed robustness problem which asks whether, for a given set of transactions and a given allocation of isolation levels, every possible interleaved execution of those transactions that is allowed under the provided allocation is always serializable. That is, whether the given allocation is indeed safe. While robustness has already been studied in the literature for the homogeneous setting where all transactions are allocated the same isolation level, the heterogeneous setting that we consider in this article, despite its practical relevance, has largely been ignored. We focus on multiversion concurrency control and consider the isolation levels that are available in PostgreSQL and Oracle: read committed (RC), snapshot isolation (SI), and serializable snapshot isolation (SSI). We show that the mixed robustness problem can be decided in polynomial time. In addition, we provide a polynomial time algorithm for computing the optimal robust allocation for a given set of transactions, prioritizing lower over higher isolation levels. The present results therefore establish the groundwork to automate isolation level allocation within existing databases supporting multiversion concurrency control.

Concurrency control allows multiple tasks that share data objects to be concurrently executed in a serializable order, thus significantly improving computation progress. However, to accumulate forward progress on energy-harvesting intermittent systems while achieving data consistency across power cycles, existing approaches based on the checkpointing paradigm typically require system suspension at runtime. The runtime overheads incurred by suspension will be more manifest when more tasks are suspended and resumed during checkpointing, offsetting the computation progress improved by concurrent task execution. This paper presents a multiversion concurrency control design, which enables concurrent task execution without system suspension during checkpointing, while maintaining the serializability of task execution and ensuring data consistency after system recovery. We integrated our design into FreeRTOS running on a Texas Instruments device. Experimental results show that, at the very best, our design can double computation progress by reducing the runtime overheads incurred by system checkpointing, especially when tasks are executed with high concurrency.

The performance of a transaction processing system with the standard two-phase locking (2PL) concurrency control (CC) method (with the general waiting policy upon a lock conflict) may be degraded significantly due to transaction blocking in a high lock contention environment. In the limit this effect leads to the thrashing phenomenon, i.e., the majority of the transactions in the system become blocked. Limiting the wait depth of blocked transactions is an effective method to increase the number of active transactions in the system and to prevent thrashing, but this is at the cost of additional processing due to transaction restarts. The no-waiting (or immediate restart) policy limits the wait-depth to zero, while cautious waiting and the running priority policies use different methods to limit the wait depth to one. A variant of the wait depth limited (WDL) policy [8] also limits the wait depth to one, while attempting to minimize the wasted processing incurred by transaction aborts. A unified methodology to analyze the performance of the 2PL CC method with limited wait depth policies in a system with multiple transaction classes is described in this paper. The analysis is based on Markov chains representing the execution steps of each transaction in isolation, but as affected by hardware resource and data contention with other transactions in the system. Since the transition rates of the Markov chain are not known a priori, an iterative solution method is developed, which is then applied to the running priority and WDL policies. Simulation is used for validating the accuracy of the approximate analytic solutions. Of interest are the conservation laws governing the rate at which locks are transferred among transactions, which can be used to verify the correctness of the analysis.

A distributed database system is one in which the database is spread among several sites and application programs “move” from site to site to access and update the data they need. The concurrency control is that portion of the system that responds to the read and write requests of the application programs. Its job is to maintain the global consistency of the distributed database while ensuring that the termination of the application programs is not prevented by phenomena such as deadlock. We assume each individual site has its own local concurrency control which responds to requests at that site and can only communicate with concurrency controls at other sites when an application program moves from site to site, terminates, or aborts. This paper presents designs for several distributed concurrency controls and demonstrates that they work correctly. It also investigates some of the implications of global consistency of a distributed database and discusses phenomena that can prevent termination of application programs.

Stability and throughout in virtual time CSMA

For next generation information systems, concurrency control mechanisms are required to handle high level abstract operations and to meet high throughput demands. The currently available single level concurrency control mechanisms for reads and writes are inadequate for future complex information systems. In this paper, we will present a new multilevel concurrency protocol that uses a semantics-based notion of conflict, which is weaker than commutativity, called recoverability . Further, operations are scheduled according to relative conflict , a conflict notion based on the structure of operations. Performance evaluation via extensive simulation studies show that with our multilevel concurrency control protocol, the performance improvement is significant when compared to that of a single level two-phase locking based concurrency control scheme or to that of a multilevel concurrency control scheme based on commutativity alone. Further, simulation studies show that our new multilevel concurrency control protocol performs better even with resource contention.

Starvation-free secure multiversion concurrency control

Examines the effect of skewed database access on the transaction response time in a multisystem data sharing environment, where each computing node has access to shared data on disks, and has a local buffer of recently accessed granules. Skewness in data access can increase data contention since most accesses go to few data items. For the same reason, it can also increase the buffer hit probability. We quantify the resultant effect on the transaction response time, which depends not only on the various system parameters but also on the concurrency control (CC) protocol. Furthermore, the CC protocol can give rise to rerun transactions that have different buffer hit probabilities. In a multisystem environment, when a data block gets updated by a system, any copies of that block in other systems' local buffers are invalidated. Combining these effects, we find that higher skew does not necessarily lead to worse performance, and that with skewed access, optimistic CC is more robust than pessimistic CC. Examining the buffer hit probability as a function of the buffer size, we find that the effectiveness of additional buffer allocation can be broken down into multiple regions that depend on the access frequency distribution. >

With the rapid growth of database applications, wireless networking technology and mobile computing devices, there is a demand for processing mobile transactions in Mobile Ad-hoc Network (MANET) databases, so that mobile users can access and manipulate data anytime and anywhere. However, in order to guarantee timely and correct results for multiple concurrent transactions, concurrency control (CC) techniques become critical. Due to the characteristics of MANET databases, existing CC algorithms cannot work effectively. In this paper, we propose a CC algorithm called Sequential Order with Dynamic Adjustment (SODA) for MANET databases. In the design of SODA, the major characteristics of MANET databases are taken into consideration. SODA is based on optimistic CC to offer high concurrency and avoid unbounded blocking time, utilizes the sequential order of committed transactions to improve response time, and dynamically adjusts the sequential order of committed transactions to reduce aborts. The simulation results confirm that SODA has lower abort rate than other existing techniques.

Multi-version concurrency control scheme for a database system

In current research operating systems, support for objects and the persistence of these objects has been recognised as an important consideration. Research systems provide support for these concepts in the kernel. In these systems a great deal of effort has gone into integrating persistence and object structures with the other low level functions of the operating system. The low level nature of the support ensures that these concepts are in every respect a seamless part of the operating system which interacts sensibly with other components of the system. There has been a tendency in such systems to ignore the issues of concurrency control and resilience, with the attitude that these concepts are well understood by the database community, and as such, can be easily added on top of the system at a later date. A research project is described which aims to investigate the implementation of concurrency control and resilience support at the lowest level of the operating system. Concurrency control and resilience are considered to be sufficiently important and complex issues that they merit support at a low layer of the operating system in the same sense that support for persistence belongs at this level. It is a motivating belief for this project that support for fully fledged concurrency control systems in a truly seamless fashion requires appropriate low level functionality provided by the system. The goal of this project is the identification a set of low level operating system primitives through which a variety of both traditional and more advanced concurrency control methods can be implemented on a persistent store. This goal is being approached through the development of a prototype persistent store which can be used as both an experimental environment and a testbed for the demonstration of the viability of the projects results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>