Optimizing Data Modification

Several researchers have recently considered a database support to enhance the process of programming. Yet there are very few, if at all, working models which could prove that the idea is sound. This paper presents an example of a software system prototyped in Prolog. Within one language system there was a provision to develop a database of structured objects (e.g., parse trees), to create a natural language interface and to write a semi-compiler. The experiences convince us that software prototyping in Prolog is unusually rapid. Much of the conveniences are due to the database and specification organized programming which is inherent to Prolog. Further enhancement will come from the use of database tools like error recovery, concurrency control, integrity constraint, etc. Prolog thus offers potentials for software development environments far beyond the existing ones.

In OLTP systems, serializability is a key property for executions; without this, integrity constraints on data can be violated due to concurrent activities. The serializability property can be guaranteed through the implementation of concurrency control mechanisms such as strict 2-phase locking (S2PL). However, modern OLTP systems are expected to execute millions of transactions concurrently and mechanisms such as S2PL do not scale to handle this level of concurrency. Thus, to get better performance, programmers may choose different isolation levels, rather than the default ones, which are more suitable for their applications. Due to differing requirements, transaction aborts may be high. Thus, this paper investigates the correctness of mixing transactions with different isolation levels and extends the mixing theorem introduced by Adya et al. The extension investigates the correctness of mixing transactions with different isolation levels, especially when optimistic concurrency control (OCC) schemes are used which are not the main focus of the theorem. We thus propose OCC2T, an OCC-based algorithm that is designed to improve the performance of DBMS by decreasing the abort rate of transactions and enhancing mixing transactions with different isolation levels. Our results show that OCC2T is able to reduce the abort rate in highly contended workloads by almost 5x over other algorithms.

Modern multimedia applications require new services from the communication systems. To negotiate and to manage the quality of service parameters, and in the case of collaborative work, to support synchronization and concurrency control, control information between the group members has to be exchanged. There are some approaches where a central group server supports the group and QoS management. To avoid the problems of central servers such as server failures and performance bottlenecks on the server side a distributed scheme for group and QoS management is more suitable. We present a protocol that supports distributed group and QoS management. The protocol is part of a conferencing system designed for applications of computer supported cooperative work. The system is based on the asynchronous transfer mode. The group communication protocol provides reliable, atomic and totally ordered delivery for a group of communicating partners. A design goal was simplicity to limit the overhead introduced by the group communication because the transmission and processing of video streams consumes a lot of bandwidth and processing power respectively.

In many distributed systems concurrent access is required to a shared object, where abstract object servers may incorporate type-specific properties to define consistency requirements. Each operation and its outcome is treated as an event, and conflicts may occur between different event types. Hence concurrency control and synchronization are required at the granularity of conflicting event types. With such a fine granularity of locking, the occurrence of conflicts is likely to be lower than with whole-object locking, so optimistic techniques become more attractive. This work describes the design, implementation, and performance of servers for a shared atomic object, a semiqueue, where each server employs either pessimistic or optimistic locking techniques on each conflicting event type. We compare the performance of a purely optimistic server, a purely pessimistic server, and a hybrid server which treats certain event types optimistically and others pessimistically, to demonstrate the most appropriate environment for using pessimistic, optimistic, or hybrid control. We show that the advantages of low overhead on optimistic locking at low conflict levels is offset at higher conflict levels by the wasted work done by aborted transactions. To achieve optimum performance over the whole range of conflict levels, an adaptable server is required, whereby the treatment of conflicting event types can be changed dynamically between optimistic and pessimistic, according to various criteria depending on the expected frequency of conflict. We describe our implementations of adaptable servers which may allocate concurrency control strategy on the basis of state information, the history of conflicts encountered, or by using preset transaction priorities. We show that the adaptable servers perform almost as well as the best of the purely optimistic, pessimistic, or hybrid servers under the whole range of conflict levels, showing the versatility and efficiency of the dynamic servers. Finally we outline a general design methodology for implementing adaptable concurrency control in servers for atomic objects, illustrated using an atomic shared B-tree.

Multi-engine analytics has been gaining an increasing amount of attention from both the academic and the industrial community as it can successfully cope with the heterogeneity and complexity that the plethora of frameworks, technologies and requirements have brought forth. It is now common for a data analyst to combine data that resides on multiple and totally independent engines and perform complex analytics queries. Multi-engine solutions based on SQL can facilitate such efforts, as SQL is a popular standard that the majority of data-scientists understands. Existing solutions propose a middleware that centrally optimizes query execution for multiple engines. Yet, this approach requires manual integration of every primitive engine operator along with its cost model, rendering the process of adding new operators or engines highly inextensible. To address this issue we present MuSQLE, a system for SQL-based analytics over multi-engine environments. MuSQLE can efficiently utilize external SQL engines allowing for both intra and inter engine optimizations. Our framework adopts a novel API-based strategy. Instead of manual integration, MuSQLE specifies a generic API, used for the cost estimation and query execution, that needs to be implemented for each SQL engine endpoint. Our engine API is integrated with a state-of-the-art query optimizer, adding support for location-based, multi-engine query optimization and letting individual runtimes perform sub-query physical optimization. The derived multi-engine plans are executed using the Spark distributed execution framework. Our detailed experimental evaluation, integrating PostgreSQL, MemSQL and SparkSQL under MuSQLE, demonstrates its ability to accurately decide on the most suitable execution engine. MuSQLE can provide speedups of up to 1 order of magnitude for TPCH queries, leveraging different engines for the execution of individual query parts.

<div class="section abstract"><div class="htmlview paragraph">As electric trucks become more central to modern logistics, the need for smarter, more adaptive route planning is growing rapidly. This paper presents a key navigation feature for analyzing and recalibrating such optimized routes in real time. Integrating map features into the navigation mode improves user experience by offering real-time navigation and dynamic route adjustments based on traffic updates, road closures, vehicle coordinates and deviation in expected energy consumption. This study compares the performance of Server sent events (SSE), web sockets, and Application programming interface (API) polling methodologies, focusing on metrics such as data transmission efficiency, latency, resource utilization, scalability, and reliability. Our results demonstrate the advantages and limitations of each method, providing insights into their suitability for real-time route optimization in electric truck logistics. The results highlight the potential of SSE in achieving efficient and timely data updates, contributing to more effective route planning and resource management. Additionally, we discuss how API Polling, Web sockets, and SSE each make sense in different scenarios when creating a navigation system (drive mode), considering factors such as the frequency of updates, network conditions, and system architecture. This research underscores the importance of choosing the right communication protocol and integrating advanced map features to enhance the performance and reliability of logistics systems.</div></div>

An efficient cluster head selection for wireless sensor network-based smart agriculture systems

In Real-Time DataBase Management Systems (RTDBMSs), both the integrity constraints of the database and the temporal constraints of the transactions must be respected. In this paper, we focus on the real-time concurrency control protocol SCC which combines the advantages of both pessimistic and optimistic concurrency control methods. This protocol distinguishes two kinds of transaction conflicts: the write-write conflicts resolved by the TWR (Thomas Write Rule [1]) method and the read-write and write-read conflicts resolved by the transaction duplication. In this paper, we propose to extend this protocol in two ways: (1) the TWR method is, to our mind, not applicable in real time context and (2) we propose a new deadline-driven method combined with the transaction duplication. Thus, more transactions will meet their deadlines.

Snapshot Isolation (SI) is a concurrency control mechanism that has been implemented by several commercial and open resources platforms. However, under SI, a set of program may experience a non-serializable execution, in which database integrity constraints can be violated. An elegant approach from Fekete (2005) shows how to guarantee serializable execution on platforms that offer both SI and traditional two-phase locking (2PL) concurrency control, by running some transactions (pivots) with 2PL and the rest at SI. While Fekete's Pivot 2PL technique performs better than running all transactions at 2PL, it often loses much performance compared to SI for all transactions. In this paper we identify causes that harm performance of Pivot 2PL, and we propose an improved approach, called Pivot Ordered2PL, in which a few transactions are rewritten (without changing their functionality). We evaluate Pivot Ordered2PL and find it ensures serializable execution with performance close to that of SI.

An extended relational document retrieval model

Log-structured merge-trees (LSM-trees) are widely used in key-value stores because of its excellent write performance. To reduce LSM-tree's read amplification due to overlapping sorted runs, each file (i.e., SSTable) in an LSM-tree is typically associated with a point or range filter to reduce unnecessary I/Os to the runs that do not contain the target key (range). However, as modern SSDs get faster, probing multiple in-memory filters per query often makes the system CPU bottlenecked, thus compromising the system's throughput. In this paper, we developed the Global Range Filter (GRF) for RocksDB that reduces the number of filter probes per query to one. We follow the pioneering Chucky's approach by storing the sorted run IDs within the filter. However, we identify two practical challenges in building a global range filter: correctness in multi-version concurrency control and efficiency in frequent updates. We solve both challenges by the novel Shape Encoding algorithm. With further optimizations, GRF achieves a dominating performance over the state-of-the-art filters under different workloads when integrated into RocksDB.

With ever-increasing volume and heterogeneity of data, advent of new specialized compute engines, and demand for complex use cases, large-scale data systems require a performant catalog system that can satisfy diverse needs. We argue that existing solutions, including recent lakehouse storage formats, have fundamental limitations and that there is a strong motivation for a specialized database engine, dedicated to serve as the catalog. We present the design and implementation of TreeCat , a database engine that features a hierarchical data model with a path-based query language, a storage format optimized for efficient range queries and versioning, and a correlated scan operation that enables fast query execution. A key performance challenge is supporting concurrent read and write operations from many different clients while providing strict consistency guarantees. To this end, we present a novel MVOCC (multi-versioned optimistic concurrency control) protocol that guarantees serializable isolation. We conduct a comprehensive experimental evaluation comparing our concurrency control scheme with prior techniques, and evaluating our overall system against Hive Metastore, Delta Lake, and Iceberg.

While optimizing database performance, parallel query technology can significantly improve the throughput and resource utilization of the database server. Interactions among different queries which run concurrently can have a great impact on system performance. However, conventional optimizing strategies of parallel query often only focus on individual queries rather than their interactions. In order to capture the impact of parallel query interactions, we propose an interaction-aware scheduling algorithm of parallel queries. Compared with other traditional algorithms, the experimental results demonstrate that our optimizing approach can effectively improve the database performance by avoiding the negative interactions. Finally, some future trends in this area are prospected and discussed.

Many real-time database systems are contained in environments that exhibit restricted access to information, such as government agencies, hospitals and military institutions, where mandatory access control for security is required. In addition to such security constraints, real-time database systems have real-time integrity constraints. These real-time constraints require deadlines to be met and data to be temporally consistent. Conventional multi-level secure database models are inadequate for time-critical applications and conventional real-time database models do not support security constraints. We propose a new concurrency control algorithm for secure real-time databases. We implement the algorithm and study the performance using a real-time database system simulation model. Results show that the algorithm performs fairly well in terms of security and timeliness compared to the non-secure algorithm. We argue and show that achieving more security does not necessarily mean a great deal of sacrifice in maintaining real-time constraints.

As mobile devices get ubiquitous and grow in computational power, their management of interdependent data also becomes increasingly important. The mobile environment exhibits all the characteristics of a distributed database plus the feature of whimsical connectivity. Consequently, transactions respecting data consistency can suffer unbounded and unpredictable delays at both mobile and stationary nodes. The currently popular multi-tier model, in which mobile devices are in one end and always-connected stationary servers in the other, has certain practical advantages. However, it assumes that all integrity constraints are evaluated at the servers and hence relies on the semantics of operations for any autonomy enhancement of the mobile devices. In this paper, we examine the idea of constraint localization in cases where two mobile nodes each own data that share a constraint. It relies on reformulation of a constraint into more flexible local constraints that give more autonomy to the mobile nodes. The scheme also involves dynamic changes of these local constraints through negotiation, which we call re-localization. To overcome the problem of simultaneous requests for such re-localization, we give algorithms along with experimental results indicating their effectiveness.