The LDBC Financial Benchmark: Transaction Workload

In developing NoSQL databases, a major motivation is to achieve better efficient query performance compared with relational databases. The graph database is a NoSQL paradigm where navigation is based on links instead of joining tables. Links can be implemented as pointers, and following a pointer is a constant time operation, whereas joining tables is more complicated and slower, even in the presence of foreign keys. Therefore, link-based navigation has been seen as a more efficient query approach than using join operations on tables. Existing studies strongly support this assumption. However, query complexity has received less attention. For example, in enterprise information systems, queries are usually complex so data need to be collected from several tables or by traversing paths of graph nodes of different types. In the present study, we compared the query performance of a graph-based database system (Neo4j) and relational database systems (MySQL and MariaDB). The effect of different efficiency issues (e.g., indexing and optimization) were included in the comparison in order to investigate the most efficient solutions for different query types. The outcome is that although Neo4j is more efficient for simple queries, MariaDB is essentially more efficient when the complexity of queries increases. The study also highlighted how dramatically the efficiency of relational database has grown during the last decade.

Graph embedding in vector spaces by node attribute statistics

Graph mining is the process of extracting and analyzing patterns from graph data. Graphs are a data structure that consists of a set of nodes and a set of edges that connect these nodes. Graphs are often used to represent real-world entities and the relationships between them. In a graph database, the importance of a pattern (also known as support) must be quantified using a counting function called a support measure. This function must adhere to several constraints, such as antimonotonicity that forbids a pattern to have support bigger than its sub-patterns. These constraints make the tasks of defining and computing support measures highly non-trivial and computationally expensive. In this paper, I use the previously discovered relationship between support measures in graph databases and flows in networks of subgraph appearances to simplify the process of computing support measures. I show that the network of pattern instances may be successfully pruned to contain just particular kinds of patterns and prove that any legitimate computing support measures in graph databases can adopt this strategy. When the suggested method is utilized, experimental evaluation demonstrates that network size reduction is significant.

Several scientific domains operate on mesh data structures. Computations on meshes share a set of basic abstractions which relates to topological and geometric structure of meshes. Due to the complexity of modeling and querying meshes, the database community has neglected the need for a dedicated mesh database and in particular a high-level mesh query language. Currently, such computations is done by using general-purpose programming languages. Such implementations are tightly coupled with dataset characteristics and thus are less reusable, their maintenance cost is high, and their performance is limited to the memory size. In this paper, we describe syntax and semantic of a high-level query language for scientific meshes (called MQuery) with a level of abstraction that can express queries based on topological and geometric navigations. The language is build on top of a graph-based conceptual mesh data model. We report our results comparing two implementations of the language on top of relational and graph databases with a mesh library.

With the advent of the Internet and Internet-connected devices, modern business applications can experience rapid increases as well as variability in transactional workloads. Database replication has been employed to scale performance and improve availability of relational databases but past approaches have suffered from various issues including limited scalability, performance versus consistency tradeoffs, and requirements for database or application modifications. This paper presents Hihooi, a replication-based middleware system that is able to achieve workload scalability, strong consistency guarantees, and elasticity for existing transactional databases at a low cost. A novel replication algorithm enables Hihooi to propagate database modifications asynchronously to all replicas at high speeds, while ensuring that all replicas are consistent. At the same time, a fine-grained routing algorithm is used to load balance incoming transactions to available replicas in a consistent way. Our thorough experimental evaluation with several well-established benchmarks shows how Hihooi is able to achieve almost linear workload scalability for transactional databases.

Telecom software systems are a major aspect of telecommunications networks used for managing, administrating, auditing and monitoring purposes. One of the most significant features is the performance of the telecom software system, since there is a great variety of network elements in use within built systems through which the telecom software interacts. Workload characterization is an essential methodology for simulating load usage for a given infrastructure that is exposed to a production environment. Hence, exploring the characteristics of workloads well leads an increase on system efficiency and performance. Nevertheless, workload characterization has not been well examined yet for Telecom Software System. In this paper, a new workload characterization architecture is proposed and implemented to model the workload and simulate the behavior of the network element. This architecture provides information regarding the accuracy of the workload models for more precise performance testing. Additionally, we verify our workload characterization results obtained from our architecture in a real software system. Performance results show that proposed workload model-based simulation environment is promising to predict the performance of the software systems.

A methodology (JAGATH) and a tool based on it for establishing the performance of a distributed system within an empirical framework are presented. A set of performance variables representing the system performance is derived in terms of a combination of a set of performance measures. These performance measures represent the actual measurements of the events in the system. These performance variables are used to display the system performance status in the form of Kiviat graphs. The causal relationship between the performance variables and the internal system control variables and the workload characteristics can be established. This can be used in a performance ‘tuning’ system.

The abnormal situation with reversal characteristic in chemical processes: Local monitoring and self-recovery

A new class of stream processing engines has recently established itself as a platform for applications in numerous domains, such as personalized content- and ad-serving, online recommender systems or social media analytics. These new engines draw inspiration from Big Data batch processing frameworks (Google's MapReduce and its descendants) as well existing stream processing engines (e.g. Borealis and STREAM). They process data on-the-fly without first storing it in a file system and their core programming abstractions hide the complexity of distributed-parallel programming.
\nAlthough stream processing applications commonly need to process ingested data within time bounds, this new class of engines so far computes results "as fast as possible". As variations in workload characteristics are often hard to predict and outside the control of the application, this may quickly lead to a situation where "as fast as possible" becomes "not fast enough" for the application.
\nThis thesis revisits the design of this new class of stream processing engines. The core question addressed by this thesis is how latency requirements can be specified and continuously enforced within these engines in a resource-efficient manner. To this end, this thesis contributes (1) a formalism and associated semantics for expressing latency requirements for stream processing applications, (2) a set of techniques for engines to enforce them and (3) an evaluation of the effectiveness of the presented techniques. The proposed techniques optimize resource efficiency by automatically adjusting the data shipping strategy between data flow tasks and adapting the mapping between tasks and execution threads at runtime. Furthermore, based on workload statistics measured at runtime, they adapt the application's parallelism by exploiting the elasticity of shared, resource-managed compute clusters. To evaluate their effectiveness, they have been implemented in a research prototype and an experimental evaluation with several application workloads has been conducted on a large commodity cluster.

A parallel team formation approach using crowd intelligence from social network

The power communication network can be abstracted as a graph based on its topology. In this paper, we propose an approach to conduct simulations of power communication network based on its graph representation. In particular, the nodes and edges in the graph refer to the ports and channels in the grid topology. Different applications on the grid can be transformed into queries over the graph. Hence, in this paper, we build our grid simulation model based on the Neo4j graph database. We also propose a fault extension algorithm based on predicate calculus. Our experiment evaluations show that the proposed approach can effectively improve the efficiency of the power grid.

The subgraph searching is a fundamental operation for the analysis and exploration of graphs. Nowadays, molecular databases are nearing close to one hundred million molecules. Since finding all the data graphs in a graph database that contain the query graph using subgraph isomorphism is an NP-complete problem, indexes are built and processed. Further, to assist the formulation of the query by a user, the visual exploratory subgraph query paradigm proposes a graphical user interface and leverages exploration time to reduce query processing time. However, state-of-the-art approaches need to scale better to dynamic graph databases and suffer from efficiency problems. In addition, the existing Summarisation-based frequent subgraph mining for visual exploratory subgraph searching (SuMExplorer) is lacking implementation and evaluation study for handling visual subgraph similarity search and modify operations. In this paper, we present a novel index structure, which aids the subgraph searching using the summarised-based weighted frequent subgraph mining on data graphs. By the structure-preserving, we exploit the indexes to support similarity and modify operations. We conduct extensive performance studies on both real-world and synthetic datasets to evaluate the overall performance of the extended SuMExplorer to the recent visual exploratory FERRARI and traditional subgraph search algorithms (such as the gIndex and the GRAPES-DD). Our results showed that our indexes can query up to 3 times faster in comparison to the FERRARI while reducing the storage footprint by 2 orders of magnitude.

In this paper, we utilize a bandwidth-centric job communication model that captures the interaction and impact of simultaneously co-allocating jobs across multiple clusters. We make use of a parallel job model that seeks to capture both local and global communication access patterns. By doing so, we are able to explore scheduling strategies that attempt to improve average job turnaround time by selectively mapping jobs across cluster boundaries in a process known as job co-allocation. In this research, we focus on scheduling strategies that make use of available information such as network link utilization, per-processor bandwidths, and job communication topology in order to make intelligent decisions regarding application partition sizes and job placement. We provide results that help to establish the relationship between the quantity of information available a priori to the scheduler and its ability to improve overall system performance. Additionally, we demonstrate the dramatic impact that salient workload characteristics can have on the effectiveness of co-allocation

Recent studies have shown that moderate to high data redundancy exists in primary storage systems, such as VM-based, enterprise and HPC storage systems, which indicates that the data deduplication technology can be used to effectively reduce the write traffic and storage space in such environments. However, our experimental studies reveal that applying data deduplication to primary storage systems will cause space contention in main memory and data fragmentation on disks. This is in part because applying data deduplication introduces significant index memory overhead to the existing system and in part because a file or block is split into multiple small data chunks that are often located in non-sequential locations on disks after deduplication. This fragmentation of data can cause a subsequent read operation to invoke many disk I/O requests, thus leading to performance degradation. The existing primary data deduplication schemes, such as iDedup[1], are to leverage spatial locality in that they only select the large requests to deduplicate and exclude the small requests (e.g., 4KB, 8KB or less) because the latter only account for a tiny fraction of the storage capacity requirement[2]. Moreover, these schemes tend to overlook the importance of cache management, leading them to manage the index cache and the read cache separately. However, previous workload studies on primary storage systems have revealed that small I/O requests dominate in the primary storage systems (more than 50%) and are at the root of the system performance bottleneck. Furthermore, the accesses in primary storage systems exhibit obvious I/O burstiness. The existing primary-storage data deduplication schemes fail to consider these workload characteristics in primary storage systems from the performance's perspective. We argue that, primary-storage data deduplication schemes should take the workload characteristics of primary storage into the design considerations. To address the two problems and take the primary-storage workload characteristics into considerations, we propose a Performance-Oriented I/O Deduplication approach, POD, to improving the I/O performance of primary storage systems in the Cloud. POD takes a two-pronged deduplication approach to improve primary storage systems, a request-based I/O and data deduplication scheme, called Select-Dedupe, aimed at alleviating data fragmentation and an adaptive memory management scheme, called iCache, to ease the main memory contention. More specifically, the former takes the workload characteristics of small-I/O-request domination into the design considerations. It deduplicates all the write requests if their write data is already stored sequentially on disks, including the small write requests that would otherwise be excluded from by the capacity-oriented deduplication schemes. For other write requests, Select-Dedupe does not deduplicate their redundant write data to maintain the performance of the subsequent read requests to these data. iCache takes the I/O burstiness characteristics into the design considerations. It dynamically adjusts the cache space between the index cache and the read cache according to the workload characteristics, and swaps these data between memory and backend storage devices accordingly. During the write-intensive bursty periods, iCache enlarges the index cache size and shrinks the read cache size to detect much more redundant write requests, thus improving the write performance. The read cache size is enlarged to cache more hot read data to improve the read performance during the read-intensive bursty periods. The prototype of the POD scheme is implemented as an embedded module at the block-device level with the fixed-size chunking method. Preliminary evaluations driven by the real traces conducted on our lightweight POD prototype implementation show that POD significantly outperforms iDedup in improving the performance of primary storage systems in the Cloud.

Analyzing and understanding the characteristics of the incoming workload is crucial in unraveling trends and tuning the performance of a database system. In this work, we present Oracle Workload Intelligence (WI), a tool for workload modeling and mining, as our attempt to infer the processes that generate a given workload. WI consists of two main functionalities. First, WI derives a model that captures the main characteristics of the workload without overfitting, which makes it likely to generalize well to unseen instances of the workload. Such a model provides insights into the most frequent code paths in the application that drives the workload, and also enables optimizations inside the database system that target sequences of query statements. Second, WI can compare the models of different snapshots of the workload to detect whether the workload has changed. Such changes might indicate new trends, regressions, problems, or even security issues. We demonstrate the effectiveness of WI with an experimental study on synthetic workloads and customer-provided application benchmarks.