NoSQL Database Technologies

Geospatial information has been indispensable for many application fields, including traffic planning, urban planning, and energy management. Geospatial data are mainly stored in relational databases that have been developed over several decades, and most geographic information applications are desktop applications. With the arrival of big data, geospatial information applications are also being modified into, e.g., mobile platforms and Geospatial Web Services, which require changeable data schemas, faster query response times, and more flexible scalability than traditional spatial relational databases currently have. To respond to these new requirements, NoSQL (Not only SQL) databases are now being adopted for geospatial data storage, management, and queries. This paper reviews state-of-the-art geospatial data processing in the 10 most popular NoSQL databases. We summarize the supported geometry objects, main geometry functions, spatial indexes, query languages, and data formats of these 10 NoSQL databases. Moreover, the pros and cons of these NoSQL databases are analyzed in terms of geospatial data processing. A literature review and analysis showed that current document databases may be more suitable for massive geospatial data processing than are other NoSQL databases due to their comprehensive support for geometry objects and data formats and their performance, geospatial functions, index methods, and academic development. However, depending on the application scenarios, graph databases, key-value, and wide column databases have their own advantages.

Big data is a collection of large scale of structured, semi-structured and unstructured data. It is generated due to Social networks, Business organizations, interaction and views of social connected users. It is used for important decision making in business and research organizations. Storage which is efficient to process this large scale of data to extract important information in less response time is the need of current competitive time. Relational databases which have ruled the storage technology for such a long time seems not suitable for mixed types of data. Data can not be represented just in the form of rows and columns in tables. NoSQL (Not only SQL) is complementary to SQL technology which can provide various formats for storage that can be easily compatible with high velocity,large volume and different variety of data. NoSQL databases are categorized in four techniques- Column oriented, Key Value based, Graph based and Document oriented databases. There are approximately 120 real solutions existing for these categories; most commonly used solutions are elaborated in Introduction section. Several research works have been carried out to analyze these NoSQL technology solutions. These studies have not mentioned the situations in which a particular data storage technique is to be chosen. In this study and analysis, we have tried our best to provide answer on technology selection based on specific requirement to the reader. In previous research, comparisons amongNoSQL data storage techniques have been described by using real examples like MongoDB, Neo4J etc. Our observation is that if users have adequate knowledge of NoSQL categories and their comparison, then it is easy for them to choose best suitable category and then real solutions can be selected from this category.

Benchmarking data warehouses is a means to evaluate the performance of systems and the impacts of different technical choices. Developed on relational models which have been for a few years the most used to support classical data warehousing applications such as Star Schema Benchmark (SSB). SSB is designed to measure performance of database products when executing star schema queries. As the volume of data keeps growing, the types of data generated by applications become richer than before. As a result, traditional relational databases are challenged to manage big data. Many IT companies attempt to manage big data challenges using a NoSQL (Not only SQL) database, and may use a distributed computing system. NoSQL databases are known to be non-relational, horizontally scalable, distributed. We present in this paper a new benchmark for columnar NoSQL data warehouse, namely CNSSB (Columnar NoSQL Star Schema Benchmark). CNSSB is derived from SSB and allows generating synthetic data and queries set to evaluate column-oriented NoSQL data warehouse. We have implemented CNSSB under HBase column-oriented database management system (DBMS), and apply its charge of queries to evaluate performance between two SQL skins, Phoenix and HQL (Hive Query Language). That allowed us to observe a better performance of Phoenix compared to HQL.KeywordsData warehousescolumnar databasesdecisional benchmark

Big data is emerging as one of the most important crucial issues in the modern world. Most studies mention that a relational database cannot handle big data. This challenge has led to the presentation of the not only structured query language (NoSQL) database as a new concept of database technology. NoSQL supports large volumes of data by providing a mechanism for data storage, retrieval and more flexible than a relational database. One of the most powerful types of NoSQL database is the document-oriented database. Recently, many software developers are willing to migrate from using relational databases to NoSQL database because of scalability, availability, and performance. The document-oriented database has challenged as to how to obtain an appropriate schema for the document-oriented database. The existing approach to migrate a relational database to a document-oriented database does not consider all the properties of the former, especially on how to handle various types of relationships. This research proposed a flexible schema for a document-oriented database (SDOD). This study evaluated the development of agility based on the schema of a document-oriented database and query execution time. The evaluation verifies the reliability of the proposed schema.

This research shows a protocol to assess the computational complexity of querying relational and non-relational (NoSQL (not only Structured Query Language)) standardized electronic health record (EHR) medical information database systems (DBMS). It uses a set of three doubling-sized databases, i.e. databases storing 5000, 10,000 and 20,000 realistic standardized EHR extracts, in three different database management systems (DBMS): relational MySQL object-relational mapping (ORM), document-based NoSQL MongoDB, and native extensible markup language (XML) NoSQL eXist. The average response times to six complexity-increasing queries were computed, and the results showed a linear behavior in the NoSQL cases. In the NoSQL field, MongoDB presents a much flatter linear slope than eXist. NoSQL systems may also be more appropriate to maintain standardized medical information systems due to the special nature of the updating policies of medical information, which should not affect the consistency and efficiency of the data stored in NoSQL databases. One limitation of this protocol is the lack of direct results of improved relational systems such as archetype relational mapping (ARM) with the same data. However, the interpolation of doubling-size database results to those presented in the literature and other published results suggests that NoSQL systems might be more appropriate in many specific scenarios and problems to be solved. For example, NoSQL may be appropriate for document-based tasks such as EHR extracts used in clinical practice, or edition and visualization, or situations where the aim is not only to query medical information, but also to restore the EHR in exactly its original form.

As one of the largest-technology penetrations, Internet of Things (IoT) has gained a considerable attention across both, research communities and industrial domains. IoT is a network of billions of connected devices that are capable of communicating over the Internet. In IoT, data is generated exponentially by different real-time applications (e.g., social network sites and sensor-based devices). The traditional relational database management technologies are inappropriate to deal with such new generation of data due to its limited processing speed, scalability issues, and limited storage capacity. The advent of Big Data requires new advanced technologies that are capable of handling (storing, retrieving and processing) large amount of unstructured data that IoT applications produce. Not Only SQL (NoSQL) databases emerged as a solution to overcome some limitations of the traditional relational database. Recently, many NoSQL databases have been announced (e.g., Cassandra, MongoDB, HBase, Apache Hadoop, Voldemort) to deal with tremendous amounts of unstructured data. These databases depend mainly on distributed algorithms for their work in the field of IoT. One of the most used distributed algorithms is Chord protocol. However, Chord algorithm is inefficient in some cases because of its way of handling messages transferred between connected devices. All the messages in Chord are passed clockwise along the Chord ring, which leads to send a huge number of redundant messages. Chord works by creating a lookup table for each node connected to the network, which generates huge number of tables that should be updated regularly with the routing information. This paper presents a proposal for implementing an enhanced version of the distributed Chord protocol using the most two popular NoSQL databases, namely: MongoDB and Cassandra. The new version of the Chord algorithm will be evaluated and compared to the current algorithm against their performance using the two databases. Moreover, the two databases will be compared regarding their performance in terms of database operations (reading, writing, updating and deleting) and the number of messages that are sent by the nodes. The home environment controlling IoT application will be simulated using SensibleThings distributed platform for simulating the implementation of the modified protocol and to compare between the performance of the two databases.

In the era of Big Data, social media analysis has grown extremely popular. Twitter, one of the most popular social media, is believed to contain many user opinion in its message. Thus, leading organizations start utilizing its data using social media analytic tools to get in-sight of their markets in real-time. However, many Twitter analytic tools are still specified only in some specific tasks. Therefore, in order to enhance the possibility of doing many analysis on Twitter, a data warehouse technology can be utilized to receive, process, and store a real-time Twitter streams. Nonetheless, data warehouse development using relational database start to show its limit on storing big data let alone real-time. Hence, NoSQL (Not Only SQL) technology has emerged as an alternative solution. In this paper, we try to develop near real-time Twitter data warehouse using NoSQL database, Cassandra, and compare its storing and querying performance with that developed using relational databases. The results show that Cassandra performs significantly better in storing data than the relational databases. Meanwhile, in its querying performance, Cassandra is slower while using small data but way faster on vast data.

This paper proposes a schema proposition model for physical and logical modeling of NoSQL (Not Only SQL) document-store databases. Unlike the traditional databases which are rigid in design, the NoSQL databases are flexible, scalable, and low-latent making them to house big data efficiently. However, issues such as dataset complexity coupled with programmers’ low competency are making database modeling and designing very challenging, especially in balancing the significant parameters like availability, consistency and scalability. The proposed model aims to address this balancing by abstractly proposing schemas to programmers with respect to user defined system requirements, Create, Read, Update and Delete (CRUD) operations, etc. Exploratory and experimental approaches were adopted in this research. System requirement formulas were introduced to compute the expected performance. A mini system was developed to validate the proposed model by comparing the manually generated schema to the schema generated through the proposed model. Results obtained indicated a significant improvement in the schema generated by the proposed model.

In the last few years, databases have become very important and very large because they play a strategic and important role in most organizations and because they receive a huge flow of information from multiple sources every moment in building BigData. This situation identified several limitations and weaknesses in relational database management systems (RDBMS), such as availability, real-time response, horizontal scalability, decision support, advanced data analysis, and especially the management of Bigdata which can reach zeta bytes in storage. This requires the storage and organization of this data in a new management system database not fixed by a rigid structure and resolves all problems associated with the storage of database in a relational system. In this view, organizations need a new NoSQL (not only SQL) system that overcomes the limitations of the relational system. The change of the database management system requires the migration of the databases from the relational system to another NoSQL, taking into consideration all stored data and keeping the majority of the possibilities and functionalities of the old system, with all the advantages of the new system. In this paper, we will identify the elements of relational databases that belong to nature: data, structure, and semantics, which we must migrate to a NoSQL system, such as a document-oriented system. Also in this paper, we will present the rules for transforming the structure of the relational system to another document-oriented NoSQL, according to the principal’s basics of a new approach to migration.KeywordsMigration DataBaseNoSQLDatabaseTransformation rulesBigData

Distributed database systems offer scalability and fault tolerance by replicating databases across geographically dispersed nodes. This redundancy aims to ensure data availability even during failures and allows for backups in case of disasters. However, maintaining strong data consistency, where all nodes reflect the latest data simultaneously, becomes a challenge in such geographically distributed setups since database systems need to prioritize availability or performance over strong data consistency. This study examines how different data consistency configurations affect the performance of popular NoSQL (Not only SQL) databases, namely Cassandra, MongoDB, and Redis, in a multi-region cloud environment. We adopt the Yahoo! Cloud Serving Benchmark (YCSB) tool to simulate various workloads, measure performance metrics, and compare the results. Our findings reveal significant performance degradation associated with strong data consistency configurations. For instance, in Cassandra, the number of writing/reading operations processed per second can decrease by up to 95% for specific workloads. Similarly, enforcing strong data consistency in Redis can result in execution times that are over 20 times slower on writing/reading operations.

The traditional OLAP (On-Line Analytical Processing) systems store data in relational databases. Unfortunately, it is difficult to manage big data volumes with such systems. As an alternative, NoSQL systems (Not-only SQL) provide scalability and flexibility for an OLAP system. We define a set of rules to map star schemas and its optimization structure, a precomputed aggregate lattice, into two logical NoSQL models: column-oriented and document-oriented. Using these rules we analyse and implement two decision support systems, one for each model (using MongoDB and HBase).We compare both systems during the phases of data (generated using the TPC-DS benchmark) loading, lattice generation and querying.

The project entitled as Big Data, Internet of Things, and Mobile Devices, in Portuguese Banco de Dados, Internet das Coisas e Dispositivos Moveis (BDIC-DM) was implemented at the Brazilian Aeronautics Institute of Technology (ITA) on the 1st Semester of 2015. It involved 60 graduate students within just 17 academic weeks. As a starting point for some features of real time Online Transactional Processing (OLTP) system, the Relational Database Management System (RDBMS) MySQL was used along with the NoSQL Cassandra to store transaction data generated from web portal and mobile applications. Considering batch data analysis, the Apache Hadoop Ecosystem was used for Online Analytical Processing (OLAP). The infrastructure based on the Apache Sqoop tool has allowed exporting data from the relational database MySQL to the Hadoop File System (HDFS), while Python scripts were used to export transaction data from the NoSQL database to the HDFS. The main objective of the BDIC-DM project was to implement an e-Commerce prototype system to manage credit card transactions, involving large volumes of data, by using different technologies. The used tools involved generation, storage, and consumption of Big Data. This paper describes the process of integrating NoSQL and relational database with Hadoop Cluster, during an academic project using the Scrum Agile Method. At the end, processing time significantly decreased, by using appropriate tools and available data. For future work, it is suggested the investigation of other tools and datasets.

In the big data environment, there are many new demands with data storage, the traditional data storage mode which based on relational database cannot meet these needs, many application systems tend to use NoSQL to solve the problem of big data storage. However, NoSQL gave up on the relationship between operation support, so that part of existing application system is difficult to use the simple way of transplantation. This thesis with reference to some typical existing big data storage scheme, we presents a big data compatible with relational storage model storage scheme, this scheme can not only meet the big data storage requirements, but also can support the most relational operation, so that the original system based on relational database can easily be ported to new storage schemes in.

The crucial role of competent software architecture is essential in managing the challenging of big data processing for both relational and nonrelational databases. Relational databases are designed to structure data for facilitating vertical scalability, while non-relational databases excel in handling vast volumes of unstructured data for enhancing horizontal scalability. Choosing the right database paradigm is determined by the needs of the organization, yet selecting the best option may often be a challenging task. Large number of applications still use relational databases due to its benefits of reliability, flexibility, robustness, and scalability. However, with the rapid growth in web and mobile applications as well as huge amounts of complex unstructured data generated via online and offline platforms, nonrelational databases are compensating for the inefficiency of relational databases. Since selecting the right nonrelational database method for high performing applications from a plethora of possibilities is a challenging task, existing studies are still at emergent stage to compare the performance of different popular nonrelational databases. This paper introduces a novel benchmarking approach for tailoring the comparative study of nonrelational databases. To illustrate our approach, we compare two leading non-relational databases, Aerospike and MongoDB, focusing on their average transaction times to evaluate the database performance. Our comprehensive analysis reveals the strengths of each database in read and write operations for single record and bulk record batch transactions.

Blockchain technologies have been developing very fast recently. Ethereum, as the second generation of blockchain, can support smart contracts with various functions. Blockchain, combined with IoT technologies like RFID, can bring more security into IoT systems. It has been noted that just storing hash values, addresses and transaction records cannot meet the needs of various blockchain applications. Thus, how to test capacity of reading/writing data for blockchain and further make comparisons with the traditional relational databases needs to be paid attention. Although there has been some research on testing and analyzing the performance of blockchains, few studies focus on analyzing blockchains and relational databases. This test was conducted to analyze the capacities of reading/writing data and processing/recording transactions of blockchains and relational databases. Ethereum and MySQL are chosen as the representatives of blockchain and relational databases. This test has three aims: (1) getting the detailed data of the blockchain’s capacity of processing transactions; (2) identifying the bottleneck or potential bottleneck in blockchain systems; (3) putting forward a testing method and testing indexes which are reasonable, practical and compatible with the current development situations of blockchain systems. With the results data, it was concluded that the maximum data volume in single transaction on blockchain network was about 1/10 of MySQL. As for the time spent in processing single transaction, the blockchain network was 80–2000 times as much as MySQL. Thus, it’s recommended to store little-size data into blockchain. For more detailed testing, these indexes and testing method can be referenced.