Looking into a REST-Based Universal API for Database-as-a-Service Systems

This chapter provides an overview of big data storage technologies. It is the result of a survey of the current state of the art in data storage technologies in order to create a cross-sectorial technology roadmap. This chapter provides a concise overview of big data storage systems that are capable of dealing with high velocity, high volumes, and high varieties of data. It describes distributed file systems, NoSQL databases, graph databases, and NewSQL databases. The chapter investigates the challenge of storing data in a secure and privacy-preserving way. The social and economic impact of big data storage technologies is described, open research challenges highlighted, and three selected case studies are provided from the health, finance, and energy sector. Some of the key insights on big data storage are (1) in-memory databases and columnar databases typically outperform traditional relational database systems, (2) the major technical barrier to widespread up-take of big data storage solutions are missing standards, and (3) there is a need to address open research challenges related to the scalability and performance of graph databases.

The amount of sources and sheer volumes of spatiotemporal data have met an unprecedented growth during the last decade. As a consequence, a rapidly increasing number of applications are seeking to generate value by crunching those data. The development of a system that will tap into the potential value of the spatiotemporal big data analysis for a multitude of applications remains one of the biggest challenges in computer engineering. This paper delves into the key-characteristics of the most prominent suchlike systems. In particular, it provides a thorough analysis of NoSQL datastores as well as a traditional relational database system in terms of their geospatial querying capabilities.

최근 급격히 증가하는 공간 데이터를 효율적으로 처리하기 위해 많은 연구들이 진행되고 있다. 기존 관계형 데이터베이스 시스템을 확장한 공간 데이터베이스 시스템은 확장성에 대한 문제가 있으며, 분산 처리 플랫폼인 하둡을 확장한 SpatialHadoop은 중간 연산 결과를 디스크에 작성하기 때문에 파일 입출력의 오버헤드로 성능이 저하되는 문제가 있다. 본 논문은 인-메모리 기반 분산 처리 프레임워크인 스파크를 확장한 공간 연산 스파크를 제안하였다. 또한 공간 연산 스파크의 성능을 향상시키기 위하여 GPGPU를 결합한 모델을 개발하였다. 공간 연산 스파크는 중간 연산 결과를 메모리에 유지시키는 스파크의 특징을 그대로 사용하고 있으며, GPGPU 기반 공간 연산 스파크의 경우 다수의 PE를 이용하여 병렬처리하기 때문에 효율적으로 공간 연산을 수행할 수 있다. 본 논문은 단일 AMD 시스템에서 공간 연산 스파크와 GPGPU 기반 공간 연산 스파크를 구현하였다. 공간 연산 스파크와 GPGPU 기반 공간 연산 스파크의 성능을 평가하기 위하여 Point-in-Polygon 연산과 Spatial Join 연산을 수행하였으며, SpatialHadoop에 비하여 최대 8배의 성능 향상을 확인하였다. Recently, as the amount of spatial information increases, an interest in the study of spatial information processing has been increased. Spatial database systems extended from the traditional relational database systems are difficult to handle large data sets because of the scalability. SpatialHadoop extended from Hadoop system has a low performance, because spatial computations in SpationHadoop require a lot of write operations of intermediate results to the disk, resulting in the performance degradation. In this paper, Spatial Computation Spark(SC-Spark) is proposed, which is an in-memory based distributed processing framework. SC-Spark is extended from Spark in order to efficiently perform the spatial operation for large-scale data. In addition, SC-Spark based on the GPGPU is developed to improve the performance of the SC-Spark. SC-Spark uses the advantage of the Spark holding intermediate results in the memory. And GPGPU-based SC-Spark can perform spatial operations in parallel using a plurality of processing elements of an GPU. To verify the proposed work, experiments on a single AMD system were performed using SC-Spark and GPGPU-based SC-Spark for Point-in-Polygon and spatial join operation. The experimental results showed that the performance of SC-Spark and GPGPU-based SC-Spark were up-to 8 times faster than SpatialHadoop.

Path constraints are capable of expressing inclusion and inverse relationships and have proved useful in modeling and querying semistructured data [Abiteboul and Vianu 1999; Buneman et al. 2000]. Types also constrain the structure of data and are commonly found in traditional databases. There has also been work on imposing structure or a type system on semistructured data for storing and querying semistructured data in a traditional database system [Alon et al. 2001; Deutsch et al. 1999a; Florescu and Kossmann 1999; Shanmugasundaram et al. 1999]. One wants to know whether complexity results for reasoning about path constraints established in the untyped (semistructured) context could carry over to traditional databases, and vice versa. It is therefore appropriate to understand the interaction between types and path constraints. In addition, XML [Bray et al. 1998], which may involve both an optional schema (e.g., DTDs or XML Schema [Thompson et al. 2001]) and integrity constraints, highlights the importance of the study of the interaction.This article investigates that interaction. In particular it studies constraint implication problems, which are important both in understanding the semantics of type/constraint systems and in query optimization. It shows that path constraints interact with types in a highly intricate way. For that purpose a number of results on path constraint implication are established in the presence and absence of type systems. These results demonstrate that adding a type system may in some cases simplify reasoning about path constraints and in other cases make it harder. For example, it is shown that there is a path constraint implication problem that is decidable in PTIME in the untyped context, but that becomes undecidable when a type system is added. On the other hand, there is an implication problem that is undecidable in the untyped context, but becomes not only decidable in cubic time but also finitely axiomatizable when a type system is imposed.

Recently, several researchers realized that the modelling capabilities of traditional relational data base systems are rather limited for advanced applications. Among others, two main research areas can be identified: One area of research is concerned with removing the limitation of flat tuples by supporting structured objects instead. Another area of research deals with the limitation of traditional query languages. Already in the late seventies it was proven that so called relationally complete query languages do not allow to express the computation of the transitive closure of a relation. Most researchers solve the problem by introducing some kind of recursion. The work presented in this paper is an attempt to combine both areas of research. It proposes the integration of recursive queries and nested relations by providing a recursion mechanism which can be nested according to the structure of a non first normal form relation. It is shown by examples that this method can be used to formulate queries which are rather cumbersome if only flat relations are used. Especially, in some cases the problem of performing a selection on a recursively defined relation disappears. This question has been the topic of several research papers recently. Moreover, many queries can be formulated in a more natural way than in the world of flat tuples. Therefore, nested recursive queries in combination with non first normal form relations provide a contribution to the problem of modelling complex data.

Relational database technology has enabled easy and efficient management of data in a variety of applications. This technology, however, has been used for computer simulations to a limited extent. In this context, users may need to analyze the results of multiple simulation runs, which typically capture interactions between various entities in the simulated world. To these users, storing such simulation data only in the form of relational tables may seem counter-intuitive or challenging, particularly if they are not familiar with database normalization theory. Furthermore, some complex queries, including those that examine the cumulative effect of an action (e.g., disease propagation after an initial outbreak), can neither be easily expressed nor efficiently executed on traditional database systems. We propose a new database approach that aims to achieve convenient and highly efficient storage and querying of simulation data by adopting a graph data model. We also discuss new challenges that arise in this research, with a focus on language design, coordination of simulations, data storage, and query processing.

Application programs on application servers issue requests to object servers in traditional database systems. On the other hand, programs named agents move around object servers whose objects are manipulated objects in a mobile agent approach. If an agent conflicts with other agents on an object server, the agents negotiate with each other to resolve the confliction. An agent moves to another servers. In traditional systems, application programs do not work if application servers are faulty. If the server is faulty, the agent find another server where the agent can be performed. In addition, agents are replicated if agents may be faulty. In the mobile agent approach, applications can be operational even if a computer where an agent exist is faulty. In this paper, we discuss how to perform agents with different constraints on multiple object servers in the presence of servers and agent faults.

An estimate of the number of items in a database that satisfy an equality or range predicate is useful for several tasks, such as cost-based query optimization, provisioning of system resources, and determining the financial costs of using database services. In a traditional database system, such estimates are computed and used internally by the system and have been well studied. In contrast, such estimates have not received much attention in the context of a cloud-based database service, where they must be computed by the application that uses the service using only the limited features of the interface provided by the service. This paper motivates and formulates the selectivity-estimation problem for database services. It describes the characteristics of this problem that distinguish it from the analogous problem in traditional database systems. It outlines some subproblems and methods to address them. It provides a method for estimating selectivities based on random sampling along with some experimental results.

The need for centralized processing and storage of data is growing every year. Data centers of still greater capacity and computing power are put into operation for this purpose. To reduce their expenses the owners of data centers pay special attention to optimizing the energy consumed by the data center infrastructure. An approach for improving the data center’s overall energy efficiency through redistributing its load with subsequently shutting down the idle servers is considered. A mechanism governing the migration of virtual machines for optimizing power consumption by the servers is outlined. The existing methods used to redistribute virtual machines either perform a full search which is impossible in the case of real dimensions or disregard the power consumption criterion. A procedure is proposed that excludes the need to perform a full search and places focus on improving the efficiency according to the criterion of total energy consumption by the data center servers due to redistribution of the operating virtual machines among the data center servers. The proposed method is based on a heuristic procedure of selecting the destination servers and consists of a static part, which collects information about the available resources, and a dynamic part in which the virtual machines are distributed among the servers depending on the initial data and the preset limiting parameters. A model example demonstrating the operability of the method is given which outlines the steps in applying the method and shows how the destination servers are selected for migrations.

The emergence and development of big data tools, techniques and systems motivate industries and organizations to embrace and explore research in big data. This is to circumvent the challenges of the traditional database systems. However, the available benchmarks and workloads are for some specific aspect of Information Technology industries, which has differences in nature and complexities with the data obtained from other sources. Hence there is need for using data from other domains in order to evaluate the performance and maturity of the big data technologies. In this paper the performance profiling of Meteorological and Oceanographic data on Hive is conducted. Hive being the commonly used data warehouse analytical platform for big data is chosen with the view to exposing the intricacies that are involved in the formating and loading of the data. The response time for indexed and non-indexed retrievals using three set of queries frequently used in the area is found. The query types are Type 1 SELECT with WHERE clause, Type 2 SELECT with JOIN clause. And Type 3 SELECT with GROUP BY clause. The experimental results show that a good response time for both indexed and Non-indexed tables are achieved. The indexed retrieval shows a significant decrease in the response time for Type 1 query for all data sizes and for Type 3 query for 100GB data size and less . It also shows additional overhead for Type 2 query for all data sizes and Type 3 query for 500GB and more data sizes. The Meteorological and Oceanographic data if properly formated it's analytics with Hive proved to be efficient compared to the traditional database systems. The results of this study has the potentials of attracting the oil and gas companies to adopt big data technologies for the handling of their exploration dataset.

Content-based retrieval on a large multimedia database attracts the interest of many researchers. However, the database architecture needed for content-based retrieval is still a problem. Traditional relation database systems do not support high-dimension feature form content description and indexing, and is thus limited in its content-based retrieval function. Some systems do support high-dimension feature form content description and indexing, but lack descriptions and query expressions on media object content and relations. In this paper, we present our study results on query mechanism and proposed CbExpr - a powerful, flexible query expression mechanism on a media object. Based on CbExpr, we proposed GMA (general mediabase architecture) - a general architecture for management and content-based retrieval on large media databases. We also present VideoBase, a content-based video retrieval system, present as example of GMA. Basic thoughts, considerations, and definitions are presented in this paper, along with some details for implementation.

At present, countries all over the world attach great importance to cultural works. These works have become an important engine of economic development and can make good contributions to economic growth. The traditional tracing control scheme of cultural creation works has some problems, such as incomplete information collection, critical point of unit tracing, information fraud, centralized data storage and so on. At the same time, there seems to be a series of problems that can be solved. This paper analyzes the current situation of the review data of cultural creation products, and puts forward the review and analysis scheme of cultural creation works based on large-scale data algorithm and block chain technology. In addition, by combining department chain technology with intranet and traditional database system, an information database about the supply chain of cultural products in crop departments is established. The processing, logistics and sales information and the information of participants are interrelated.

This article explores the evolution and capabilities of NoSQL and NewSQL databases as solutions that address the limitations of traditional relational database systems in today's data-intensive landscape. It examines the architectural foundations of various NoSQL categories—key-value stores, document databases, column-family stores, and graph databases—highlighting their distinct strengths for different use cases. The emergence of NewSQL databases is presented as a hybrid approach that maintains SQL's transactional integrity while incorporating distributed architectures for enhanced scalability. Through detailed analysis of data modeling strategies, performance considerations, and real-world implementations, the article guides organizations navigating the complex database ecosystem. This article encompasses migration paths from relational systems, polyglot persistence strategies, and emerging trends that are shaping the future of data management technologies in cloud-native and edge computing environments.

Traditional disk-oriented storage systems can no longer meet the high requirements of applications in latency and bandwidth, for which RAMCloud, was proposed to replace the disk with RAM. We consider using the memory of end-users in existing Internet to achieve a similar memory storage system. However the end-users' behavior are not as controllable as the servers in data center, which would lead to a high data loss rate. So how to ensure the data's reliability is the biggest challenge. To improve the reliability, as well as the resource utilization, we present a thought of Contributor scoring, establishing a scoring model according to Contributors' behavior. This model will classify Contributors into certain levels. Moreover, we train the Model with genetic algorithms. Experimental results show that the optimized model can effectively divide the Contributor, ensure certain reliability.It has great significance to this user memory storage system for improving the reliability and resource utilization.

‘A visual idea is more powerful than verbal idea’, ‘A picture is worth more than ten thousand words’, ‘No words can convey what a picture speaks’, ‘A picture has to be seen and searched as a picture only’ are few of the well-known sayings that imply the certainty for the widespread availability of images. Common sense evidence suggests that images are required for a variety of reasons, like, illustration of text articles, conveying information or emotions that are difficult to describe in words, display of detailed data for analysis (medical images), formal recording of design data for later use (architectural plans) etc. The advent of digital photography combined with decreasing storage and processing cost, allows more and more people to have their personal collection of photographs and other visual content available on the internet. Organising these digital images into a small number of categories and providing effective indexing is imperative for accessing, browsing and retrieving useful data in “real time”. The process of digitization does not in itself make image collections easier to manage. Some form of indexing (cataloguing) is still necessary. People’s interest to have their own digital libraries has burgeoned and hence requires a data structure to preserve the images for a long time and also provide easy access to the desired images. These requirements have indeed forced the design of specialized imaging systems/ image databases, such that an access to any image is effective and efficient. An efficient image archival and retrieval system is characterized by its ability to retrieve relevant images based on their visual and semantic contents rather than using simple attributes or keywords assigned to them. Thus, it is necessary to support queries based on image semantics rather than mere-pixel-to-pixel matching. An image archival and retrieval system should therefore allow adequate abstraction mechanisms for capturing higher level semantics of images in order to support content addressability as far as possible. That is, for two images to be similar, not only the shape, color and texture properties of individual image regions must be similar, but also they must have the same arrangement (i.e., spatial relationships) in both the images. In fact, this is the strategy, which is generally being employed by our vision system most of the times. An effective method of representing images depends on the perception of knowledge embedded in images in terms of objects/components (generally known as elements) present in them along with their topological relationships. The perception of topological relationships, especially spatial relationships existing among the significant elements of an image, helps in making the image database system more intelligent, fast and flexible. An obvious method to search an image database is sequential scanning. The query is matched with all stored images (i.e., the representation of the query is matched with all representations stored in the image database) one by one. Retrievals may become extremely slow, especially when database search involves time consuming image matching operations. To deal with slow retrieval response times, and high complexity matching, an image database must utilize indexing methods that are faster than sequential scanning methods. In traditional image database systems, the use of indexing to allow database accessing has been well established. Analogously, image indexing techniques have been studied during the last decade to support representation of pictorial information in an image database and also to retrieve information from an image database. The use of significant elements present in images along with their topological relationships as indexes is the basic issue of the indexing methodologies developed to this aim.