GPGPU를 활용한 스파크 기반 공간 연산

The amount of sources and sheer volumes of spatiotemporal data have met an unprecedented growth during the last decade. As a consequence, a rapidly increasing number of applications are seeking to generate value by crunching those data. The development of a system that will tap into the potential value of the spatiotemporal big data analysis for a multitude of applications remains one of the biggest challenges in computer engineering. This paper delves into the key-characteristics of the most prominent suchlike systems. In particular, it provides a thorough analysis of NoSQL datastores as well as a traditional relational database system in terms of their geospatial querying capabilities.

Recently, several researchers realized that the modelling capabilities of traditional relational data base systems are rather limited for advanced applications. Among others, two main research areas can be identified: One area of research is concerned with removing the limitation of flat tuples by supporting structured objects instead. Another area of research deals with the limitation of traditional query languages. Already in the late seventies it was proven that so called relationally complete query languages do not allow to express the computation of the transitive closure of a relation. Most researchers solve the problem by introducing some kind of recursion. The work presented in this paper is an attempt to combine both areas of research. It proposes the integration of recursive queries and nested relations by providing a recursion mechanism which can be nested according to the structure of a non first normal form relation. It is shown by examples that this method can be used to formulate queries which are rather cumbersome if only flat relations are used. Especially, in some cases the problem of performing a selection on a recursively defined relation disappears. This question has been the topic of several research papers recently. Moreover, many queries can be formulated in a more natural way than in the world of flat tuples. Therefore, nested recursive queries in combination with non first normal form relations provide a contribution to the problem of modelling complex data.

Path constraints are capable of expressing inclusion and inverse relationships and have proved useful in modeling and querying semistructured data [Abiteboul and Vianu 1999; Buneman et al. 2000]. Types also constrain the structure of data and are commonly found in traditional databases. There has also been work on imposing structure or a type system on semistructured data for storing and querying semistructured data in a traditional database system [Alon et al. 2001; Deutsch et al. 1999a; Florescu and Kossmann 1999; Shanmugasundaram et al. 1999]. One wants to know whether complexity results for reasoning about path constraints established in the untyped (semistructured) context could carry over to traditional databases, and vice versa. It is therefore appropriate to understand the interaction between types and path constraints. In addition, XML [Bray et al. 1998], which may involve both an optional schema (e.g., DTDs or XML Schema [Thompson et al. 2001]) and integrity constraints, highlights the importance of the study of the interaction.This article investigates that interaction. In particular it studies constraint implication problems, which are important both in understanding the semantics of type/constraint systems and in query optimization. It shows that path constraints interact with types in a highly intricate way. For that purpose a number of results on path constraint implication are established in the presence and absence of type systems. These results demonstrate that adding a type system may in some cases simplify reasoning about path constraints and in other cases make it harder. For example, it is shown that there is a path constraint implication problem that is decidable in PTIME in the untyped context, but that becomes undecidable when a type system is added. On the other hand, there is an implication problem that is undecidable in the untyped context, but becomes not only decidable in cubic time but also finitely axiomatizable when a type system is imposed.

As more and more data-centric services are emerging on the Web, the diversity of APIs in this field increases rapidly. In addition to the traditional relational model, the scope of Database-as-a-Service (DaaS) products is broadened by new approaches, such as the more lenient key-value stores. Many DaaS APIs address the same concepts, making them incompatible with each other, yet not fundamentally different. In this paper, we address this issue by drafting a concept for a "universal" API for DaaS systems based on the REST architectural style. The new API is universal in the sense that it is aimed to be suitable for DaaS systems ranging from those that allow storage of schema-less data to systems with the capabilities of a traditional relational database system. After the presentation of the concept, we critically review it, revealing general problems of such a universal API, and point out a more promising path for future work.

This chapter provides an overview of big data storage technologies. It is the result of a survey of the current state of the art in data storage technologies in order to create a cross-sectorial technology roadmap. This chapter provides a concise overview of big data storage systems that are capable of dealing with high velocity, high volumes, and high varieties of data. It describes distributed file systems, NoSQL databases, graph databases, and NewSQL databases. The chapter investigates the challenge of storing data in a secure and privacy-preserving way. The social and economic impact of big data storage technologies is described, open research challenges highlighted, and three selected case studies are provided from the health, finance, and energy sector. Some of the key insights on big data storage are (1) in-memory databases and columnar databases typically outperform traditional relational database systems, (2) the major technical barrier to widespread up-take of big data storage solutions are missing standards, and (3) there is a need to address open research challenges related to the scalability and performance of graph databases.

In this paper we propose four architectures for intrusion-tolerant database systems. While traditional secure database systems rely on prevention controls, an intrusion-tolerant database system can operate through attacks in such a way that the system can continue delivering essential services in the face of attacks. With a focus on attacks by malicious transactions, Architecture I can detect intrusions, and locate and repair the damage caused by the intrusions. Architecture II enhances Architecture I with the ability to isolate attacks so that the database can be immunized from the damage caused by a lot of attacks. Architecture III enhances Architecture I with the ability to dynamically contain the damage in such a way that no damage will leak out during the attack recovery process. Architecture IV enhances Architectures II and III with the ability to adapt the intrusion-tolerance controls to the changing environment so that a stabilized level of trustworthiness can be maintained. Architecture IV enhances Architecture IV with the ability to deliver differential, quantitative QoIA services to customers who have subscribed for these services even in the face of attacks.

In this paper, we present the design and implementation of ITDB, a self-healing or intrusion-tolerant database prototype system. While traditional secure database systems rely on preventive controls and are very limited in surviving malicious attacks, ITDB can detect intrusions, isolate attacks, contain, assess, and repair the damage caused by intrusions in a timely manner such that sustained, self-stabilized levels of data integrity and availability can be provided to applications in the face of attacks. ITDB is implemented on top of a COTS DBMS. We have evaluated the cost effectiveness of ITDB using several micro-benchmarks. Preliminary testing measurements suggest that when the accuracy of intrusion detection is satisfactory, ITDB can effectively locate and repair the damage on-the fly with reasonable (database) performance penalty.

Implementing crash recovery in an Object-Oriented Database System (OODBMS) raises several challenging issues for performance that are not present in traditional DBMSs. These performance concerns result both from significant architectural differences between OODBMSs and traditional database systems and differences in OODBMS's target applications. This paper compares the performance of several alternative approaches to implementing crash recovery in an OODBMS based on a client-server architecture. The four basic recovery techniques examined in the paper are termed page differencing, sub-page differencing, whole-page logging, and redo-at-server. All of the recovery techniques were implemented in the context of QuickStore, a memory-mapped store built using the EXODUS Storage Manager, and their performance is compared using the OO7 database benchmark. The results of the performance study show that the techniques based on differencing generally provide superior performance to whole-page logging.

Aggregation in traditional database systems is performed in batch mode: a query is submitted, the system processes a large volume of data over a long period of time, and, eventually, the final answer is returned. This archaic approach is frustrating to users and has been abandoned in most other areas of computing. In this paper we propose a new online aggregation interface that permits users to both observe the progress of their aggregation queries and control execution on the fly. After outlining usability and performance requirements for a system supporting online aggregation, we present a suite of techniques that extend a database system to meet these requirements. These include methods for returning the output in random order, for providing control over the relative rate at which different aggregates are computed, and for computing running confidence intervals. Finally, we report on an initial implementation of online aggregation in POSTGRES.

Application programs on application servers issue requests to object servers in traditional database systems. On the other hand, programs named agents move around object servers whose objects are manipulated objects in a mobile agent approach. If an agent conflicts with other agents on an object server, the agents negotiate with each other to resolve the confliction. An agent moves to another servers. In traditional systems, application programs do not work if application servers are faulty. If the server is faulty, the agent find another server where the agent can be performed. In addition, agents are replicated if agents may be faulty. In the mobile agent approach, applications can be operational even if a computer where an agent exist is faulty. In this paper, we discuss how to perform agents with different constraints on multiple object servers in the presence of servers and agent faults.

: The primary accomplishment of this project is a new paradigm for secure database system design, intrusion tolerant database systems. In particular, an innovative intrusion tolerant database system framework, denoted ITDB, is developed. While traditional secure database systems rely on preventive controls, ITDB can detect intrusions, isolate attacks, contain, assess and repair the damage caused by intrusions in a timely manner such that a self-stabilized level of data integrity and availability can be provided to applications. Built on top of COTS DBMS, ITDB arms commercial database servers with the ability to deliver sustained valid data access services even in the face of intensive attacks. To validate ITDB, a prototype ITDB system is designed and implemented. The prototype is a seamless integration of five major subsystems, namely the Malicious Transaction Detection subsystem, the Attack Recovery subsystem, the Attack Isolation subsystem, the Damage Containment subsystem, and the Self-Stabilization subsystem. Extensive evaluation of the prototype based on practical database applications, simulated workload and injected attacks is done. Preliminary testing measurements suggest that when the accuracy of the intrusion detector is satisfactory, ITDB can effectively tolerate database intrusions with reasonable performance penalty.

Query classification in object-oriented engineering design systems

An estimate of the number of items in a database that satisfy an equality or range predicate is useful for several tasks, such as cost-based query optimization, provisioning of system resources, and determining the financial costs of using database services. In a traditional database system, such estimates are computed and used internally by the system and have been well studied. In contrast, such estimates have not received much attention in the context of a cloud-based database service, where they must be computed by the application that uses the service using only the limited features of the interface provided by the service. This paper motivates and formulates the selectivity-estimation problem for database services. It describes the characteristics of this problem that distinguish it from the analogous problem in traditional database systems. It outlines some subproblems and methods to address them. It provides a method for estimating selectivities based on random sampling along with some experimental results.

Query optimization in Big Data becomes a promising research direction due to the popularity of massive data analytical systems such as Hadoop system. The query optimization is getting hard to efficiently execute JOIN queries on top of Hadoop query language, Hive, over limited Big Data storages. According to our previous work, HiveQL Optimization for JOIN query over Multi-session Environment (HOME) system has been introduced over Hadoop system to improve its performance by storing the intermediate results to avoid repeated computations. Time overheads and Big Data storages limitation are considered the main drawback of the HOME system, especially in the case of using additional physical storages or renting extra virtualized storages. In this paper, an index-based system for reusing data called indexing HiveQL Optimization for JOIN over Multi-session Big Data Environment (iHOME) is proposed to overcome HOME overheads by storing only the indexes of the joined rows instead of storing the full intermediate results directly. Moreover, the proposed iHOME system addresses eight cases of JOIN queries which classified into three groups; Similar-to-iHOME, Compute-on-iHOME, and Filter-of-iHOME. According to the experimental results of the iHOME system using TPC-H benchmark, it is found that the execution time of eight JOIN queries using iHOME on Hive has been reduced. Also, the stored data size in the iHOME system is reduced relative to the HOME system, as well as, the Big Data storage is saved. So, by increasing stored data size, the iHOME system guarantees the space scalability and overcomes the storage limitation.

Fiscal institutions face unknown challenges in delivering high-performance distributed operations that can reuse massive sales volumes while maintaining sub-millisecond quiescence conditions. The transition from traditional relational database systems to NoSQL infrastructures represents a significant shift in how financial institutions handle real-time data processing, threat assessment, and request analysis. NoSQL databases offer vertical scalability capabilities that enable fiscal operations to distribute computational loads across multiple nodes, furnishing the outturn necessary for competitive advantage in ultramodern trading environments. The perpetration of microservices infrastructures allows independent scaling of specific fiscal functions, while sophisticated cargo balancing algorithms ensure optimal resource application during peak trading hours. Event-driven infrastructures grease asynchronous processing of nonstop data aqueducts, enabling complex fiscal computations to execute in background processes without impacting stoner-facing operations. Geographic data position optimization reduces network quiescence by strategically situating constantly penetrated information closer to recycling bumps, while time-grounded partitioning strategies enable effective operation of literal compliance data alongside real-time request information. Advanced covering fabrics give comprehensive visibility into system performance, enabling visionary identification and resolution of backups before client impact occurs. The integration of circuit swell patterns and bulkhead insulation mechanisms ensures system adaptability during times of request volatility when sales volumes can spike dramatically.