CloudMdsQL: querying heterogeneous cloud data stores with a common language

The production of huge amount of data and the emergence of cloud computing have introduced new requirements for data management. Many applications need to interact with several heterogeneous data stores depending on the type of data they have to manage: relational and NoSQL (i.e., document, graph, key-value, and column) data stores. Interacting with heterogeneous data models via different APIs and query languages imposes challenging tasks to the developers of multiple data stores applications. Indeed, the execution of complex queries over heterogeneous data models cannot, currently, be achieved in a declarative way as it is used to be with single data store application, and therefore requires extra implementation efforts. In this paper we propose a mediation based component to optimize and execute complex queries over multiple data stores in Cloud environments. This component is referred to as virtual data store (VDS). The key ingredients of our solution are (1) a simple global schema describing the different data sources and their relationships, (2) a cost model to evaluate the cost of the operations, (3) an inter data stores parallelism execution model, and (4) a dynamic programming based approach to generate optimal execution plan. Quantitative and qualitative experiments are conducted to validate our approach.

Developing a collaborative application running on a collection of heterogeneous, possibly mobile, devices, each potentially hosting data stores, using existing middleware technologies such as JXTA, BREW, compact .NET and J2ME requires too many ad-hoc techniques as well as cumbersome and time-consuming programming. Our System on Mobile Devices (SyD) middleware, on the other hand, has a modular architecture that makes such application development very systematic and streamlined. The architecture supports transactions over mobile data stores, with a range of remote group invocation options and embedded interdependencies among such data store objects. The architecture further provides a persistent uniform object view, group transaction with Quality of Service (QoS) specifications, and XML vocabulary for inter-device communication. This paper presents the basic SyD concepts and introduces the architecture and the design of the SyD middleware and its components. We also provide guidelines for SyD application development and deployment process. We include the basic performance figures of SyD components and a few SyD applications on Personal Digital Assistant (PDA) platforms. We believe that SyD is the first comprehensive working prototype of its kind, with a small code footprint of 112 KB with 76 KB being device-resident, and has a good potential for incorporating many ideas for performance extensions, scalability, QoS, workflows and security.

The paper is devoted to organization the storage of heterogeneous data in the karst processes monitoring system. We discussed the basic requirements for the storage system and described its structure. The system includes a databases monitoring of surface karst, monitoring of underground karst processes, monitoring of hydrogeological processes, the photographic collection of surface karst forms and cartographic collection. We present the results of the practical implementation of the system on the example research data of area allocated for the construction of the nuclear power plant in Nizhny Novgorod on Monakovo.

In order to solve the storage and management problems of heterogeneous drug data, this paper uses big data technology to complete the cleaning and distributed storage of drug data, and improve the function of data sharing and traceability. At the same ...

With the rapid development of the Internet of Things (IoT), the IoT is characterized by a wide variety of data sources, large scale and heterogeneous structure. But those characteristics bring great difficulties to the storage and rapid retrieval of IoT data. By considering the common attributes of IoT data, based on plug-in ideas, combined with Redis and HBase, the paper proposes a framework named HSFRH-IoT, which solves the problem of efficient storage and retrieval of massive heterogeneous IOT. Finally, the insertion and query performance of the proposed HSFRH-IoT framework is tested in detail, and the results shows that it has better performance than other RDBMS based solutions.

The data connection of electric power marketing and distribution network is an integrated platform of operation and distribution data based on geographic information platform, which integrates the data of marketing business application system, production management information system and grid geospatial information service platform, and shares the whole face of production data and marketing customer data, so as to achieve the connection of operation and distribution data and business, and realize fault location and electric failure Scope analysis, line loss statistics, business expansion and installation, etc. Graph database has the technical advantages of high efficiency of topological data organization, low redundancy of topological data storage, strong expansibility of topological data model and high efficiency of topological analysis and calculation, which can meet the storage, query and retrieval requirements of large-scale operation and distribution data. After application verification, the results show that the graph database can provide data storage, query and retrieval technology scheme for the operation and distribution of power grid. With the business development of operation and distribution of power grid, there are performance bottlenecks in the existing data storage mode, which seriously restricts the deepening of application. It is necessary to introduce advanced computing technology to achieve breakthrough, and improve the data storage efficiency of the existing data platform The parallel and concurrent ability of data retrieval and query can meet the management requirements of geometric growth of power grid equipment, the analysis requirements of real-time data of power grid, and the concurrent access requirements of largescale growth of users.

The era of time series data as its implementations and interpretation have recently become increasingly relevant in different domains and fields. Numerous fields of industry and science rely on collecting and examining vast quantities of time series data—finance and economics, the Internet of things, medicine, environmental protection, hardware surveillance, among several others. The main structural problems in time series data analysis are the collection and retrieval in vast quantities of data. In addition, time series data analysis seems to be of tremendous value, since previous patterns are helpful in predicting future results. Owing to the considerable latency in data volume changes, the frequency of persistence and the absence of inborn structures in time series, the traditional hierarchical data retrieval method would not seem to be able to analyze time series data efficiently. Often, many traditional data storage solutions do not support time series-based operators, which result in unreliable time series service. The description of effective and stable local data storage for time series data has been provided in this paper. The study of wind/solar time series data is represented by changing the number of rows, the number of days and the number of columns with a time resolution of 10, 15, 30 and 60 min. The results of the experiment display time series data access time, write period and storage size of the parquet data storage format that has proved to be an effective and reliable data storage device for the local centralized server. The data storage model of Apache Parquet appears to be effective as compared to other file storage formats such as CSV, HDF5 and FEATHER.

Number of networks and their sizes in terms of connected devices are increasing very rapidly. These devices are producing massive amount of raw data for different progressive purposes that need to be processed and stored somewhere. Local data storage and complete processing of massive data will not be possible by resource constrained Internet of Things (IoT) enabling devices anymore in the future. Therefore, there is a need of data storage and processing capabilities to meet the desired goals of applications. Cloud computing and edge computing are the possible solution to circumvent limited data storage and computation issues of IoT devices. This paper presents a comparative study of popular architectures in which IoT devices are integrated with cloud. For this purpose, various IoT and Cloud of Things (CoT) architectures have been studied. Strengths and weaknesses of the studied IoT and CoT architectures are comprehensively analyzed on the basis of five comparison parameters namely; resource based segregation of IoT enabling devices, range based segregation of the communication network, segregation of security solutions, cloud connectivity, and edge computing services.

In the last years, polystore databases have been proposed to cope with the challenges stemming from increasingly dynamic and heterogeneous workloads. A polystore database provides a logical schema to the application, but materializes data in different data stores, different data models, and different physical schemas. When the access pattern to data changes, the polystore can decide to migrate data from one store to the other or from one data model to another. This necessitates a schema evolution in one or several data stores and the subsequent migration of data. Similarly, when applications change, the global schema might have to be changed as well, with similar consequences on local data stores in terms of schema evolution and data migration. However, the aspect of schema evolution in a polystore database has so far largely been neglected. In this paper, we present the challenges imposed by schema evolution and data migration in Polypheny-DB, a distributed polystore database. With our work-in-progress approach called PolyMigrate, we show how schema evolution and data migration affect the different layers of a distributed polystore and we identify different approaches to effectively and efficiently propagate these changes to the underlying stores.

The significance of the cloud enables the data owners (DOs) to store data remotely in cloud server (CS). The external and internal attacks on the stored data at CS can deliberately remove data. Furthermore, the CS removes the stored data to make empty location for the user's upcoming new data. However, it is a legal expectation of DOs to know whether their data are correctly stored or altered in CS. In this article, we propose a novel privacy-aware and hash-parity-bits-based public auditing (PA-HPPA) framework to secure full data, left half of the data, and the right half of the data, generated by a DO. DO generates two private key pairs with the assistance of a virtual key and a user ID (IP). The virtual key is the sequence number of DO who is registered and provided by the trusted data manager (TDM) while IP is the sequence number of DO working in an organization. Subsequently, DO blinds the categorized data and generates their signatures and hashes. In addition, DO generates the parity bits using xor and assigns to each hard drive (HD) in CS, which assistants to TDM in public auditing. Second, how to identify the error in the stored data and how to securely recover the error/missed data? Extension to the framework, the novel proposed data error identification and secure data recovery produce tags for installed HDs of CS using truth table and recover the altered/missed data via a authenticator, which is produced using xor function. Third, how to protect a valid user from revocation and, in case a user has revoked on merit basis, then how to securely access the stored data of it? This novel work has proposed three conditions to meet the validity of the valid user from revocation and securely generating the public-private key pairs to access the stored data of the revoked user securely from CS. Fourth, there is an efficient novel proposed dynamic operation scheme to insert, update, or delete the stored data at CS without regenerating the signatures, hashes, and tags for the whole stored data in cloud. The security analysis and the performance evaluation of the proposed solutions are provably efficient and secure with reduced communication costs.

SyD: a middleware testbed for collaborative applications over small heterogeneous devices and data stores

The emergence of cloud computing brings the infinite imagination space, both in individual and organizations, due to its unprecedented advantages in the IT history: on-demand self-service, ubiquitous network access, location independent resource pooling, rapid resource elasticity, usage-based pricing and transference of risk. Many of the individuals or organizations ease the pressure on their local data storage, and mitigate the maintenance overhead of local data storage by using outsource data to cloud. However, the data outsourcing is not absolutely safe in the cloud. In order to enhance the users’ confidence of the integrity of their outsource data in the cloud. To promote the rapid deployment of cloud data storage service and regain security assurances with outsourced data dependability, many scholars tend to design the Remote Data Auditing (RDA) technique as a new concept to enable public auditability for the outsourced data in the cloud. The RDA is a useful technique to ensure the correctness of the data outsourced to cloud servers. This paper presents a comprehensive survey on techniques of remote data auditing in cloud server. Recently, more and more remote auditing approaches are categorized into the three different classes, that is, replication-based, erasure coding-based, and network coding-based to present a taxonomy. This paper also aims to the explore major issues.

Clinical and translational research often involves the generation, collection, storage, management, analysis, and dissemination of heterogeneous and multi-dimensional data, information, and knowledge resources. Addressing such fundamental informatics needs and requirements is a challenging problem that usually requires the collaboration of multi-disciplinary teams. Central to the ability to form and operate such teams is the development of a workforce with sufficient expertise at the basic and applied science levels as relevant to the clinical and translational science domain. Over the last decade, significant advances have been made in education and training programs targeting such workforce development. While still early in their maturity, such efforts have already begun to impact the advancement of biomedical science and human health, and serve to illustrate the critical nature of computational and informatics theories and methodologies across the broader translational research spectrum.KeywordsTranslational Research InformaticsApplied Science LevelBiomedical InformaticsTranslational SpectrumClinical And Translational Science Awards (CTSA)These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Accurate model aggregation is pivotal for the efficient dispatch and control of massive distributed photovoltaic (PV) and energy storage (ES) resources. However, the lack of unified standards across equipment manufacturers results in inconsistent data formats and resolutions. Furthermore, external disturbances like noise and packet loss exacerbate the problem. The resulting data are massive, multi-source, and heterogeneous, which poses severe challenges to building effective aggregation models. To address these issues, this paper proposes a hierarchical aggregation method based on multi-source multi-scale data fusion. First, a Multi-source Multi-scale Decision Table (Ms-MsDT) model is constructed to establish a unified framework for the flexible storage and representation of heterogeneous PV-ES data. Subsequently, a two-stage fusion framework is developed, combining Information Gain (IG) for global coarse screening and Scale-based Trees (SbT) for local fine-grained selection. This approach achieves adaptive scale optimization, effectively balancing data volume reduction with high-fidelity feature preservation. Finally, a hierarchical aggregation mechanism is introduced, employing the Analytic Hierarchy Process (AHP) and a weight-guided improved K-Means algorithm to perform targeted clustering tailored to the specific control requirements of different voltage levels. Validation on an IEEE-33 node system demonstrates that the proposed method significantly improves data approximation precision and clustering compactness compared to conventional approaches.

In the 21st century, transportation brought great convenience to people, but at the same time, automobile transportation is the major factor causing greenhouse gas emissions and climate change. Movements of the world towards green environments, there is hike in use and production of electric vehicles (energy vehicles). However, with the continuous growth in the number of energy vehicles, it is necessary for the government to provide strong support in the construction of charging piles. Real-time and effective management has become a practical problem for the relevant departments which needs to be solved. This paper uses the information research method to fuse the huge amount of heterogeneous data generated by the charging pile resultant to the new energy electric vehicle in the vehicle network and introduces cloud computing as its storage module to facilitate the storage and related expansion of the big data. This paper proposes a system scheme of heterogeneous data fusion based on cloud computing for the acquisition, storage, and fusion of heterogeneous data in the vehicle network. After testing the results, it shows that the system is stable and effective in practical application, which can meet the design requirements of the system. What is the significance of analyzing big data of charging point? Considering from the supply side, obtaining the user’s charging behaviour data is helpful to build a digital map of the charging pile of new energy vehicles, connect the service information between the vehicle enterprises and the charging pile enterprises, and provide the most comprehensive and effective real-time charging information covering the widest range of vehicles, which can solve many problems of information asymmetry in the current charging information service.