Research on Indexing Technology for AIS Data Stream

With the popularization of AIS technology on ships, the scale of maritime traffic data are rapidly increasing. The traditional management system of AIS data based on relational databases is faced with difficulties in expansion and low access efficiency. In view of that, we present AISHS, a distributed system based HBase and Spark to offer efficient storage and near-real-time query to massive AIS Data. In order to avoid the communication overhead caused by regroup the trajectory in query process, AISHS stores all data of a ship in one Region of HBase, constructs a global secondary index by the spatial-temporal attributes of AIS data. Based on this, we implement the co-location between HBase Regions and Spark RDD partitions to ensure efficient spatial-temporal query. Experiments on real datasets show that AISHS is suitable for the storage and query of AIS data.

The rise of the digital economy is shaping the future of business and society. In this digital era, data collection and analysis have become crucial to support decision-making, innovation, and efficiency improvement. Cloud computing and the Internet of Things technology, as the core driving forces of the digital economy, are opening new possibilities and providing strong support for data collection and analysis platforms. To broaden the scope of network big data applications, improve the accuracy of network data storage and management to a greater extent, and reduce the time for network data processing and control, a research method for network big data based on cloud computing and the Internet of Things in the context of the digital economy is proposed. The author first uses hierarchical network coding to transmit network data, based on the transmitted data, the CRC algorithm is used to calculate network data, and then the data is stored in a group storage manner, secondly, the high-precision query of network data is carried out using the hierarchical reverse stacking positioning method, thereby completing the research on network big data. Finally, to demonstrate the overall performance of network big data research methods based on cloud computing and the Internet of Things, a simulation experiment is conducted. The experimental results show that when the data query parameter ε is 8, the data query accuracy curve is unstable, and the query accuracy is low. When the data query parameter ε is 4-5, the data query accuracy curve is relatively flat, and the query accuracy is high. The method proposed by the author can comprehensively and concretely study network big data, improve data processing accuracy and network data calculation speed, increase network data storage capacity and query efficiency, reduce the loss rate of network data during operation, expand the operating range of network data, and provide a strong basis for subsequent research on network big data.

There are nearly 50 million AIS data generated every day in the Yangtze River Basin. Efficient storage and query of AIS data become the key problem of smart maritime administration. To overcome this problem, a distributed storage and query scheme based on Hadoop platform is proposed. Firstly, the original AIS message is parsed, the abnormal data is removed and the parsed data is pushed to the Kafka message queue. Secondly, the messages in the queue are pushed to different servers to complete the storage of real-time cache, historical trajectory and regional information. Finally, the query interface is provided to realize the real-time query of ship status, trajectory and regional ship in a certain period of time. The real applications show that compared with the traditional relational database, the solution can meet the requirements of real-time query and has high scalability.

Relational database technology has enabled easy and efficient management of data in a variety of applications. This technology, however, has been used for computer simulations to a limited extent. In this context, users may need to analyze the results of multiple simulation runs, which typically capture interactions between various entities in the simulated world. To these users, storing such simulation data only in the form of relational tables may seem counter-intuitive or challenging, particularly if they are not familiar with database normalization theory. Furthermore, some complex queries, including those that examine the cumulative effect of an action (e.g., disease propagation after an initial outbreak), can neither be easily expressed nor efficiently executed on traditional database systems. We propose a new database approach that aims to achieve convenient and highly efficient storage and querying of simulation data by adopting a graph data model. We also discuss new challenges that arise in this research, with a focus on language design, coordination of simulations, data storage, and query processing.

New generation of health care IT systems are collecting and storing more and more data of patients. Useful knowledge can be extracted from the data in EMR or PHR to provide medical advises to patients, while through data analysis the result statistics can be used to support the scientific research. However, RDBMSs-based framework is not able to support the requirements of massive health care data storage, management and analysis. To solve the problem, this paper proposes a massive data management and analysis solution based on Hadoop to archive better performance, scalability and fault tolerance. The data management framework is presented. Besides, 2 different data analysis methods based on MapReduce and Hive are proposed. Experiment results of data upload, data query and data analysis show that the performance of the proposed framework is greatly improved, and a brief summary of the performance and the differences between 2 methods of MapReduce and Hive is also discussed.

In the realm of data analytics, Power BI and SQL serve as essential tools for businesses aiming to derive actionable insights from their data. However, as datasets grow in complexity and volume, the efficiency of queries and the speed of data loading become critical factors affecting overall performance. This paper explores advanced performance tuning techniques for both Power BI and SQL, focusing on enhancing query efficiency and minimizing data load times. We investigate key strategies, including the optimization of data models, effective use of DAX (Data Analysis Expressions) functions, and the implementation of indexing and partitioning in SQL databases. Additionally, we analyze the impact of data source configurations and the importance of using best practices in report design to improve user experience. By conducting empirical tests and case studies, we illustrate how these tuning methods can lead to significant improvements in report rendering times and user interactions. Ultimately, our findings emphasize the need for a holistic approach to performance tuning, combining technical optimizations with best practices in data management. This research provides valuable insights for data professionals seeking to enhance the efficiency of their Power BI reports and SQL queries, ensuring that organizations can make data-driven decisions swiftly and effectively. The outcomes of this study not only contribute to the field of business intelligence but also pave the way for future research on optimizing analytics platforms.

NoSQL and relational databases have been integrated in response to the increasing demand for scalable and efficient data management in hybrid cloud environments. The differences in data structures and query processing methods between these databases present both challenges and opportunities when designing an optimized hybrid system. This study explores the integration of NoSQL and relational databases to maximize data storage, retrieval, and processing efficiency across multiple cloud systems. NoSQL databases excel in handling unstructured and semi-structured data, offering flexibility and scalability, whereas relational databases provide robust consistency and structured query capabilities. Despite the wide availability of relational databases, certain applications require the dynamic adaptability of NoSQL systems, making integration a viable solution. The research evaluates key performance parameters, including query execution speed, scalability, data consistency, and resource utilization. Cloudsim is used for simulation, allowing for an in-depth comparison between standalone and hybrid database models. Experimental results indicate that the proposed hybrid model improves query performance by 30%, reduces latency by 20%, and enhances scalability by 40% compared to using relational or NoSQL databases alone. The novelty of this approach lies in its ability to overcome the limitations of each database type by their strengths in a dynamic cloud environment. The results highlight the effectiveness of hybrid database integration in optimizing cloud-based data management while ensuring seamless operation, adaptability, and resource efficiency.

How to realize the safe and efficient use of power terminal equipment data is a key research content to enhance the value of smart grid data. In order to solve this problem, based on the master-slave blockchain theory, this paper designs an intelligent terminal data management system architecture based on the master-slave blockchain. The architecture includes three modules: master blockchain, slave blockchain, and system service interface. In order to improve the security of data in the process of data use, this paper conducts research from the two dimensions of data security storage and data access control, and proposes a data security storage mechanism and a data access control mechanism. In order to improve the efficiency of data query when the amount of data is large, this paper adopts the grouping bloom filtering strategy and the B+ tree index strategy to reduce the number of data queries and improve the efficiency of data query. In the experimental part, from the performance analysis of the improved Bloom filter based on grouping and the performance analysis of the block query mechanism based on the B+ tree index, it is verified that the data usage mechanism proposed in this paper has high block creation and data query efficiency.KeywordsSmart gridPower terminalData securityBlockchain

Nowadays, many applications require to represent both data and knowledge together. Regardless of the requirement, the normal practice is either using only database or only ontology. These approaches are proven to have drawbacks. There is still no proper methodology to obtain the benefits from both ontology and database management systems together. The issues raised by the rapprochements between such worlds are well known and addressed by consolidated mapping and transforming languages which may result in loss of data or hindering the reasoning power in knowledge. Nevertheless, to the best of our knowledge, a best practice for establishing such rapprochement is missing which we present through this research; a guideline based cooperation between database and ontology. We have introduced a hybrid approach of relational database and ontology for optimal management and querying of data and knowledge. Our method optimizes the storing mechanism by allowing of distributing data between ontology and database. Further, it optimizes the query execution by allowing some part of the data to be queried through database via SQL and some part of the data to be queried through ontology via SPARQL. We were able to obtain promising results about the performance in terms of storage and query execution and accuracy from the proposed approach.

Efficient secondary memory processing of window queries on spatial data

The 3D geoinformation is becoming important for cities and their policies. The cities are therefore exploring the possibilities of 3D virtual city models for more efficient decision making. To maximize the economic benefit of such data, the cities can provide their 3D geospatial data for further usage, and so, new applications can be created. The paper defines a way how the freely available 3D geospatial data of Prague can be transformed from the proprietary data format into the open data model. The 3D geospatial data about the buildings, bridges and digital terrain model were transformed from the 3D shapefile into the CityGML. This is an application independent information model and exchange format. This will allow for the wider use of the 3D city model by different groups of users. The generated CityGML files were further imported into the spatial database with appropriate database CityGML-based scheme. It enables more efficient management and querying of CityGML data. To enable the wider audience to explore the 3D city model, the visualization in the web environment was also explored. The paper also presents the way how the attributes from the external data sources can be connected to the 3D objects in the web environment.

The efficient index structure has important applications in the massive data query. According to the deficiency of the R-tree in the data cluster and the blank space utilization, a new hybrid index structure R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BRT</sup> tree was presented based on the minimum bounding rectangle (MBR) and the minimum bounding right trapezoid (MBRT). The characteristics of the spatial plane partition and the spatial data structure were discussed and the operating algorithms for the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BRT</sup> tree were studied. The methods for the extension of the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BRT</sup> tree were also discussed, Furthermore, the comparison and analysis for the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BRT</sup> tree and the R-tree were made. It concludes that the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">BRT</sup> tree has advantages on the blank space utilization, search accuracy and data storage for some datasets compared with R tree.

As the country promotes the reform of science and technology project management, the number of science and technology projects has been increasing. At the same time, science and technology management data is growing rapidly with the characteristics of complex correlation and multi-dimensionality. How to effectively store the enormous data and mine the contained value of the data have become a crucial problem for current researches. First, we propose a science and technology management data model to illustrate the internal relation of project, topic, unit, and person, and then define the model by mathematical representation. According to science and technology management data query requirements, we present a data query method of science and technology management based on relational engine, which comprises query parser, query optimizer, and query executor. Furthermore, we design a prototype system to achieve the query requirements of science and technology management data. This research contributes to data query by providing a query method with relational engine, and improves the efficiency of data query in science and technology management.

The emergence of new hardware architectures, and the continuous production of data open new challenges for data management. It is no longer pertinent to reason with respect to a predefined set of resources (i.e., computing, storage and main memory). Instead, it is necessary to design data processing algorithms and processes considering unlimited resources via the “pay-as-you-go” model. According to this model, resources provision must consider the economic cost of the processes versus the use and parallel exploitation of available computing resources. In consequence, new methodologies, algorithms and tools for querying, deploying and programming data management functions have to be provided in scalable and elastic architectures that can cope with the characteristics of Big Data aware systems (intelligent systems, decision making, virtual environments, smart cities, drug personalization). These functions, must respect QoS properties (e.g., security, reliability, fault tolerance, dynamic evolution and adaptability) and behavior properties (e.g., transactional execution) according to application requirements. Mature and novel system architectures propose models and mechanisms for adding these properties to new efficient data management and processing functions delivered as services. This paper gives an overview of the different architectures in which efficient data management functions can be delivered for addressing Big Data processing challenges.

Background: Efficient data management and escalation processes are crucial for facilitating continuous improvement initiatives. Stroke care necessitates quick identification of opportunities for improvement (OFIs) related to core measures and performance improvement initiatives (PI). This abstract presents an overview of a tailored data management and escalation process designed to enhance stroke team performance through the utilization of SharePoint™. Methods: SharePoint™ list was configured to simultaneously capture OFIs identified by the stroke team. An automated email notification system triggers a prompt alert to the assigned parties, including unit department administrators and stroke champions, regarding the new entry. The assigned parties receive the email notification along with an audit link, granting them direct access to the corresponding audit form. Stakeholders then analyze the OFIs, develop action plans and assess trends in the data. This process supports continuous learning and adaptation, enabling the stroke team to refine strategies and optimizes compliance with The Joint Commission stroke standards. Results: From October 2021 through July 2024, 2,292 escalations were issued. The mean number of escalations per unit was 134.82. Most departments (8 out of 17) have escalations between 0 and 50. The highest number of escalations were in the ICU 639 (28%), followed by Interventional Radiology 542 (24%), and PACU 226 (10%). There were a total of 22,668 site visits to the SharePoint™ site, with 210 unique viewers. Figure 1 shows the SharePoint™ entry form that is utilized for escalation and analysis. Figure 2 displays PowerBI™ visualization depicting the unit summary of escalations by month, unit, and OFI type. Figure 3 demonstrates the site usage traffic. Conclusion: The data management and escalation process outlined in this abstract represents a paradigm shift in stroke team performance improvement strategies. By harnessing the capabilities of SharePoint™ and leveraging automated notifications, real-time data analytics, and collaborative action planning, healthcare organizations can effectively navigate the complexities of performance improvement in a rapidly evolving landscape. This process not only enhances operational efficiency but also empowers stakeholders to drive meaningful change, ultimately leading to improved patient outcomes and enhanced quality of care driven by sustainable PI.