Data partition optimisation for column-family NoSQL databases

A new computer analysis tool capable of predicting rocket-based combined-cycle engine performance data (thrust and Isp) over a wide range of e ight conditions and engine operating modes is presented. The tool is called SCCREAM, which stands for simulated combined-cycle rocket engine analysis module. SCCREAM is an objectorientedworkstation-levelcodewritteninC++thatcanberemotelyexecutedthroughaweb-baseduserinterface.It uses quasi-one-dimensional e ow analysis techniques, componentand combustion efe ciencies, and an inlet pressure recovery schedule as simplifying assumptions. SCCREAM was created for the conceptual launch vehicle design environment and is capable of quickly generating large tables of engine performance data for use in subsequent trajectory optimization. An overview of SCCREAM and the program logic is presented. Results from SCCREAM are compared with published rocket-based combined-cycle engine performance data and with data generated by other engine analysis tools for a representative reference vehicle cone guration.

With the diversification of applications and the diversity of users, the era of big data is always producing various new types of data, such as graph data, time series data, and spatial data. When predicting the future trend of the system, we always need to learn from the historical working state, and time series data plays a huge value in this aspect. Hardware resources are becoming cheaper, computing power has made major breakthroughs, and multi-machine and multi-core scenarios have become commonplace. At present, the focus of large-scale time series data processing is either focused on data collection and storage, or on single-machine processing, which has been unable to meet the existing needs. In response to this problem, this paper designs and implements a distributed large-scale time series processing and analysis system based on the Spark platform. The system framework is mainly divided into storage layer, operator layer and algorithm layer. At the storage layer, the system completes the organization and indexing of large-scale time series data based on HDFS and Hive. At the operator layer, the system provides users with basic operations commonly used for time series data on the Spark platform, and allows users to directly use these operators to implement custom time series related processing algorithms. At the algorithm level, the system implements some commonly used time series analysis algorithms in the Spark platform, including time series similarity query, clustering and prediction, so that users can directly use these algorithms to analyze time series. Finally, the feasibility and practicability of this system are verified by testing the performance and function of the system.

To improve the ability of biologists (both researchers and students) to ask biologically interesting questions of the Gene Ontology (GO) database and to explore the ontologies by seeing large portions of the ontology graphs in context, along with details of individual terms in the ontologies. GoGet and GoView are two new tools built as part of an extensible web application system based on Java 2 Enterprise Edition technology. GoGet has a user interface that enables users to ask biologically interesting questions, such as (1) What are the DNA binding proteins involved in DNA repair, but not in DNA replication? and (2) Of the terms containing the word triphosphatase, which have associated gene products from mouse, but not fruit fly? The results of such queries can be viewed in a collapsed tabular format that eases the burden of getting through large tables of data. GoView enables users to explore the large directed acyclic graph structure of the ontologies in the GO database. The two tools are coordinated, so that results from queries in GoGet can be visualized in GoView in the ontology in which they appear, and explorations started from GoView can request details of gene product associations to appear in a result table in GoGet. Free access to the GoGet query tool and free download of the GoView ontology viewer are provided to all users at http://db.math.macalester.edu/goproject. In addition, source code for the GoView tool is also available from this site, along with a user manual for both tools.

The direct searching for relevant reducts in the set of all reducts of a given data table can be often computationally infeasible, especially for large data tables. Hence, there is a need for developing efficient methods for extracting relevant information about reducts from data tables which could help us to perform efficiently the inducing process of the high quality data models such as rule based classifiers. Such relevant information could help, e.g., to reduce the dimensionality of the attribute set. We discuss methods for generating relevant information about reduct sets from information systems or decision tables. In particular, we consider a binary relation on attributes satisfied for two given attributes if and only if there is no reduct consisting them both. Moreover, we prove that for any fixed natural k , there exists a polynomial in time algorithm which for a given decision table T and given k conditional attributes recognizes if there exists a decision reduct of T covering these k attributes. We also present a list of problems related to the discussed issues. The reported results create a step toward construction of a software library reducing the searching costs for relevant reducts.

Processing DICOM datasets for research and education is challenging due to the format's complexity and frequent patient-specific workflow exceptions. Proper handling demands substantial technical expertise and meticulous care to ensure data fidelity in downstream applications. We developed MedicalDataHandler to streamline the reading, visualization, and processing of DICOM data. By consolidating essential tasks into a user-friendly environment, it minimizes reliance on advanced coding skills and promotes reproducible data handling without custom scripting. Implemented in Python with the third-party Dear PyGui toolkit, MedicalDataHandler organizes DICOM files by patient identifiers and groups each patient's radiation therapy (RT) images, structure sets, plans, and doses based on mutual unique identifiers (UIDs). A comprehensive table of patient data enables metadata inspection, data visualization, and data processing. The GUI supports interactive visualization in axial, coronal, and sagittal views, with intuitive scrolling, zooming, panning, and window width/level adjustments. Segmentation labels, colors, and data orientation can be modified on the fly, and hovering over a voxel reveals its image/dose values and relevant segmented structures. Multithreading and multiprocessing enable rapid data reading and conversion to the deep-learning-friendly Nearly Raw Raster Data (NRRD) format. Additional features include metadata inspection, voxel grid resampling, Hounsfield-Unit-to-Relative-Electron-Density mapping, plan-sum dose creation, and partial or bulk data saving options. We validated MedicalDataHandler with an end-to-end testing approach. DICOM data from 61 radiotherapy patients were processed, and the resulting dataset was used to train a deep-learning-based dose prediction model. MedicalDataHandler streamlined the workflow by eliminating the need for complex, patient-specific code and accelerating the preparation of a research-ready dataset. MedicalDataHandler streamlines DICOM data management and accelerates preprocessing, serving as a valuable tool for researchers and trainees. Its intuitive interface, flexible editing, and rapid data conversion empower a broader audience to manage DICOM data efficiently and consistently in research and education settings.

Business Intelligence (BI) is recognized as one of the most important IT applications in the coming big data era. In recent years, non-uniform memory access (NUMA) has become the de-facto architecture of multiprocessors on the new generation of enterprise servers. Such new architecture brings new challenges to optimization techniques on traditional operators in BI. Aggregation, for example, is one of the basic building blocks of BI, while its processing performance with existing hash-based algorithms scales poorly in terms of the number of cores under NUMA architecture. In this paper, we provide new solutions to tackle the problem of parallel hash-based aggregation, especially targeting at domains of extremely large cardinality. We propose a NUMA-aware radix partitioning (NaRP) method which divides the original huge relation table into subsets, without invoking expensive remote memory access between nodes of the cores. We also present a new efficient aggregation algorithm (EAA), to aggregate the partitioned data in parallel with low cache coherence miss and locking costs. Theoretical analysis as well as empirical study on an IBM X5 server prove that our proposals are at least two times faster than existing methods.

Correspondence analysis in the study of lithogeochemical data: General strategy and the usefulness of various data-coding schemes

The echo state network (ESN) employs a huge reservoir with sparsely and randomly connected internal nodes and only trains the output weights, which avoids the suboptimal problem, exploding and vanishing gradients, high complexity and other disadvantages faced by traditional recurrent neural network (RNN) training. In light of the outstanding adaption to nonlinear dynamical systems, ESN has been applied into a wide range of applications. However, in the era of Big Data, with an enormous amount of data being generated continuously every day, the data are often distributed and stored in real applications, and thus the centralized ESN training process is prone to being technologically unsuitable. In order to achieve the requirement of Big Data applications in the real world, in this study we propose an algorithm and its implementation for distributed ESN training. The mentioned algorithm is based on the parallel particle swarm optimization (P-PSO) technique and the implementation uses Spark, a famous large-scale data processing framework. Four extremely large-scale datasets, including artificial benchmarks, real-world data and image data, are adopted to verify our framework on a stretchable platform. Experimental results indicate that the proposed work is accurate in the era of Big Data, regarding speed, accuracy and generalization capabilities.

This systematic review provides an in-depth examination of regression algorithms applied in predictive analytics across multiple domains, including engineering, transportation, and education. The study synthesizes developments in both traditional statistical and modern machine learning (ML) regression models, outlining their evolution, taxonomy, and comparative performance. Traditional multiple linear regression (MLR) has been foundational for predictive analysis but struggles with scalability and computational efficiency in the era of big data. To address these limitations, adaptations such as MapReduce-based MLR have been proposed, enabling distributed computation for large-scale data processing. Simultaneously, hybrid and ML-based regression models—such as support vector regression (SVR), regression trees (CART), and integrated linear–deep learning frameworks—demonstrate superior performance in modeling complex, non-linear relationships.This review establishes a structured taxonomy differentiating regression approaches based on assumptions, computational properties, and domain-specific requirements, highlighting the balance between model interpretability and predictive power. Comparative evaluations using metrics such as variance explained (R²), mean squared error (MSE), and computational scalability reveal that hybrid regression models often outperform conventional techniques in accuracy and adaptability. The review also identifies persistent challenges related to big data management, model interpretability, data harmonization, and real-time scalability. Emerging research trends emphasize explainable AI (XAI), ensemble regression strategies, and enhanced distributed computing frameworks to overcome these obstacles. Ultimately, this study contributes a holistic understanding of regression-based predictive analytics, guiding researchers and practitioners toward developing scalable, interpretable, and domain-adaptive models for next-generation data-driven decision-making.

Semi-Supervised Learning (SSL) techniques have become very relevant since they require a small set of labeled data. In this context, graph-based algorithms have gained prominence in the area due to their capacity to exploiting, besides information about data points, the relationships among them. Moreover, data represented in graphs allow the use of collective inference (vertices can affect each other), propagation of labels (autocorrelation among neighbors) and use of neighborhood characteristics of a vertex. An important step in graph-based SSL methods is the conversion of tabular data into a weighted graph. The graph construction has a key role in the quality of the classification in graph-based methods. This paper explores a method for graph construction that uses available labeled data. We provide extensive experiments showing the proposed method has many advantages: good classification accuracy, quadratic time complexity, no sensitivity to the parameter k > 10, sparse graph formation with average degree around 2 and hub formation from the labeled points, which facilitates the propagation of labels.

In the era of big data, due to the rapid expansion of the data, the existing incremental text clustering algorithm has the drawback that the efficiency of algorithm will sharp decline with the time and data volume increasing. Because of poor timeliness and robustness, the algorithms are hard to be applied in practice. In this paper, we propose a distributed model framework of Single-Pass algorithm based on MapReduce, the experiments result of increment text cluster is accuracy, the algorithm effectively improve the computing efficiency of the algorithm and real-time of result. Algorithm has a great prospect under the background of big data.

Big data has emerged as the next technological revolution in IT industry after cloud computing and the Internet of Things. With the development of climate observing systems, particularly satellite meteorological observation and high-resolution climate models, and the rapid growth in the volume of climate data, climate prediction is now entering the era of big data. The application of big data will provide new ideas and methods for the continuous development of climate prediction. The rapid integration, cloud storage, cloud computing, and full-sample analysis of massive climate data makes it possible to understand climate states and their evolution more objectively, thus predicting the future climate more accurately. This paper describes the application status of big data in operational climate prediction in China; it analyzes the key big data technologies, discusses the future development of climate prediction operations from the perspective of big data, speculates on the prospects for applying climatic big data in cloud computing and data assimilation, and puts forward the notion of big data-based super-ensemble climate prediction methods and computerbased deep learning climate prediction methods.

The rapid growth in global sensor networks is leading to an explosion in the volume, velocity and variety of geospatial and geoscientific data. This coupled with the increasing integration of geospatial data into our everyday lives is also driving an increasing expectation of spatial information on-demand, and with minimal delay. The challenge we face in meeting these expectations is how to convert the vast and increasing volume of data being delivered from an increasingly diverse network of sensor things into meaningful information in a timely fashion. A key factor in meeting these challenges is the development and implementation of international standards (through the Open Geospatial Consortium [OGC®] and other international standards bodies) focused on establishing common methods and protocols that enable machine-to-machine transfer of data from sensors to people via the internet. However, until recently there has been a gap in the standardization of the spatial data architectures and technologies used to manage the geospatial data at the server-side and this has led to numerous ad-hoc and bespoke spatial data infrastructures being constructed which are potentially incompatible with each other and the wider ecosystem of the Internet of Things. Over the past 3 years the OGC has been working at filling this gap through the development the first international standard for Discrete Global Grid Systems (DGGS), with this new standard expected to be published in 2017. DGGS are a form of Earth reference system that represents the Earth using a hierarchical equal-area tessellation of the surface of the Earth and are designed to ensure a repeatable representation of measurements that is better suited to today's requirements and technologies. A DGGS presents a common framework that is capable of linking very large multi-resolution and multi-domain datasets together to enable the next generation of analytic processes to be applied. To date, DGGS haven't been used directly in the context of sensor networks. This paper proposes to demonstrate the application of DGGS to the Internet of Things and to highlight the immense benefit and value of using standards to enable interoperability in the era of Big Data.

With the rapid growth of image data in recent years, efficient management and retrieval of image data have become increasingly important. In this paper, we propose an image warehouse architecture in the era of big data that combines data preprocessing, compression and deduplication, distributed processing and parallel computing, machine learning, and deep learning, and security and privacy techniques to improve the efficiency and scalability of image warehouses. We conducted experiments on a large-scale image dataset, and the results show that our approach significantly outperforms existing methods in terms of retrieval accuracy and efficiency. The proposed architecture provides a promising solution for managing and retrieving large-scale image data in the era of big data

ObjectiveThis study aims to explore the development prospects and trends of digital geriatric medical care in the era of big data. By taking the Longhu Ecological Senior Apartment in Bengbu City as a case example, the study examines the integration of intelligent healthcare systems for elderly populations and proposes solutions to bridge the digital divide, ensuring accessible healthcare for seniors in the context of rapid technological advancements.MethodsThis study employed a mixed-methods approach, combining a review of national policies and global practices with an in-depth case study at the Longhu Ecological Senior Apartment in Bengbu City. Quantitative data on residents’ demographics, health status, and technology adoption were collected from institutional records. Qualitative insights were obtained through semi-structured interviews with administrative staff and caregivers. Data were analyzed using descriptive statistics and thematic analysis.ResultsMore than 95% of the residents suffered from chronic illnesses, and 73.30% were aged 80 years or above. Despite the availability of smart medical devices, 48.80% of residents reported difficulty in using digital tools due to insufficient technological literacy, and 46.30% were unaware of product functions. Economic burden and safety concerns further hindered acceptance. The gap between the rapid growth of digital health services and seniors’ limited ability to engage created a pronounced digital divide.ConclusionTo cope with the growing elderly population, digital healthcare must be further developed and tailored to meet the specific needs of the elderly. Bridging the digital divide through user-friendly technologies and policy support is crucial. The integration of 5G, telemedicine, and virtual reality holds great potential for improving elderly healthcare services, but a coordinated effort from the government, healthcare providers, and technology developers is necessary for sustainable and effective implementation.