Memory footprint matters

Data analytics platforms today largely employ data processing kernels (e.g. implementation of selection and join operator algorithms) that were developed for a now bygone hardware era. Hardware has made fundamental shifts in recent years, driven by the need to consider energy as a first-class design parameter. Consequently, across the processor-IO hierarchy, the hardware paradigm today looks very different than it did just a few years ago. I argue that because of this shift, we are now building a 'deficit' between the pace at which the hardware is evolving and the pace that is demanded of data processing kernels to keep up with the growth of big data. This deficit is unsustainable in the long run. One way to 'pay off' this deficit is to have hardware and software co-evolve to exploit the full potential of the hardware. I will provide some examples of recent work from our Wisconsin Quickstep project that demonstrates the merit of this line of thinking. I'll focus on analytical data processing environments, and argue that our new way of storing data, called BitWeaving, and flattening databases into BitWeaved de-normalized tables, which we call WideTables, provides a dramatic new way to build 'sustainable' analytical data processing systems. I will also discuss the implications of our approach on future hardware-software co-design for data analytics platforms.

While the real-time processing applications are ubiquitous, which lead to the sprout of many distributed real-time stream processing systems. In this paper, we design a high-performance streamed-oriented big data processing system, including a messaging queue on Kafka and a Spark Streaming-based processing engine, and we evaluate the effectiveness of our system through a real-time traffic statistics application. Then we focus on the relationship between the parameters of Kafka and Spark Streaming and the performance of our system. In the last, we discuss the scalability of the optimized system of Kafka and Spark Streaming.

This experience report describes the successful application of statistical software testing to a large-scale realworld equipment replacement project. The Data Processing System at Dungeness B nuclear power station is obsolescent and needs to be replaced by a new system involving different technology. The new Data Processing and Control System (DPCS) involves a large amount of software code, with the potential for software failures, and therefore confidence needs to be established in the dependability of the new software-based system. Statistical software testing (ST) was used as one technique to do this. The aim was to support the claim that with high confidence the higher integrity parts of the replacement software have a failure probability of better than 10-2 failures per demand (fpd). This paper describes how this has been achieved by careful design of the statistical tests and by the development of a large number of supporting tools and procedures. A total of 395 statistical tests with no failure was achieved providing at least 98% confidence that the safety claim has been met.

Permanent change of methods of analytical processing of digital data for objective analysis and adequate decisions based on available information. In the educational process, analytics of significant amounts of data is used to adequately study and understand the data that are continuously generated in the educational sphere for further in-depth analysis to improve management methods that ensure a sufficient level of qualification of students of higher educational institutions. For this purpose, an automated information and reference system of higher education institution management was developed and tested, and on its basis, analytical analysis of the accumulated data flows was carried out. Since analytics is a painstaking process of determining, systematic analysis and interpreting particularly significant patterns from large continuous streams and volumes of data. Analytical data processing is based on innovative methods of intellectual analysis of large amounts of data, such as association, clustering, classification, categorization, correlation, evaluation, forecasting, and analysis in light of current trends and visualization. The purpose of this work is to use predictive analytics to improve the quality of the educational process by providing objective consolidated information to guide higher education institutions in making informed management decisions. This research work mainly focuses on the urgent need to introduce methods of analytical processing of large data flows into the educational management system of the educational institution, and the following strategies are proposed to meet such needs. When implementing automated information and reference system of educational process management, it is necessary to use various components and their functions. In the information and reference system of educational process management, there are various components, such as electronic learning tools, electronic means of providing access to educational content, the use of information technology, various services, etc. This document also focuses on key issues related to the basic components and their functionality in the studied system for processing both dynamic and analytical data. The effective use and processing of analytical educational data by different methods have a significant potential for identifying and forecasting significant accumulated knowledge from educational data, which provides the education management system to recommend appropriate solutions for a flexible planning process that will ensure the quality of educational services for the future.KeywordsEducationInformation systemInformation technologyAnalytical data processingEducational management

The article analyses main principles and mechanism of international environmental policy of the different economic levels. Particularly is evaluated the role of United Nations, European Union and national states in framework of sustainable concept of economic development and Millennium development goals. Are used market-based instruments such as taxes, subsidies and certificates, quantitative methods of regulation; government spending on public goods, environmental investments (e.g. water pollution) and planning environmental education etc. Among the institutional instruments of the international environmental policy are first of all three types of international agreements depending of deploying of situation in global environment. The focus of main environmental problems, according to the OECD / Eurostat classification, include air pollution control, water management, solid waste management, soil remediation and purification, noise reduction and vibration, environmental research and development, environmental commitments and engineering, analytical services, data processing, analysis and evaluation, education, training, and information.

Fast arithmetic circuits are key elements of high performance computers and data processing systems. In the majority of these applications, multipliers have been a critical and obligatory component in dictating the overall circuit performance when constrained by power consumption and computation speed. Compressors are a critical component of the multiplier circuit, which greatly influence the overall multiplier speed. The authors propose two novel high performance 5 : 2 compressor architectures. The main objective of their designs is to limit the carry propagation to a single stage, thereby reducing the overall propagation delay. The designs are compared with the best one in the literature in terms of delay and are found to have lower values. The analytical techniques use the node capacitances in the signal delay paths to identify the worst delay path. The architectures are implemented with various XOR–XNOR circuits to identify the best one in terms of power and delay. The simulation results of the proposed architectures show lower power and 25% improvement in speed compared to the best architecture reported in the literature for supply voltages ranging from 1.5 V to 3.3 V.

The ASTRI Mini-Array is an international project led by the Italian National Institute for Astrophysics (INAF) to build and operate an array of nine 4-m class Imaging Atmospheric Cherenkov Telescopes (IACTs) at the Observatorio del Teide (Tenerife, Spain). The system is designed to perform deep observations of the galactic and extragalactic gamma-ray sky in the TeV and multi-TeV energy band, with important synergies with other ground-based gamma-ray facilities in the Northern Hemisphere and space-borne telescopes. As part of the overall software system, the ASTRI (Astrofisica con Specchi a Tecnologia Replicante Italiana) Team is developing dedicated systems for Data Processing, Simulation, and Archive to achieve effective handling, dissemination, and scientific exploitation of the ASTRI Mini-Array data. Thanks to the high-speed network connection available between Canary Islands and Italy, data acquired on-site will be delivered to the ASTRI Data Center in Rome immediately after acquisition. The raw data will be then reduced and analyzed by the Data Processing System up to the generation of the final scientific products. Detailed Monte Carlo simulated data will be produced by the Simulation System and exploited in several data processing steps in order to achieve precise reconstruction of the physical characteristics of the detected gamma rays and to reject the overwhelming background due to charged cosmic rays. The data access at different user levels and for different use cases, each one with a customized data organization, will be provided by the Archive System. In this contribution we present these three ASTRI Mini-Array software systems, focusing on their main functionalities, components, and interfaces.

The article analyzes the results of a decade-long experience in implementing performance-based ranking of academic staff at Ternopil Volodymyr Hnatiuk National Pedagogical University. The ranking procedure is examined as a component of the university's internal quality assurance system, enabling the tracking of changes in professional activity, identification of strengths, and outlining areas for improving the institution’s human resources. The study highlights the stages of system development, its regulatory framework, trends in the evolution of evaluation criteria and indicators, as well as the outcomes that demonstrate increased engagement of academic staff in the evaluation process. Attention is given to the European experience in organizing quality assurance in higher education, particularly institutional and individual assessment mechanisms that incorporate elements of performance-based ranking. The article explores the implementation of an information system for ranking evaluation, which ensures the collection, storage, verification, analytical processing, and public visualization of data through an interactive dashboard that allows for dynamic performance monitoring. This approach promotes transparency, openness, academic integrity, and trust in the evaluation process. The paper presents trends in academic staff participation in ranking assessments over a ten-year period and summarizes the overall performance indicators. The study reveals that approaches to ranking-based evaluation across Ukrainian higher education institutions are characterized by a significant similarity in the criteria and indicators applied. This finding highlights the need to develop a unified national regulation on the ranking evaluation of academic staff, which would harmonize institutional approaches and ensure comparability of results. Prospects for further research are outlined, focusing on improving data collection and analysis methods, as well as examining the impact of ranking systems on the professional development of academic staff, taking into account best international and national practices.

With the rapid development of Internet of Things (IoT) technologies, the increasing volume and diversity of sources of geospatial big data have created challenges in storing, managing, and processing data. In addition to the general characteristics of big data, the unique properties of spatial data make the handling of geospatial big data even more complicated. To facilitate users implementing geospatial big data applications in a MapReduce framework, several big data processing systems have extended the original Hadoop to support spatial properties. Most of those platforms, however, have included spatial functionalities by embedding them as a form of plug-in. Although offering a convenient way to add new features to an existing system, the plug-in has several limitations. In particular, while executing spatial and nonspatial operations by alternating between the existing system and the plug-in, additional read and write overheads have to be added to the workflow, significantly reducing performance efficiency. To address this issue, we have developed Marmot, a high-performance, geospatial big data processing system based on MapReduce. Marmot extends Hadoop at a low level to support seamless integration between spatial and nonspatial operations of a solid framework, allowing improved performance of geoprocessing workflow. This paper explains the overall architecture and data model of Marmot as well as the main algorithm for automatic construction of MapReduce jobs from a given spatial analysis task. To illustrate how Marmot transforms a sequence of operators for spatial analysis to map and reduce functions in a way to achieve better performance, this paper presents an example of spatial analysis retrieving the number of subway stations per city in Korea. This paper also experimentally demonstrates that Marmot generally outperforms SpatialHadoop, one of the top plug-in based spatial big data frameworks, particularly in dealing with complex and time-intensive queries involving spatial index.

The study of the typology of ulul albab as an ideal human character in Islam is an important aspect in understanding the concept of spirituality and wisdom in religion. In the Qur'an, ulul albab is described as an individual who reaches a level of perfection in his thinking activities, reflected in the depth of his spirituality and wisdom in living life. By considering a psychological perspective on human thinking activities, this study aims to explore the meaning and implications of the ulul albab typology in the context of human life. Through a descriptive qualitative approach and library research, data was collected from various relevant reading sources. The data analysis process uses a flowing data analysis method, including reading, recording, processing and concluding. The findings show that thinking involves the entire human personality, explained in the Qur'an. The integration of the Ulul Albab typology in Islam highlights the importance of thinking activities as a basis for devotion and wisdom. In the modern era, Al-Qur'an-based education emphasizing the development of the Ulul Albab character through thinking activities can be a relevant solution in forming an intelligent and devout generation. This study contributes to enriching understanding of thinking activities within a psychological framework and Islamic teachings.

This paper, taking a match of the province of north China as an example, based on the schools for the past ten years and awards for qualitative and quantitative data analysis and processing. Then through the model to predict the next year, and on the basis of the present situation and awards by the normal given the competition for the school places allocation scheme, in order to improve the quality and efficiency. This paper presents a method and process of big data analysis and processing by using SPSS and MATLAB and other mathematical software based on actual cases. The prediction results are reliable and can be used for reference. The innovation of this paper is that the weight ratio of analytic hierarchy process is used as the reference for quota allocation. Its advantage is that it can make the allocation result more flexible and easy to be controlled artificially. In addition, the paper adopts arithmetic average method, geometric average method and eigenvalue method respectively in the weight calculation, and then takes the average weight of the three methods to make the result more robust.

The daily growing volume of electronic data poses challenges to traditional ways of organizing data storage, processing, and analysis. The feasibility of further research in this area is confirmed by the high demand for data storage and analytical data processing services. The article is devoted to the study of big data security issues. Despite earlier technological capabilities, the term or Big data, has only been actively used in the last decade. This term, originally coming from the exact sciences, programming, has caused and continues to cause a lot of controversy: what exactly is meant by Big data. In the course of the work, the specifics of Big data were described, the main characteristics of Big data were highlighted, and modern approaches to big data protection were analyzed. The role of Big data in the development of the digital economy and the need to address issues of their legal regulation are outlined. The following problems of Big data protection were formulated: limiting the speed of data access, organizing access via network protocols through General-purpose networks. A review of known developers of protection methods in these areas and recommendations for their application is made. Problems of legal regulation of Big data are revealed. The complexity of the problem, the need for multi-level protection and the use of modern methodological, theoretical and software developments from the first steps of working with Big data are shown.

The functioning of modern information and communication networks is impossible without data processing. With the emergence of new network services, the amount of information that needs to be processed increases, while the requirements to the data processing quality become more and more stringent. Therefore, the problem of designing and maintaining a scalable data processing system with a flexible quality of service management is becoming more and more important for a network operator. Such data processing systems have a complex internal structure with many interrelated parameters, which makes them difficult to analyze, manage, and expand. This study proposes to use an ontological model to store, represent, and manipulate the information in the operator’s data processing system. The ontological model allows to structure and systematize the data of an information processing system, and transparently reflects the relationships between the parameters of the system to simplify its analysis and scaling. The proposed ontology of a data processing system consists of three related subsystems. The paper describes the proposed ontological model and additionally analyzes the sources of information that needs to be processed in the information and communication network.

The article describes the development of an architecture for cloud storage of melons based on a Residue Number System (RNS). The developed architecture is evaluated in terms of the reliability of the stored data, and its comparative analysis is carried out with existing data storage methods. The architecture of a multi-cloud big data storage and processing system based on the principles of modular arithmetic is proposed. The proposed modification of the cloud storage and data processing system allows increasing the reliability and fault tolerance of the stored and processed data. Single-level and two-level models for reliable storage of big data were developed based on the simulation results. The comparative analysis showed the advantage of models based on modular arithmetic over models based on traditional arithmetic.

With the ever more rapid development of information technology, computer and network infrastructures have become an indispensable part of modern life and work. In this evolving context, the importance of computer-assisted information analysis is increasingly evident, highlighting the need for a solid framework to support its further development. This paper attempts to establish a comprehensive, multi-dimensional technological framework to improve capacity in this crucial area. The proposed framework covers core technology areas such as databases, data warehouses, online transaction processing (OLTP), business intelligence (BI), online analytical processing (OLAP) and decision support systems (ss). By combining these fundamentals with cutting-edge technologies, including data extraction, social network analysis, and massive data analysis and advanced online analytical processing, the framework aims to create a holistic system for computer-aided information analysis. The system aims not only to optimize the efficiency and effectiveness of data processing and analysis, but also to promote innovation in the way information is used for decision-making and strategic planning, contributing to the continued development and impact of all industries.