MURS: Mitigating Memory Pressure in Service-Oriented Data Processing System

This experience report describes the successful application of statistical software testing to a large-scale realworld equipment replacement project. The Data Processing System at Dungeness B nuclear power station is obsolescent and needs to be replaced by a new system involving different technology. The new Data Processing and Control System (DPCS) involves a large amount of software code, with the potential for software failures, and therefore confidence needs to be established in the dependability of the new software-based system. Statistical software testing (ST) was used as one technique to do this. The aim was to support the claim that with high confidence the higher integrity parts of the replacement software have a failure probability of better than 10-2 failures per demand (fpd). This paper describes how this has been achieved by careful design of the statistical tests and by the development of a large number of supporting tools and procedures. A total of 395 statistical tests with no failure was achieved providing at least 98% confidence that the safety claim has been met.

The ASTRI Mini-Array is an international project led by the Italian National Institute for Astrophysics (INAF) to build and operate an array of nine 4-m class Imaging Atmospheric Cherenkov Telescopes (IACTs) at the Observatorio del Teide (Tenerife, Spain). The system is designed to perform deep observations of the galactic and extragalactic gamma-ray sky in the TeV and multi-TeV energy band, with important synergies with other ground-based gamma-ray facilities in the Northern Hemisphere and space-borne telescopes. As part of the overall software system, the ASTRI (Astrofisica con Specchi a Tecnologia Replicante Italiana) Team is developing dedicated systems for Data Processing, Simulation, and Archive to achieve effective handling, dissemination, and scientific exploitation of the ASTRI Mini-Array data. Thanks to the high-speed network connection available between Canary Islands and Italy, data acquired on-site will be delivered to the ASTRI Data Center in Rome immediately after acquisition. The raw data will be then reduced and analyzed by the Data Processing System up to the generation of the final scientific products. Detailed Monte Carlo simulated data will be produced by the Simulation System and exploited in several data processing steps in order to achieve precise reconstruction of the physical characteristics of the detected gamma rays and to reject the overwhelming background due to charged cosmic rays. The data access at different user levels and for different use cases, each one with a customized data organization, will be provided by the Archive System. In this contribution we present these three ASTRI Mini-Array software systems, focusing on their main functionalities, components, and interfaces.

The article describes the development of an architecture for cloud storage of melons based on a Residue Number System (RNS). The developed architecture is evaluated in terms of the reliability of the stored data, and its comparative analysis is carried out with existing data storage methods. The architecture of a multi-cloud big data storage and processing system based on the principles of modular arithmetic is proposed. The proposed modification of the cloud storage and data processing system allows increasing the reliability and fault tolerance of the stored and processed data. Single-level and two-level models for reliable storage of big data were developed based on the simulation results. The comparative analysis showed the advantage of models based on modular arithmetic over models based on traditional arithmetic.

We consider batch processing jobs to minimize the mean completion time. A batch processing machine can handle up to $B$ jobs simultaneously. Each job is represented by an arrival time and a processing time. Jobs processed in a batch have the same completion time, i.e., their common starting time plus the processing time of their longest job. For batch processing, non-preemptive scheduling is usually required and we discuss this case. The batch processing problem reduces to the ordinary uniprocessor system scheduling problem if $B=1$. We focus on the other extreme case $B=+\infty$. Even for this seemingly simple extreme case, we are able to show that the problem is NP-hard for the weighted version. In addition, we establish a polynomial time algorithm for a special case when there are only a constant number of job processing times. Finally, we give a polynomial time approximation scheme for the general case.

Rapid economic development causes increasingly fierce business competition. Moreover, the development of increasingly sophisticated technology, companies are competing to use technology to increase competitive advantage and efficiency in the company. One of the most popular information technology investments today is the Application of Systems and Products in Data Processing (SAP), systems that do business using a single database that can be accessed by all divisions within the company. The purpose of this research is to study and discuss the application of Systems and Products in Data Processing (SAP) conducted at PT. Haier Sales Indonesia. This study uses qualitative methods through in-depth interviews with IT Managers and Logistics Managers and warehouse administration staff using Pareto diagrams and fishbone diagrams. The results of the evaluation of the application of Application Systems and Products in Data Processing (SAP) in Inventory Logistics that occurred at PT. Haier Sales Indonesia is getting better, all data from every activity has been documented and integrated into the system much faster than the manual process that has been done before and can produce accurate reports. Suggestions are given in order to continue training in SAP Systems and Applications for users in developing SAP Systems and Applications.

The functioning of modern information and communication networks is impossible without data processing. With the emergence of new network services, the amount of information that needs to be processed increases, while the requirements to the data processing quality become more and more stringent. Therefore, the problem of designing and maintaining a scalable data processing system with a flexible quality of service management is becoming more and more important for a network operator. Such data processing systems have a complex internal structure with many interrelated parameters, which makes them difficult to analyze, manage, and expand. This study proposes to use an ontological model to store, represent, and manipulate the information in the operator’s data processing system. The ontological model allows to structure and systematize the data of an information processing system, and transparently reflects the relationships between the parameters of the system to simplify its analysis and scaling. The proposed ontology of a data processing system consists of three related subsystems. The paper describes the proposed ontological model and additionally analyzes the sources of information that needs to be processed in the information and communication network.

The expected development of grid monitoring applications and control functionalities in low voltage distribution networks calls for advanced simulation tools for performance assessment of these applications. Power hardware-in-the-loop simulations form a cost effective approach to accurately capture the (dynamic) system behaviour. This paper describes the development of a data acquisition and processing system that enables power hardware-in-the-loop simulation of a real physical feeder within a larger simulated three-phase low voltage network simulated in real-time. With the aid of the data acquisition and processing system, the performance of newly developed monitoring and control applications can be assessed and evaluated in terms of accuracy, speed and effectiveness. To this extend, the paper shows how the data acquisition and processing system is implemented and discusses the possibilities that it creates for verifying newly developed applications for distribution grid automation.

This computerized system, which collects, processes, and stores data on and analyzes and reports the performance of production equipment and operations, is one step closer to the optimum profitability of producing properties. It is designed primarily to be used by the production man properties. It is designed primarily to be used by the production man directly responsible for day-to-day economic decisions. Introduction In any attempt at optimizing operations by means, for example, of a failure-control program, basic information is required for defining the nature and magnitude of the problems involved and for estimating the economic stakes. The information determines whether or not one should try to solve a particular problem; and if more than one solution is possible, it aids in deciding on the most profitable one. In short, the basic information makes it possible to set economic priorities to the solution of problems. Methods of obtaining basic information related to equipment performance have been the subject of much work. Many oil-producing companies have created and now operate data gathering and processing systems, all of which have two things in common: they obtain accurate and timely information and process and condense it into an easily usable and process and condense it into an easily usable and digestible form; and they use computers to process and report data. Atlantic Richfield has created a system for data gathering and handling that has been in use throughout its Continental U. S. and Canadian operations since late 1969. This system, known as the Equipment Performance Reporting System, relies heavily on the use of computers. It was developed as a cooperative effort with field personnel to the point that it is now part of the Company procedures. By late 1971 the system had more than 34,000 records in its data banks. Since its inception, the original concept of a data gathering and processing system for use by production personnel has been expanded according to needs production personnel has been expanded according to needs discovered as a result of using the system. These needs are: (1) to set better specifications for subsurface and surface equipment, (2) to predict future performance of equipment, and (3) to follow closely performance of equipment, and (3) to follow closely the economic impact of remedial action such as area-wide corrosion treating programs. The Equipment Performance Report System-Part One The Equipment Performance Report System consists of four parts. The following discussion will be limited to Part One, which is concerned with subsurface failures, and to Part Three, which handles well inhibition. Parts Two and Four will be described later. As early as 1964, Fincher and Nunn defined a usable data gathering and processing system as one that contains1. A simple report form,2. Complete data about each reported event,3. Computer processing for rapid data handling, and4. Output in the form of orderly and concise data summaries for easy review. JPT p. 185

Information technology in intellectual property law – Problem solving or window dressing?

In the big data processing systems, the amount of data is increasing. At the same time, the real-time requirement of data processing and analysis is higher and higher. Therefore, it is required that the big data processing and analysis systems have better performance. Job scheduling plays an important role in improving the overall system performance in big data processing frameworks. However, job scheduling is a difficult NP-hard problem. There are many factors that need to be considered for job scheduling. For example, jobs have dependencies among stages, therefore we should not allocate resources to tasks that are not ready. Sometimes, there are constraints between jobs. These are a challenge to the scheduling performance of big data processing and analysis systems. In this paper, we try to solve the problem by translating it into Boolean Satisfiability Problem (SAT) which is an exact method. SAT-based scheduling algorithm is not a new approach, but in the past it mainly used to solve the static scheduling problems. For dynamic scheduling system, it requires all problems to be solved within a limited time, which is a challenge for SAT encoding. In this paper, we refer to the previous SAT solution to the Job Shop Scheduling Problem, and adjust the algorithm to meet the requirements of the big data processing system. At the same time, we optimized the coding approach and reduced the number of clauses. Thus, the efficiency of the problem solved is improved to meet the performance requirements. The experimental results show that the number of clauses is reduced by more than 30%, and the processing time of the SAT solver to get the solution can be reduced by more than 50%. To demonstrate its effectiveness, we have also implemented our new job scheduler in Apache Hadoop YARN, and validated its effectiveness

In this article we describe the solution to make an estimate of allowed alternations in model components parameters that make up a data processing and control system. They reflect properties of physical and information components of the aforementioned system. This solution is derived according to the limitations of possible changes in integral property that describes the system behavior in different modes of operation. The proposed solution makes it possible to solve not only the direct problem - making a conclusion whether the vulnerability of the data processing system is negligible with respect to alternations in parameters of this systems components but the inverse as well, finding tolerable levels in alterations in systems components parameters from an acceptable level of uncertainty of targeted efficiency. The proposed solution follows known demands: inner properties of the system must be so that customer demands are fulfilled.

Large-scale data processing systems frequently require users to make timely and high-value business decisions based upon information that is received from a variety of heterogeneous sources. Such heterogeneity is especially true of service-oriented systems, which are often dynamic in nature and composed of multiple interacting services. However, in order to establish user trust in such systems, there is a need to determine the validity and reliability of all the data sources that go into the making of a decision. This paper analyses the concept of provenance and discusses how the establishment of personalized provenance recording and retrieval systems can be used to increase the utility of data and engender user trust in complex service-based systems. An overview of current provenance research is presented, and a real-world project to address the abstract concepts of trust and data quality in industrial and clinical settings is presented. From this, we conclude that the addition of provenance into data processing and decision making systems can have a tangible benefit to improving the trust of system users.

BACKGROUND:Biomedical engineers in particular require fast and powerful data processing systems to process computerized tomography and magnetic resonance imaging scans and other medical imaging technologies. However, current computer data processing technologies are unable to satisfy such requirements. A promising approach to addressing these limitations is processing in memory (PIM). Unfortunately, several issues, such as the compatibility and interconnection of PIM with legacy systems, still remain.OBJECTIVE:This paper proposes a standard memory bus-based PIM interface for medical image processing and a PIM platform. The proposed PIM interface can overcome problems of compatibility with legacy systems.METHODS:We will adapt an embedded system based on a commercial application processor (AP) to a medical image system to verify the functions and the performance of the proposed PIM interface. Using the PIM platform, we apply the proposed PIM interface and the AP to execute an image processing program, measure the image processing times, and compare the results of the measurements.RESULTS: Experimental results show that while the functions of the proposed PIM interface are normal, the processing time of PIM is more than 81% faster than that of the AP.CONCLUSION:The experimental results prove that the proposed PIM interface is able to solve problems of compatibility with legacy systems. We foresee that not only the medical image processing field but also a number of academic fields and professional sectors will use PIM in their data-intensive applications.

The paper presents Data processing and optimization system for studying and making rational decisions on the formation of interstate electric power interconnections, with aim to increasing effectiveness of their functioning and expansion. The technologies for building and integrating a Data processing and optimization system including an object-oriented database and a predictive mathematical model for optimizing the expansion of electric power systems ORIRES, are described. The technology of collection and pre-processing of non-structured data collected from various sources and its loading to the object-oriented database, as well as processing and presentation of information in the GIS system are described. One of the approaches of graphical visualization of the results of optimization model is considered on the example of calculating the option for expansion of the South Korean electric power grid.

Design Concepts of Japanese-Language Data Processing Systems