PROOF-Lite running experience in High Energy Physics data analysis

With the development of the new generation of High Energy Physics (HEP) experiments, huge amounts of data are being generated. Efficient parallel algorithms/frameworks and High IO throughput are key to meet the scalability and performance requirements of HEP offline data analysis. Though Hadoop has gained a lot of attention from scientific community for its scalability and parallel computing framework for large data sets, it’s still difficult to make HEP data processing tasks run directly on Hadoop. In this paper we investigate the application of Hadoop to make HEP jobs run on it transparently. Particularly, we discuss a new mechanism to support HEP software to random access data in HDFS. Because HDFS is streaming data stored only supporting sequential write and append. It cannot satisfy HEP jobs to random access data. This new feature allows the Map/Reduce tasks to random read/write on the local file system on data nodes instead of using Hadoop data streaming interface. This makes HEP jobs run on Hadoop possible. We also develop diverse MapReduce model for HEP jobs such as Corsika simulation, ARGO detector simulation and Medea++ reconstruction. And we develop a toolkit for users to submit/query/remove jobs. In addition, we provide cluster monitoring and account system to benefit to the system availability. This work has been in production for HEP experiment to gain about 40,000 CPU hours per month since September, 2016.

Quantum generative modeling is emerging as a powerful tool for advancing data analysis in high-energy physics, where complex multivariate distributions are common. However, efficiently learning and sampling these distributions remains challenging. We propose a quantum protocol for a bivariate probabilistic model based on shifted Chebyshev polynomials, trained as a circuit-based representation of two correlated variables, with sampling performed via quantum Chebyshev transforms. As a key application, we study fragmentation functions (FFs) of charged pions and kaons from single-inclusive hadron production in electron-positron annihilation. We learn the joint distribution of momentum fraction z and energy scale Q, and infer their correlations from the entanglement structure. Building on the generalization capabilities of the quantum model and extended register architecture, we perform fine-grid multivariate sampling for FF dataset augmentation. Our results highlight the growing potential of quantum generative modeling to advance data analysis and scientific discovery in high-energy physics. Quantum computing holds great promise for advancing data analysis in science, including high-energy physics. This work presents a quantum protocol for learning and sampling multivariate distributions, using Chebyshev polynomials and quantum Chebyshev transforms to study fragmentation functions and energy-momentum correlations in hadron production.

In high-energy physics experiments, traditional data analysis methods increasingly show limitations when handling large-scale datasets, highlighting the urgent need for a transition to automated and intelligent analysis techniques to improve accuracy and efficiency. To address this challenge, we utilized an open dataset from CERN and proposed a novel Z-Bagging model, an ensemble learning approach designed to distinguish between two decay processes of the Z boson: Z [Formula: see text]+[Formula: see text] (Zmumu) and Z [Formula: see text] e+e[Formula: see text] (Zee). Initially, we tested several classical machine learning models to evaluate their performance in distinguishing these two decay modes. While these models showed some improvements in accuracy, further optimization was identified. Consequently, we introduced the Z-Bagging model, which combines multiple weak learners, such as Random Forest, Support Vector Machine and K-Nearest Neighbors, into a single strong learner. By integrating diverse models, Z-Bagging achieves more accurate and stable predictions. Experimental results demonstrate that the Z-Bagging model significantly outperforms existing models in classification accuracy for distinguishing Zmumu and Zee decay processes. This improvement provides a more precise tool for data analysis in high-energy physics.

Data processing in high energy physics experiments is a multi-tiered process in which raw detector signals are first processed locally into physics objects, and then collated into event records which can be scrutinized by a fast online trigger system. The resulting selection of events are reconstructed and pass through a number of software filters before arriving at a final offline analysis where hard physical constants are extracted. Although sophisticated statistical data analysis techniques are routinely employed high energy physics, the use of statistical signal processing in the field is has until now been rare. Our paper will begin with an overview of a typical high energy physics data acquisition system, outlining the technologies and tradeoffs involved at each stage. We will then proceed to argue that the dominant roles of model dependence and systematic errors in final physics analyses render statistical signal processing techniques largely inapplicable at this level. We observe, however, that at the low-level pattern recognition and event reconstruction levels, statistical signal processing techniques have been making inroads in high energy physics for a number of years, and examples from the literature will be cited. The viability of the technique for second level triggers will be assessed. Parallels to other other approaches, such as neural networks, will also be drawn. It will be argued that the falling cost of computing hardware favors the growth of statistical signal processing methods in high energy physics

Interest in many-core architectures applied to real time selections is growing in High Energy Physics (HEP) experiments. In this paper we describe performance measurements of many-core devices when applied to a typical HEP online task: the selection of events based on the trajectories of charged particles. We use as benchmark a scaled-up version of the algorithm used at CDF experiment at Tevatron for online track reconstruction - the SVT algorithm - as a realistic test-case for low-latency trigger systems using new computing architectures for LHC experiment. We examine the complexity/performance trade-off in porting existing serial algorithms to many-core devices. We measure performance of different architectures (Intel Xeon Phi and AMD GPUs, in addition to NVidia GPUs) and different software environments (OpenCL, in addition to NVidia CUDA). Measurements of both data processing and data transfer latency are shown, considering different I/O strategies to/from the many-core devices.

In High Energy Physics (HEP) experiments, it is useful for physics analysis and outreach if the event display software can provide sophisticated visualization effects. Unity is professional software that provides 3D modeling and animation production. GDML format files are commonly used for detector descriptions in HEP experiments. In this work, we present a method for automating the import of GDML files to the Unity engine modeling through geometric transformation. With the conversion of the file format, the complicated geometric description of the HEP detector in a GDML file can be imported directly into the Unity engine using an FBX file. This format conversion was applied to BESIII experiments to achieve an automated geometry conversion to build geometry models in the Unity engine.

This paper presents a quantum convolutional neural network (QCNN) for the classification of high energy physics events. The proposed model is tested using a simulated dataset from the Deep Underground Neutrino Experiment. The proposed quantum architecture demonstrates an advantage of learning faster than the classical convolutional neural networks (CNNs) under a similar number of parameters. In addition to the faster convergence, the QCNN achieves a greater test accuracy compared to CNNs. Based on our results from numerical simulations, it is a promising direction to apply QCNN and other quantum machine learning models to high energy physics and other scientific fields.

The high-precision measurements of detectors in a High-Energy Physics (HEP) experiment need a continuous sampling of recorded events during collisions. Therefore, significant hardware changes are required to do online data processing due to a large amount of data generated by such detectors. Because of all these changes, a new Detector Control System (DCS) design is required. This paper presents a definition of the software requirements to be considered during the design, integration, and operation of a detector’s DCS into physics data-taking, for continuous and non-continuous measurement conditions in the experiments. For this, the main operating processes, elements, characteristics, and guidelines of the DCS in the most important HEP experiments around the world were analyzed. Additionally, characteristics, functional and non-functional requirements, and use-cases of the main actors involved in the different processes of this control system software are defined. A visual modeling and design tool based on Unified Modeling Language (UML) is used to obtain a description of these requirements.

We attempted to study gamma-ray attenuation and sensing properties of conventional and modern inorganic scintillators being employed in the high energy physics (HEP) experiments. The mass attenuation coefficient (μm ), effective atomic number (Zeff ) and mean free path (mfp)were theoretically evaluated for the conventional scintillators materials such as CsI and NaI (Tl) and compared with advanced scintillator materials: PWO, PbF2, and BGO along with rare earth elements based scintillators such as LYSO:Ce, LuAG:Ce, BaF2:Y which have been proposed for applications in the future HEP experiments. Thegamma-ray attenuation parameters are analyzed within the framework of online software toolkit ‘py-MULBF’ over wide photon energy range from 0.015 MeV to 15 MeV.Variationof μm (and, Zeff ) with photon energy follows a trend similar for most of the inorganic scintillator materialsinvestigated here.CsI, however, maintained almost same effective atomic number value with respect to photon energy which signifies that CsI may be suitable for specific gamma-ray detection and sensing applications. Lead-based scintillator materials such as PbF2, PWO along with high-Z BGO are observed to exhibit better radiation attenuation capabilities.

High energy physics (HEP) experiments are facing unprecedented challenges to deal with huge volumes of data. Grid is the excellent technology for it. However, Incompatibility of a variety of grid middewares is an obstacle to exploit the resources on multiple grid infrastructures. This paper firstly introduces HEP experiments and HEP grid, and then reviews the solutions for grid interoperability and interoperations. Because current HEP applications are tightly with gLite middleware, this paper highlights a gateway approach to interoperate with gLite and described it in detail. Direct access to gLite storage system is very important, and a dedicated tool for HEP data sharing is then proposed and implemented. Finally, the implementation of the interoperability of GOS and gLite is introduced.

One burden of high energy physics data analysis is uncertainty within the measurement, both systematically and statistically. Even with sophisticated neural network techniques that are used to assist in high energy physics measurements, the resulting measurement may suffer from both types of uncertainties. Fortunately, most types of systematic uncertainties are based on knowledge from information such as theoretical assumptions, for which the range and behaviour are known. It has been proposed to mitigate such systematic uncertainties by using a new type of neural network called adversarial neural network (ANN) that would make the discriminator less sensitive to these uncertainties, but this has not yet been demonstrated in a real-life LHC analysis. This work investigates ANNs using as a benchmark the search for the production of four top quarks, an extremely rare physics process at the LHC and one of the important processes that can prove or disprove the Standard Model. The search for four top quarks in some cases is sensitive to large systematic uncertainties. The expected cross section upper limit for four top quark production is calculated using traditional neural networks and adversarial neural networks based on simulated proton-proton collisions within the Compact Muon Solenoid detector within Large Hadron Collider, and are compared to existing results. The improvement and further considerations to the search for rare processes at the LHC will be discussed.

In high-energy physics, particles produced in collision events decay in a format of a hierarchical tree structure, where only the final decay products can be observed using detectors. However, the large combinatorial space of possible tree structures makes it challenging to recover the actual decay process given a set of final particles. To better analyse the hierarchical tree structure, we propose a graph-based deep learning model to infer the tree structure to reconstruct collision events. In particular, we use a compact matrix representation termed as lowest common ancestor generations matrix, to encode the particle decay tree structure. Then, we introduce a perturbative augmentation technique applied to node features, aiming to mimic experimental uncertainties and increase data diversity. We further propose a supervised graph contrastive learning algorithm to utilize the information of inter-particle relations from multiple decay processes. Extensive experiments show that our proposed supervised graph contrastive learning with perturbative augmentation method outperforms state-of-the-art baseline models on an existing physics-based dataset, significantly improving the reconstruction accuracy. This method provides a more effective training strategy for models with the same parameters and makes way for more accurate and efficient high-energy particle physics data analysis.

The results of analysis using correlative and multivariate methods, as developed for data analysis in high-energy physics and implemented in the Toolkit for Multivariate Analysis software package, of the relations of the variation of increased radon concentration with climate variables in shallow underground laboratory is presented. Multivariate regression analysis identified a number of multivariate methods which can give a good evaluation of increased radon concentrations based on climate variables. The use of the multivariate regression methods will enable the investigation of the relations of specific climate variable with increased radon concentrations by analysis of regression methods resulting in 'mapped' underlying functional behaviour of radon concentrations depending on a wide spectrum of climate variables.

The availability of low-power, radiation-resistant components has an enormous importance in the development of the electronic systems for modern detectors in a High Energy Physics (HEP) experiment. This paper describes the characterization in terms of radiation effects of two serializer blocks within a high speed transmitter, prior developed with the objective of achieving a power consumption of less than 30 mW at the operating speed of 4.8 Gbit/sec. Within the first serializer, called “simple TMR”, a traditional solution, based on the hardware redundancy, has been implemented. In the second case a new architecture, less power consuming, called “code protected”, has been proposed. The tests previously performed shown an average consumption of ∼30 mW and ∼19 mW, respectively, for a bit rate of 4.8 Gbit/sec but do not fully clarify if the blocks are suitable for working under extremely high radiation levels. Hence, a deep radiation hardness investigation has been performed and presented here to confirm the availability of these blocks in a HEP electronic system. SEU sensitivities are measured and bit error rates better than 2 E-15 are obtained, confirming that the “code protected” solution assures reliable communications in HEP experiments environment with a smaller power consumption. These blocks have also been designed and tested to cope with a total ionizing dose of 100 Mrad over 10 years of operation.

Development of FPGA based phase alignment logic for the high speed protocol in HEP Experiments