Yandex School of Data Analysis

Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale. Still, using these models requires high-end hardware unavailable to many researchers. In some cases, LLMs can be used more affordably via RAM offloading or hosted APIs. However, these techniques have innate limitations: offloading is too slow for interactive inference, while APIs are not flexible enough for research that requires access to weights, attention or logits. In this work, we propose PETALS -a system for inference and fine-tuning of large models collaboratively by joining the resources of multiple parties. We demonstrate that this strategy outperforms offloading for very large models, running inference of BLOOM-176B on consumer GPUs with ≈ 1 step per second, which is enough for many interactive LLM applications. Unlike most inference APIs, PETALS also natively exposes hidden states of served models, allowing to train and share custom model extensions based on efficient fine-tuning methods. The system, its source code, and documentation are available at https://petals.ml.

Portfolio construction is an important practical problem in finance. In the traditional approach, introduced by Markowitz, one assumes normally distributed returns and constructs a portfolio with a minimum risk (measured by the standard deviation of portfolio returns) for a specified (and minimally acceptable) return.In practice, returns are not normally distributed and have heavy tails. As a result, the normality assumption severely underestimates risk. It has been long suggested that a more appropriate way is to model returns by using alpha-stable distributions.In this paper, we use elliptical stable distributions for optimal stable portfolio construction. We illustrate this by considering portfolios from S&P 500 sector exchange-traded funds (ETFs).Our main results indicate that stable portfolios are in general comparable with standard Markowitz portfolios. But we discovered a few properties of stable distributions that are promising for optimal portfolio selection problem. Stable risk estimation for next year is much closer to the actual portfolio risk, stable portfolios are more balanced, so they better handle maximum restriction on component weights, and stable portfolios are more resilient to extreme market conditions.Both stable and normal optimal portfolios outperform S&P 500 index and equally-weighted ETF portfolio in most years.We propose an investment strategy that uses both stable and normal distributions for building optimal portfolios. The return of this strategy exceeds the return of the strategies which use only stable or only normal distributions.

The centrality of heavy-ion collisions is directly related to the created medium in these interactions. A procedure to determine the centrality of collisions with the LHCb detector is implemented for lead-lead collisions at √s NN = 5 TeV and lead-neon fixed-target collisions at √s NN = 69 GeV. The energy deposits in the electromagnetic calorimeter are used to determine and define the centrality classes. The correspondence between the number of participants and the centrality for the lead-lead collisions is in good agreement with the correspondence found in other experiments, and the centrality measurements for the lead-neon collisions presented here are performed for the first time in fixed-target collisions at the LHC.

The optimization of big industrial setups and the accompanying detailed simulations of internal physical processes require complex and time-consuming simulation calculations. We propose a versatile approach that can alleviate difficulties in solving this problem and show this using an example of electromagnetic calorimeter optimization at a Large Hadron Collider experiment. Our approach consists of a block representation of the calorimeter optimization process from setting sensitive characteristics of modules and their locations to obtaining a quality metric and applying machine learning methods. The main blocks are signal and background particles generation and their propagation to the calorimeter, the generation of electromagnetic showers of signal and noise in modules with a given technology, the combination of signal and noise with the simulation of different luminosities, the energy and spatial reconstruction of the signal and obtaining the final metric. This approach allows us to evaluate the operational characteristics of the calorimeter and find a more suitable configuration with the necessary quality without extensive use of time-consuming resources.

Advanced detector R&D for both new and ongoing experiments in HEP requires performing computationally intensive and detailed simulations as part of the detector-design optimisation process. We propose a versatile approach to this task that is based on machine learning and can substitute the most computationally intensive steps of the process while retaining the GEANT4 accuracy to details. The approach covers entire detector representation from the event generation to the evaluation of the physics performance. The approach allows the use of arbitrary modules arrangement, different signal and background conditions, tunable reconstruction algorithms, and desired physics performance metrics. While combined with properties of detector and electronics prototypes obtained from beam tests, the approach becomes even more versatile. We focus on the Phase II Upgrade of the LHCb Calorimeter under the requirements on operation at high luminosity. We discuss the general design of the approach and particular estimations, including spatial and energy resolution for the future LHCb Calorimeter setup at different pile-up conditions.

Using proton-proton collision data, collected with the LHCb detector and corresponding to 1.0, 2.0 and 1.9 fb−1 of integrated luminosity at the centre-of-mass energies of 7, 8, and 13 TeV, respectively, the decay {Lambda}_{mathrm{b}}^0to {upchi}_{mathrm{c}1} (3872)pK− with χc1(3872) → J/ψ π+π− is observed for the first time. The significance of the observed signal is in excess of seven standard deviations. It is found that (58 ± 15)% of the decays proceed via the two-body intermediate state χc1(3872)Λ(1520). The branching fraction with respect to that of the {Lambda}_{mathrm{b}}^0 → ψ(2S)pK− decay mode, where the ψ(2S) meson is reconstructed in the J/ψ π+π− final state, is measured to be:βΛb0→χc13872pK−βΛb0→ψ2SpK−×βχc13872→J/ψπ+π−βψ2S→J/ψπ+π−=5.4±1.1±0.2×10−2,\\documentclass[12pt]{minimal}\t\t\t\t\\usepackage{amsmath}\t\t\t\t\\usepackage{wasysym}\t\t\t\t\\usepackage{amsfonts}\t\t\t\t\\usepackage{amssymb}\t\t\t\t\\usepackage{amsbsy}\t\t\t\t\\usepackage{mathrsfs}\t\t\t\t\\usepackage{upgreek}\t\t\t\t\\setlength{\\oddsidemargin}{-69pt}\t\t\t\t\\begin{document}$$ \\frac{\\beta \\left({\\Lambda}_{\\mathrm{b}}^0\\to {\\upchi}_{\\mathrm{c}1}(3872){\\mathrm{pK}}^{-}\\right)}{\\beta \\left({\\Lambda}_{\\mathrm{b}}^0\\to \\uppsi \\left(2\\mathrm{S}\\right){\\mathrm{pK}}^{-}\\right)}\\times \\frac{\\beta \\left({\\upchi}_{\\mathrm{c}1}(3872)\\to \\mathrm{J}/\\uppsi {\\uppi}^{+}{\\uppi}^{-}\\right)}{\\beta \\left(\\uppsi \\left(2\\mathrm{S}\\right)\\to \\mathrm{J}/\\uppsi {\\uppi}^{+}{\\uppi}^{-}\\right)}=\\left(5.4\\pm 1.1\\pm 0.2\\right)\\times {10}^{-2}, $$\\end{document}where the first uncertainty is statistical and the second is systematic.

The Search for Hidden Particles (SHiP) Collaboration has shown that the CERN SPS accelerator with its 400 GeV/c proton beam offers a unique opportunity to explore the Hidden Sector [1–3]. The proposed experiment is an intensity frontier experiment which is capable of searching for hidden particles through both visible decays and through scattering signatures from recoil of electrons or nuclei. The high-intensity experimental facility developed by the SHiP Collaboration is based on a number of key features and developments which provide the possibility of probing a large part of the parameter space for a wide range of models with light long-lived super-weakly interacting particles with masses up to \U0001d4aa(10) GeV/c2 in an environment of extremely clean background conditions. This paper describes the proposal for the experimental facility together with the most important feasibility studies. The paper focuses on the challenging new ideas behind the beam extraction and beam delivery, the proton beam dump, and the suppression of beam-induced background.

We report the measurements of the CP -violating parameters in Bs0 → Ds∓K± decays observed in pp collisions, using a data set corresponding to an integrated luminosity of 3.0 fb−1 recorded with the LHCb detector. We measure Cf = 0.73 ± 0.14 ± 0.05, AfΔΓ = 0.39 ± 0.28 ± 0.15, {A}_{overline{f}}^{varDelta varGamma }=0.31pm 0.28pm 0.15 , Sf = −0.52 ± 0.20 ± 0.07, {S}_{overline{f}}=-0.49pm 0.20pm 0.07 , where the uncertainties are statistical and systematic, respectively. These parameters are used together with the world-average value of the Bs0 mixing phase, −2βs, to obtain a measurement of the CKM angle γ from Bs0 → Ds∓K± decays, yielding γ = (128− 22+ 17)° modulo 180°, where the uncertainty contains both statistical and systematic contributions. This corresponds to 3.8 σ evidence for CP violation in the interference between decay and decay after mixing.

A search is performed in the invariant mass spectrum of the Bc+π+π− system for the excited Bc+ states Bc(21S0)+ and Bc(23S1)+ using a data sample of pp collisions collected by the LHCb experiment at the centre-of-mass energy of sqrt{s}=8 TeV, corresponding to an integrated luminosity of 2 fb−1. No evidence is seen for either state. Upper limits on the ratios of the production cross-sections of the Bc(21S0)+ and Bc(23S1)+ states times the branching fractions of Bc(21S0)+ → Bc+π+π− and Bc(23S1)+ → Bc*+π+π− over the production cross-section of the Bc+ state are given as a function of their masses. They are found to be between 0.02 and 0.14 at 95% confidence level for Bc(21S0)+ and Bc(23S1)+ in the mass ranges [6830, 6890] MeV/c2 and [6795, 6890] MeV/c2, respectively.