Efficient Machine Learning Through Self-Supervised Learning: Methodologies and Applications

Spatiotemporal self-supervised predictive learning for atmospheric variable prediction via multi-group multi-attention

Globally, wind energy if properly harnessed, could serve as a source of energy generation in Africa. This study compared the performance of two Machine Learning (ML) algorithms (Linear regression and Random Forest) in predicting wind speed in five major cities in Africa (Yaoundé, Pretoria, Nairobi, Cairo and Abuja). Wind data were collected between January 1, 2000, and December 31, 2022, using the Solar Radiation Data Archive. The data preprocessing was carried out with 80% of the data used for training and 20% for validation. The performance of these ML algorithms was evaluated using Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and coefficient of determination (R2). The result shows that Nairobi (3.814795 m/s) closely followed by Cairo (3.606453 m/s) has the highest mean wind speed while Yaoundé (1.090512 m/s) has the lowest. Based on the performance metrics used, the two Machine Learning algorithms were competitive. Still, the Linear Regression (LR) algorithm outperformed the Random Forest Algorithm in predicting wind speed in all the selected major African cities. In Yaoundé (RMSE = 0.3892, MAE= 0.3001, MAPE =0.5030), Pretoria (RMSE=1.2339, MAE=0.9480, MAPE=0.7450) Nairobi (RMSE= 0.4223, MAE =0.6499, MAPE =0.1872), Nairobi (RMSE=0.6499, MAE=0.5171, MAPE =0.1872), Cairo (RMSE =1.0909, MAE =0.8544, MAPE =0.3541) and Abuja (RMSE = 0.70245, MAE =0.5441, MAPE= 0.4515) the Linear regression algorithms was found to outperformed Random Forest Regression. Therefore, the Linear regression algorithm is more reliable in predicting wind speed compared with the Random Forest regression.

Full text Figures and data Side by side Abstract Editor's evaluation Introduction Materials and methods Results Discussion Data availability References Decision letter Author response Article and author information Metrics Abstract Background: Oxygen uptake (VO2) is one of the most important measures of fitness and critical vital sign. Cardiopulmonary exercise testing (CPET) is a valuable method of assessing fitness in sport and clinical settings. There is a lack of large studies on athletic populations to predict VO2max using somatic or submaximal CPET variables. Thus, this study aimed to: (1) derive prediction models for maximal VO2 (VO2max) based on submaximal exercise variables at anaerobic threshold (AT) or respiratory compensation point (RCP) or only somatic and (2) internally validate provided equations. Methods: Four thousand four hundred twenty-four male endurance athletes (EA) underwent maximal symptom-limited CPET on a treadmill (n=3330) or cycle ergometer (n=1094). The cohort was randomly divided between: variables selection (nrunners = 1998; ncyclist = 656), model building (nrunners = 666; ncyclist = 219), and validation (nrunners = 666; ncyclist = 219). Random forest was used to select the most significant variables. Models were derived and internally validated with multiple linear regression. Results: Runners were 36.24±8.45 years; BMI = 23.94 ± 2.43 kg·m−2; VO2max=53.81±6.67 mL·min−1·kg−1. Cyclists were 37.33±9.13 years; BMI = 24.34 ± 2.63 kg·m−2; VO2max=51.74±7.99 mL·min−1·kg−1. VO2 at AT and RCP were the most contributing variables to exercise equations. Body mass and body fat had the highest impact on the somatic equation. Model performance for VO2max based on variables at AT was R2=0.81, at RCP was R2=0.91, at AT and RCP was R2=0.91 and for somatic-only was R2=0.43. Conclusions: Derived prediction models were highly accurate and fairly replicable. Formulae allow for precise estimation of VO2max based on submaximal exercise performance or somatic variables. Presented models are applicable for sport and clinical settling. They are a valuable supplementary method for fitness practitioners to adjust individualised training recommendations. Funding: No external funding was received for this work. Editor's evaluation The authors have established new formulas to predict maximum oxygen uptake for cyclists and runners based on submaximal exercise testing and anthropometric characteristics. This is an important study with a large and comprehensive dataset, which may be helpful for many exercise labs. The work is convincing, using appropriate and validated methodology in line with the current state-of-the-art, as shown by references to common exercise books. https://doi.org/10.7554/eLife.86291.sa0 Decision letter Reviews on Sciety eLife's review process Introduction The oxygen uptake (VO2) is considered an important metric in assessing cardiorespiratory fitness, health status, or endurance performance potential (Guazzi et al., 2012). With the application of standardised procedures and interpretation protocols, during graded exercise tests (GXT), the (maximal oxygen uptake) VO2max can be established (Bentley et al., 2007). GXT is the most widely used assessment to examine the dynamic relationship between exercise and integrated physiological systems (Albouaini et al., 2007; Bentley et al., 2007). The information from GXT during cardiopulmonary exercise testing (CPET) can be applied across the spectrum of sport performance, occupational safety screening, research, and clinical diagnostics (Guazzi et al., 2017). VO2 max is often used as a boundary between severe and extreme intensity domains and by definition requires maximal effort from the tested subject (Gaesser and Poole, 1996). However, it is not always recommended or possible to undertake a test to exhaustion (Guazzi et al., 2012). For the athletes, the proximity of competition or injury history can allow submaximal testing, but not testing to exhaustion (Sassi et al., 2006). Testing that requires maximal effort may be disruptive to the training process or interfere with race performance (Coutts et al., 2007; Lamberts et al., 2011). Due to practical constraints, tests to exhaustion or peak-power-output tests are often performed only two or three times a year (Coutts et al., 2007). However, VO2 values are widely used in sport science and the decision-making process (Mann et al., 2013). VO2 is widely considered one of the major endurance performance determinants (Joyner and Coyle, 2008). Using VO2max to guide the selection process, prescribing training intensity, assessing training adaptations, or predicting race times is a common practice in high-performance sports (Bassett and Howley, 2000; Bentley et al., 2007; Hawley and Noakes, 1992; Noakes et al., 1990). VO2max is also one of the critical vital signs coordinating the function of the cardiovascular, respiratory, and muscular systems, it is an indicator of overall body health status (Kaminsky et al., 2017). Quantifying VO2max provides additional input regarding clinical decision-making, risk stratification, evaluation of therapy, and physical activity guidelines (Guazzi et al., 2012). For patients undertaking a test to exhaustion is rarely needed or possible due to health restraints or cardiac risk (Guazzi et al., 2016). For many years researchers have studied indirect methods of estimating VO2max(Sartor et al., 2013). Protocols such as the Astrand-Ryhming Test, Six-Minute Walk Test, or YMCA Step Test have been established and validated (Astrand and Ryhming, 1954; Beutner et al., 2015; Carey, 2022; Jalili et al., 2018). Moreover, estimation of the VO2 and heart rate (HR) values below the ventilatory threshold can be based on cardiorespiratory kinetics assessment using randomised changes in the work rate known as a pseudo-random binary sequences testing (Hoffmann et al., 2022). However, with the development of technology, the accessibility of laboratory testing and mobile testing improved (Montoye et al., 2020; Pritchard et al., 2021). Therefore, new opportunities to develop more precise yet simple and accessible methods and models to assess VO2max occur (Jurov et al., 2023). This appears to be especially important considering the low prediction accuracy of most of the VO2max formulae that were validated in our previous study (Wiecha et al., 2023). Recently, we have been observing the development of prediction methods with the usage of machine learning (ML) and artificial intelligence (AI) (Ashfaq et al., 2022). Both ML and AI are used in sport science as forecasting and decision-making support tools (Abut and Akay, 2015; Bobowik and Wiszomirska, 2022; Chmait and Westerbeek, 2021; Hammes et al., 2022; Rossi et al., 2021). There is growing evidence that VO2max prediction based on ML models, especially support vector ML and artificial neural network models, exhibits more robust and accurate results compared to MLR only (Abut and Akay, 2015; Ashfaq et al., 2022). Therefore, in this research, with the support of ML, we look for algorithms and prediction patterns that allow us to use values obtained during submaximal CPET and somatic measurements to estimate maximal VO2max values in male runners and cyclists. We stipulate that prediction models allow for accurate calculation of VO2max based on somatic or submaximal CPET variables. Materials and methods We have applied the development and validation of the prediction TRIPOD guidelines to conduct the study (see Supplementary Material 1TRIPOD Checklist for Prediction Model Development and Validation) (Collins et al., 2015). The study is based on retrospective data analysis from the CPET registry collected from 2013 to 2021 at the medical clinic (Sportslab, Warsaw, Poland). All CPET have been performed at the individual request of participants, as a part of regular training monitoring or performance assessment. Ethical approval Request a detailed protocol The Institutional Review Board of the Bioethical Committee at the Medical University of Warsaw (AKBE/32/2021) has approved the study protocol. The regulations of the Declaration of Helsinki were met during all parts of the study. Each study participant delivered written consent to undergo CPET and participate in the study. Derivation cohort Request a detailed protocol We selected the cohort with the use of rigorous exclusion/inclusion criteria. Due to the insufficient number of women in our database and the number of potential variables in the regression models for adequate power, we had to limit ourselves to conduct analysis in the male population only (Martens and Logan, 2021). Out of 6439 healthy, adult male cyclists and long-distance runners that undergone CPET, 4423 met the criteria as further: (1) age ≥18 years, (2) declared regular cycling or running training for ≥3 months, (3) had no extreme outliers ≤ or ≥±3 standard deviations (SD) from mean for all of the testing variables (beyond ≥±3 SD in VO2max), (4) lack of any injury, medical condition, or addiction in medical history that may affect exercise capacity, (5) not taking any medications with a modifying effect on exercise capacity, (6) maximum exertion achieved during CPET. We defined the maximum exertion in CPET as the fulfilment of the minimum six of the following criteria: (1) respiratory exchange ratio (RER) ≥1.10, (2) present VO2 plateau (growth <100 mL·min–1 in VO2 despite increased running speed or cycling power), (3) respiratory frequency (fR) ≥45 breaths·min–1, (4) declared subjective exertion intensity during CPET ≥18 in the Borg scale (Borg, 1970), (5) blood lactate concentration [La-]b ≥8 mmol·L–1, (6) growth in speed/power ≥10% of respiratory compensation point (RCP) values after exceeding the RCP, (7) peak heart rate (HRpeak) ≥15 beats·min–1 below predicted maximal heart rate (HRmax) (Lach et al., 2021). Participants’ selection procedure has been shown in Figure 1. Figure 1 Download asset Open asset Flowchart of the preliminary inclusion and exclusion process. Abbreviations: EA, endurance athlete; CPET, cardiopulmonary exercise testing; SD, standard deviation; TE, treadmill; RER, respiratory exchange ratio; VO2, oxygen uptake (mL·min−1·kg−1); [La−]b, lactate concentration (mmol·L−1); fR, breathing frequency (breaths·min−1); RCP, respiratory compensation point; HRpeak, peak heart rate (beats·min−1); HRmax, maximal heart rate (bpm). At both stages of the selection, some participants met several (>1) exclusion criteria. Somatic measurements and CPET protocols Request a detailed protocol Body mass was measured with a body composition (BC) analyser (Tanita, MC 718, Japan) with the multifrequency of 5 kHz/50 kHz/250 kHz via the bioimpedance analysis and normal testing mode. The participants’ skin was cleaned with alcohol before placing the electrodes on the skin. Prior to the test, the participants received instructions to refrain from exercising for 2 hr, consume a light meal rich in carbohydrates 2–3 hr beforehand, and maintain hydration by drinking isotonic beverages. Additionally, they were advised to abstain from medications, caffeine, and cigarettes on the day of the test. Running CPET (TE) was performed on a mechanical treadmill (h/p/Cosmos Quasar, Germany). Cycling CPET (CE) was performed on Cyclus-2 (RBM elektronik-automation GmbH, Leipzig, Germany). Hans Rudolph V2 mask (Hans Rudolph, Inc, Shawnee, KS, USA), breath-by-breath method with Cosmed Quark CPET gas exchange analysing device (Cosmed Srl, Rome, Italy), and Quark PFT Suite to Omnia 1.6 software were utilised. The gas analyser device was regularly calibrated with the reference gas (16% O2; 5% CO2) in accordance with the manufacturer’s instructions (Airgas USA, LLC, Plumsteadville, PA, USA). From 2013 to 2021, three Cosmed Quark CPET units were used. HR was measured with the Cosmed torso belt (Cosmed srl, Rome, Italy). [La-]b was measured via enzymatic-amperometric electrochemical technique with Super GL2 analyser (Müller Gerätebau GmbH, Freital, Germany). The [La-]b analyser was regularly calibrated before each measurement series. The 40 m2 indoor, air-conditioned laboratory with 20–22°C temperature and 40–60% humidity, and 100 m ASL provided the same conditions for all BC and CPET. Each CPET began with a 5 min personalised warm-up (walk or easy jog with ‘conversational’ intensity for running, easy pedalling with ‘conversational’ intensity for cycling). Then after the preparation (about 5 min), the continuous progressive step test was conducted. Due to the population diversity (training status), the running test speed started from 7 to 12 km·hr–1 with a 1% treadmill incline. The choice of initial starting speed was determined by the interview and sports results achieved. For example, those running less than 60 min at a distance of 10 km started the test at 7 km/hr, while those running 10 km for less than 35 min started the test at an initial speed of 12 km/hr. The pace increased by 1 km·hr–1 every 2 min with no change in incline. The cycling test began at 60–150 W, depending on the athletes training status. The power increased by 20–30 W every 2 min. It was recommended to maintain a constant cadence of 80–90 (repetition·min–1) during the test. The tests were terminated due to exhaustion: volitional inability to continue the activity or/and VO2 and HR plateau with increasing load or/and observed disturbance of coordination in running or/and inability to maintain the set cadence. Due to the graded protocol used, the cycling power and running speed values have been calculated as a function of time to better reflect the actual level for the test moment being determined (Kuipers et al., 1985). Before the test, after every step, and 3 min after the termination of the effort technician took a 20 µL blood sample from a fingertip. Samples were collected during the test without interrupting the effort. The samples were taken from the initial puncture. The first blood drop was collected into the swab and the second blood drop was drawn for further analysis into the capillary. VO2max was recorded as the highest value (15 s intervals) before the termination of the test. HRmax was recorded as the highest value obtained at the end of the test, without averaging. The anaerobic threshold (AT) was established with the following criteria: (1) common start of VE/VO2 and VE/VCO2 curves, (2) end-tidal partial pressure of oxygen raised constantly with the end-tidal partial pressure of carbon dioxide (Beaver et al., 1986). The was established with the following criteria: (1) PetCO2 must decrease after reaching maximal amount, (2) the presence of fast nonlinear growth in VE (second deflection), (3) the VE/VCO2 ratio achieved minimum and started to rise, and (4) a nonlinear increase in VCO2 versus VO2 (lack of linearity) (Beaver et al., 1986). The [La-]b was estimated for AT and RCP in relation to power or speed (Wiecha et al., 2022). Data analysis Request a detailed protocol Our comprehensive ML approach enables the evaluation of each formula by preliminary variables precision (at the stage of selection), then accuracy (during the model’s building) and recall (in internal validation). Individual CPET results were saved into the Excel file (Microsoft Corporation, Redmond, WA, USA) and a custom-made script was used to generate the database in Excel (Python programming). Further, mean, SD, and 95% confidence intervals (CI) were calculated. The normality of the distribution of the data was examined using the Shapiro-Wilk test and intergroup differences were calculated using the Student’s t-test for independent variables. Three-step variable selection procedures based on random forests were applied using the R package VSURF in RStudio software (R Core Team, Vienna, Austria; version 3.6.4) (Genuer et al., 2016). For each level of measurement (AT, RCP) and their combination (AT+RCP), significant variables were identified separately. The first step was dedicated to eliminate irrelevant variables from the dataset. Second step aimed to select all variables related to the response for interpretation purposes. The third step refined the selection by eliminating redundancy in the set of variables selected by the second step, for prediction purposes (Genuer et al., 2017). Each time for variables selection, the anthropometric variables as in Tables 1–2 and the CPET parameters given in Tables 3–4 from a specific level of measurement (AT; RCP) and their combinations were visible. Table 1 Basic anthropometric characteristics for runners. Variable (unit)Derivation group n=1998Testing group n=666Validation group n=666MeanCISDMeanCISDMeanCISDAge (years)36.235.6–36.98.4535.935.5–36.38.0535.534.9–36.28.14Height (cm)180.0179.6–180.56.04179.4179.1–179.76.13179.7179.2–180.26.61BM (kg)77.777.0–78.49.3577.777.3–78.19.2977.977.1–78.610.1BMI (kg·m–2)23.923.8–24.12.4324.124.0–24.22.4124.123.9–24.32.56BF (%)15.415.1–15.74.5515.515.3–15.74.5215.415.1–15.84.55FM (kg)12.211.9–12.64.6812.312.1–12.54.6512.311.9–12.74.92FFM (kg)65.565.0–66.06.4365.465.1–65.76.3165.665.1–66.16.86 BM, body mass; BMI, body mass index; BF, body fat; FM, fat mass; FFM, fat-free mass; CI, 95% confidence interval; SD, standard deviation. Table 2 Basic anthropometric characteristics for cyclists. Variable (unit)Derivation group n=656Testing group n=219Validation group n=219MeanCISDMeanCISDMeanCISDAge (years)37.336.6–38.09.1337.135.9–38.49.5037.636.5–38.88.46Height (cm)179.9179.4–180.46.27180.1179.2–181.06.96180.2179.4–181.06.13BM (kg)78.878.1–79.69.8079.177.7–80.510.479.878.4–81.310.9BMI (kg·m–2)24.324.1–24.62.6324.424.0–24.72.8024.624.2–25.02.96BF (%)16.415.7–17.14.9916.115.7–16.54.8116.215.5–16.84.87FM (kg)13.312.6–14.15.6613.012.6–13.45.2713.312.5–14.05.85FFM (kg)65.864.9–66.66.2565.865.4–66.36.0666.665.7–67.46.58 BM, body mass; BMI, body mass index; BF, body fat; FM, fat mass; FFM, fat-free mass; CI, 95% confidence interval; SD, standard deviation. Table 3 Cardiopulmonary exercise testing (CPET) characteristics for runners. Variable (unit)Derivation group n=1998Testing group n=666Validation group n=666MeanCISDMeanCISDMeanCISDrVO2AT (mL·min–1·kg–1)38.438.1–38.85.0138.538.3–38.74.8838.137.7–38.55.16RERAT0.870.86–0.870.040.870.86–0.870.040.870.86–0.870.04HRAT (beats·min–1)151.5150.8–152.310.3151.0150.6–151.510.8152.0151.2–152.810.8VEAT (L·min–1)79.178.1–80.012.278.377.8–78.912.077.276.3–78.212.0SPEEDAT (km·h–1)11.010.9–11.11.4511.011.0–11.11.3610.910.8–11.01.42LAAT (mmol·L–1)2.082.02–2.140.631.801.76–1.830.622.352.27–2.420.72rVO2RCP (mL·min–1·kg–1)47.547.0–48.05.8847.747.4–48.06.1547.346.8–47.86.16RERRCP1.001.00–1.000.041.001.00–1.000.041.001.00–1.000.03HRRCP (beats·min–1)173.4172.7–174.19.21173.2172.8–173.69.30174.3173.5–175.09.50VERCP (L·min–1)114.7113.5–116.015.9113.9113.1–114.616.7112.7111.4–114.016.2SPEEDRCP (km·h–1)14.013.9–14.11.7714.114.0–14.11.7013.913.8–14.11.75LARCP (mmol·L–1)4.724.63–4.821.044.404.34–4.451.044.814.69–4.931.14rVO2max (mL·min–1·kg–1)53.853.3–54.36.6754.354.0–54.66.9553.853.3–54.37.09 CI, 95% confidence interval; SD, standard deviation; rVO2AT, oxygen uptake at anaerobic threshold relative to body mass; RERAT, respiratory exchange ratio at anaerobic threshold; HRAT, heart rate at anaerobic threshold; VEAT, pulmonary ventilation at anaerobic threshold; SPEEDAT, velocity at anaerobic threshold; LAAT, blood lactate concentration at anaerobic threshold; rVO2RCP, oxygen uptake at respiratory compensation point relative to body mass; RERRCP, respiratory exchange ratio at respiratory compensation point; HRRCP, heart rate at respiratory compensation point; VERCP, pulmonary ventilation at respiratory compensation point; SPEEDRCP, velocity at respiratory compensation point; LARCP, blood lactate concentration at respiratory compensation point; rVO2max, maximal oxygen uptake relative to body mass. Table 4 Cardiopulmonary exercise testing (CPET) characteristics for cyclists. Variable (unit)Derivation group n=656Testing group n=219Validation group n=219MeanCISDMeanCISDMeanCISDrVO2AT (mL·min–1·kg–1)33.032.5–33.45.8433.232.4–33.95.6833.732.9–34.55.89RERAT0.870.87–0.870.040.870.87–0.880.040.870.87–0.880.04HRAT CI, 95% confidence interval; SD, standard deviation; rVO2AT, oxygen uptake at anaerobic threshold relative to body mass; RERAT, respiratory exchange ratio at anaerobic threshold; HRAT, heart rate at anaerobic threshold; VEAT, pulmonary ventilation at anaerobic threshold; power at anaerobic threshold relative to body mass; LAAT, blood lactate concentration at anaerobic threshold; rVO2RCP, oxygen uptake at respiratory compensation point relative to body mass; RERRCP, respiratory exchange ratio at respiratory compensation point; HRRCP, heart rate at respiratory compensation point; VERCP, pulmonary ventilation at respiratory compensation point; LARCP, blood lactate concentration at respiratory compensation point; power at respiratory compensation point relative to body mass; rVO2max, maximal oxygen uptake relative to body mass. selection variables were in the further only selected parameters were into multiple linear regression The data for MLR model building were randomly into that is testing, validation and of the a only significant were in the Derived are by the of mean and mean analysis was used to the model’s precision and accuracy during validation and tests to the fulfilment of MLR test the of in MLR test assessment between and test of Each model was examined the and any have not been 2 package in RStudio (R Core Team, Vienna, Austria; version version for and software version were used in was considered as the Results Somatic measurements and CPET results data of the runners models for testing, and validation are in Table while cyclists are in Table The runners of and for testing, and validation the cyclists and differences between of runners and cyclists were in BMI and between testing in all between validation only in CPET results for runners models are in Table 3 and for cyclists in Table Runners in the cohort achieved relative to body mass VO2max of in testing group and in validation group cyclists mean was and for testing, and validation to body mass oxygen uptake at anaerobic threshold in runners for ± ± and ± of in testing, and validation it was ± ± and ± of rVO2max, relative to body mass oxygen uptake at respiratory compensation point in runners for ± ± and ± of for testing, and validation while in cyclists for ± ± and ± of rVO2max, There were no significant differences in values between testing, and validation the runners and cyclists between runners and cyclists results were all significant Prediction models based on AT and RCP Full of MLR prediction models for cyclists are in Table for runners in Table The models prediction performance is as with and for cyclists from for somatic parameters to for RCP equations. For runners from for to for AT and equations. for cyclists models was the for RCP and the highest for For from for AT and to for equation. observed for cyclists was the for RCP in the validation group and the highest for while in runners the for AT and and the highest for The performance of prediction is in Figure Figure 2 Download asset Open asset of prediction for Abbreviations: maximal oxygen anaerobic threshold; RCP, respiratory compensation point; All values are in performance for running while the performance for cycling equations. performance of the prediction model for for for AT and for somatic-only equation. Table 5 VO2max prediction for cyclists. linear regression group group = = = = based on anaerobic threshold; RCP, based on respiratory compensation point; based on somatic variables mean mean maximal oxygen uptake relative to body mass rVO2AT, oxygen uptake at anaerobic threshold relative to body mass power at anaerobic threshold relative to body mass rVO2RCP, oxygen uptake at respiratory compensation point relative to body mass VERCP, pulmonary ventilation at respiratory compensation point BF, body fat BM, body mass Table VO2max prediction for runners. linear regression group group = = = = based on anaerobic threshold; RCP, based on respiratory compensation point; based on somatic variables mean mean maximal oxygen uptake relative to body mass rVO2AT, oxygen uptake at anaerobic threshold relative to body mass SPEEDAT, velocity at anaerobic threshold FFM, fat mass VEAT, pulmonary ventilation at anaerobic threshold HRAT, heart rate at anaerobic threshold BF, body fat rVO2RCP, oxygen uptake at respiratory compensation point relative to body mass SPEEDRCP, velocity at respiratory compensation point Models validation of each model for cyclists is in Table while for runners in Table the performance of our prediction was to that observed in the

Traditional approaches for learning representations from somatic mutation data often rely on the distribution of motif counts, which have limitations, including the use of small sequence contexts and the exclusion of critical information, such as mutation location and interactions among mutations. Recent efforts have leveraged advances in natural language processing (NLP) to improve representation learning from somatic mutations by incorporating larger sequence contexts into shallow networks. More recently, studies have demonstrated the feasibility of leveraging attention mechanisms to learn meaningful representations. However, these methods primarily use supervised or weakly supervised learning paradigms, which are inherently task-specific. The development of general-purpose representations, by harnessing the vast amount of somatic mutation data through self-supervised techniques, remains underexplored. Such representations could drive progress in applications including tumor subtype classification, biomarker discovery, patient stratification, drug response prediction, and pathway enrichment analysis. This study aims to develop a deep, unsupervised learning framework that leverages abundant somatic mutation data to generate robust, general-purpose representations. By adapting recent advances in self-supervised learning, the proposed approach seeks to establish a foundation model capable of capturing meaningful patterns in mutation data for diverse downstream tasks. The framework employs novel tokenization and embedding strategies to encode mutation sequences and utilizes transformer-based architectures to model contextual dependencies. The process of inputting mutation data into a transformer architecture, particularly the tokenization step, is critical for this problem, and we explore various tokenization strategies. Benchmarking these representations will include evaluating their utility across tumor subtype classification, biomarker discovery, patient stratification, drug response prediction, identification of mutational signatures. Although the model is under development, experiments from prior work—limited to mutation identifiers without leveraging sequence motifs, mutation location, or other contextual information—have demonstrated the feasibility of encoding mutational data into meaningful representations using transformer-based architectures and self-supervised learning. Building on these findings, this framework will benchmark its performance on multiple downstream tasks, including tumor subtype classification and patient stratification. Citation Format: Marzieh Haghighi, Gustavo Alonso Arango Argoty, Etai Jacob. Self-supervised representation learning of somatic mutational data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5059.

We introduce a new instrumental variable (IV) estimator for heterogeneous treatment effects in the presence of endogeneity. Our estimator is based on double/debiased machine learning (DML) and uses efficient machine learning instruments (MLIV) and kernel smoothing. We prove consistency and asymptotic normality of our estimator and also construct confidence sets that are more robust towards weak IV. Along the way, we also provide an accessible discussion of the corresponding estimator for the homogeneous treatment effect with efficient machine learning instruments. The methods are evaluated on synthetic and real datasets and an implementation is made available in the R package IVDML.

&amp;lt;p&amp;gt;Accurate water level (WL) forecasting is important for water resources management and planning purposes in the Great Lakes. The objectives of this research are two-fold.&amp;amp;#160; The first objective is to apply machine learning (ML) (i.e., random forest (RF) and support vector regression (SVR)) and hybrid convolutional neural network(CNN)-long-short term memory (LSTM) deep learning (DL) models for multi-step (i.e., one-, two- and three-monthly step ahead) WL forecasting in the Great Lakes (Michigan and Ontario). The second objective is to integrate the boundary corrected (BC) maximal overlap discrete wavelet transform (MODWT) with SVR, RF, and CNN-LSTM models to improve the performance of the individual models. By employing a BC-wavelet decomposition method, the &amp;amp;#8216;future data&amp;amp;#8217; issue (i.e., data from the future that is not available), often overlooked in the literature and a major barrier to achieving realistic forecasting performance is overcome.&amp;amp;#160;&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;For Lakes Michigan and Ontario, 1212 monthly WL (m) records (spanning Jan 1918&amp;amp;#8211;Dec 2018) were used to develop the models. For the non-wavelet-based models (SVR, RF, and CNN-LSTM), candidate model inputs included the WL recorded over the previous 12 months.&amp;amp;#160; For the BC-MODWT-based models (BC-MODWT-SVR, BC-MODWT-RF, and BC-MODWT-CNN-LSTM), the lagged input time series were decomposed into BC-wavelet and scaling coefficients by using different mother wavelets (Haar, Daubechies, Symlets, Fejer-Korovkin and Coiflets), filter lengths (from two up to 12) and decomposition levels (from one up to seven).&amp;amp;#160; For each method (SVR, RF, and CNN-LSTM), mother wavelet, and decomposition level a model was generated.&amp;amp;#160; For both wavelet- and non-wavelet-based models, the particle swarm optimization (PSO) method was used to select the most appropriate inputs to include in the proposed multi-step WL forecasting models.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;The datasets were partitioned into calibration and validation subsets. After calibrating the models, various performance evaluation metrics, e.g., coefficient of determination (R&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt;), root mean square error (RMSE), mean absolute error (MAE), root mean square percentage error (RMSPE), mean absolute percentage error (MAPE) and the Nash-Sutcliffe efficiency coefficient (NSC) were used to assess model accuracy.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;Of the ML models, the SVR outperformed RF while the DL models outperformed the ML models for each forecast lead time (one-, two-, and three-step(s) ahead). Results from this case study indicate that not all wavelet families and decomposition levels perform equally and in some cases, the wavelet-based models do not improve performance over the non-wavelet-based models. However, the BC-MODWT-CNN-LSTM using suitable mother wavelets (e.g., Haar) outperforms the individual ML and BC-MODWT-ML-based models. More accurate forecasts were obtained for Lake Michigan although the performance in both Great Lakes was accurate. The outcomes of this research indicate that the BC-MODWT-CNN-LSTM model is a promising tool for generating accurate WL forecasts.&amp;lt;/p&amp;gt;

Background Forecasting the time of forthcoming pandemic reduces the impact of diseases by taking precautionary steps such as public health messaging and raising the consciousness of doctors. With the continuous and rapid increase in the cumulative incidence of COVID-19, statistical and outbreak prediction models including various machine learning (ML) models are being used by the research community to track and predict the trend of the epidemic, and also in developing appropriate strategies to combat and manage its spread. Methods In this paper, we present a comparative analysis of various ML approaches including Support Vector Machine, Random Forest, K-Nearest Neighbor and Artificial Neural Network in predicting the COVID-19 outbreak in the epidemiological domain. We first apply the autoregressive distributed lag (ARDL) method to identify and model the short and long-run relationships of the time-series COVID-19 datasets. That is, we determine the lags between a response variable and its respective explanatory time series variables as independent variables. Then, the resulting significant variables concerning their lags are used in the regression model selected by the ARDL for predicting and forecasting the trend of the epidemic. Results Statistical measures—Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE)—are used for model accuracy. The values of MAPE for the best-selected models for confirmed, recovered and deaths cases are 0.003, 0.006 and 0.115, respectively, which falls under the category of highly accurate forecasts. In addition, we computed 15 days ahead forecast for the daily deaths, recovered, and confirm patients and the cases fluctuated across time in all aspects. Besides, the results reveal the advantages of ML algorithms for supporting the decision-making of evolving short-term policies.

Background Forecasting the time of forthcoming pandemic reduces the impact of diseases by taking precautionary steps such as public health messaging and raising the consciousness of doctors. With the continuous and rapid increase in the cumulative incidence of COVID-19, statistical and outbreak prediction models including various machine learning (ML) models are being used by the research community to track and predict the trend of the epidemic, and also in developing appropriate strategies to combat and manage its spread. Methods In this paper, we present a comparative analysis of various ML approaches including Support Vector Machine, Random Forest, K-Nearest Neighbor and Artificial Neural Network in predicting the COVID-19 outbreak in the epidemiological domain. We first apply the autoregressive distributed lag (ARDL) method to identify and model the short and long-run relationships of the time-series COVID-19 datasets. That is, we determine the lags between a response variable and its respective explanatory time series variables as independent variables. Then, the resulting significant variables concerning their lags are used in the regression model selected by the ARDL for predicting and forecasting the trend of the epidemic. Results Statistical measures—Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE)—are used for model accuracy. The values of MAPE for the best-selected models for confirmed, recovered and deaths cases are 0.003, 0.006 and 0.115, respectively, which falls under the category of highly accurate forecasts. In addition, we computed 15 days ahead forecast for the daily deaths, recovered, and confirm patients and the cases fluctuated across time in all aspects. Besides, the results reveal the advantages of ML algorithms for supporting the decision-making of evolving short-term policies.

Background Forecasting the time of forthcoming pandemic reduces the impact of diseases by taking precautionary steps such as public health messaging and raising the consciousness of doctors. With the continuous and rapid increase in the cumulative incidence of COVID-19, statistical and outbreak prediction models including various machine learning (ML) models are being used by the research community to track and predict the trend of the epidemic, and also in developing appropriate strategies to combat and manage its spread. Methods In this paper, we present a comparative analysis of various ML approaches including Support Vector Machine, Random Forest, K-Nearest Neighbor and Artificial Neural Network in predicting the COVID-19 outbreak in the epidemiological domain. We first apply the autoregressive distributed lag (ARDL) method to identify and model the short and long-run relationships of the time-series COVID-19 datasets. That is, we determine the lags between a response variable and its respective explanatory time series variables as independent variables. Then, the resulting significant variables concerning their lags are used in the regression model selected by the ARDL for predicting and forecasting the trend of the epidemic. Results Statistical measures—Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE)—are used for model accuracy. The values of MAPE for the best-selected models for confirmed, recovered and deaths cases are 0.003, 0.006 and 0.115, respectively, which falls under the category of highly accurate forecasts. In addition, we computed 15 days ahead forecast for the daily deaths, recovered, and confirm patients and the cases fluctuated across time in all aspects. Besides, the results reveal the advantages of ML algorithms for supporting the decision-making of evolving short-term policies.

Background Forecasting the time of forthcoming pandemic reduces the impact of diseases by taking precautionary steps such as public health messaging and raising the consciousness of doctors. With the continuous and rapid increase in the cumulative incidence of COVID-19, statistical and outbreak prediction models including various machine learning (ML) models are being used by the research community to track and predict the trend of the epidemic, and also in developing appropriate strategies to combat and manage its spread. Methods In this paper, we present a comparative analysis of various ML approaches including Support Vector Machine, Random Forest, K-Nearest Neighbor and Artificial Neural Network in predicting the COVID-19 outbreak in the epidemiological domain. We first apply the autoregressive distributed lag (ARDL) method to identify and model the short and long-run relationships of the time-series COVID-19 datasets. That is, we determine the lags between a response variable and its respective explanatory time series variables as independent variables. Then, the resulting significant variables concerning their lags are used in the regression model selected by the ARDL for predicting and forecasting the trend of the epidemic. Results Statistical measures—Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE)—are used for model accuracy. The values of MAPE for the best-selected models for confirmed, recovered and deaths cases are 0.003, 0.006 and 0.115, respectively, which falls under the category of highly accurate forecasts. In addition, we computed 15 days ahead forecast for the daily deaths, recovered, and confirm patients and the cases fluctuated across time in all aspects. Besides, the results reveal the advantages of ML algorithms for supporting the decision-making of evolving short-term policies.

Background Forecasting the time of forthcoming pandemic reduces the impact of diseases by taking precautionary steps such as public health messaging and raising the consciousness of doctors. With the continuous and rapid increase in the cumulative incidence of COVID-19, statistical and outbreak prediction models including various machine learning (ML) models are being used by the research community to track and predict the trend of the epidemic, and also in developing appropriate strategies to combat and manage its spread. Methods In this paper, we present a comparative analysis of various ML approaches including Support Vector Machine, Random Forest, K-Nearest Neighbor and Artificial Neural Network in predicting the COVID-19 outbreak in the epidemiological domain. We first apply the autoregressive distributed lag (ARDL) method to identify and model the short and long-run relationships of the time-series COVID-19 datasets. That is, we determine the lags between a response variable and its respective explanatory time series variables as independent variables. Then, the resulting significant variables concerning their lags are used in the regression model selected by the ARDL for predicting and forecasting the trend of the epidemic. Results Statistical measures—Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE)—are used for model accuracy. The values of MAPE for the best-selected models for confirmed, recovered and deaths cases are 0.003, 0.006 and 0.115, respectively, which falls under the category of highly accurate forecasts. In addition, we computed 15 days ahead forecast for the daily deaths, recovered, and confirm patients and the cases fluctuated across time in all aspects. Besides, the results reveal the advantages of ML algorithms for supporting the decision-making of evolving short-term policies.

Brain Age Estimation (BAE) has become a popular challenge in the field of medical and computer sciences in recent years. In the medical sciences field, the investigation on the brain structure and its relationship with aging is considerable. In the computer sciences field, creating an efficient Machine Learning (ML) model of BAE would lead to have accurate regression models. In this paper, a 3D Convolutional Neural Network (3D-CNN) model is used to train a brain age estimation system. To reach a more accurate system, two other regression methods are also applied on the final feature vector generated by 3D-CNN system. The system is applied on the samples of IXI dataset normalized by SPM14. Next, to ensure the model’s generalization, 47 healthy samples of ADNI1 dataset are used. Furthermore, some MRI images achieved from Alzheimer patients are feed to the proposed model and the effects of Alzheimer disease on brain aging are investigated. The best Mean Absolute Error (MAE) on evaluation dataset is about 5 years, with Root Mean Square Error (RMSE) = 13.5. The model generalization by a new healthy dataset was evaluated and the result is with the MAE value of about 6 years.

Deep learning has become a valuable tool for the automation of certain medical image segmentation tasks, significantly relieving the workload of medical specialists. Some of these tasks require segmentation to be performed on a subset of the input dimensions, the most common case being 3D→2D. However, the performance of existing methods is strongly conditioned by the amount of labeled data available, as there is currently no data efficient method, e.g. transfer learning, that has been validated on these tasks. In this work, we propose a novel convolutional neural network (CNN) and self-supervised learning (SSL) method for label-efficient 3D→2D segmentation. The CNN is composed of a 3D encoder and a 2D decoder connected by novel 3D→2D blocks. The SSL method consists of reconstructing image pairs of modalities with different dimensionality. The approach has been validated in two tasks with clinical relevance: the en-face segmentation of geographic atrophy and reticular pseudodrusen in optical coherence tomography. Results on different datasets demonstrate that the proposed CNN significantly improves the state of the art in scenarios with limited labeled data by up to 8% in Dice score. Moreover, the proposed SSL method allows further improvement of this performance by up to 23%, and we show that the SSL is beneficial regardless of the network architecture. Our code is available at https://github.com/j-morano/multimodal-ssl-fpn.

Forecasting streamflow in stream basin systems plays a crucial role in facilitating effective urban planning to mitigate floods. In addition to employing intricate hydrological modeling systems, machine learning and statistical techniques offer an alternative means for streamflow forecasts. Nonetheless, the precision and dependability of these methods are subjects of paramount importance. This study rigorously investigates the effectiveness of three distinct machine learning techniques and two statistical approaches when applied to streamflow data from the Göksu Stream in the Marmara Region of Turkey, spanning from 1984 to 2022. Through a comparative analysis of these methodologies, this examination aims to contribute innovative advancements to the existing methodologies used in the prediction of streamflow data. The methodologies employed include machine learning methods such as Extreme Gradient Boosting (XGBoost), Random Forest (RF), and Support Vector Machine (SVM) and statistical methods such as Simple Exponential Smoothing (SES) and Autoregressive Integrated Moving Average (ARIMA) model. In the study, 444 data points between 1984 and 2020 were used as training data, and the remaining data points for the period 2021–2022 were used for streamflow forecasting in the test validation period. The results were evaluated using various metrics, such as the correlation coefficient (r), mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), coefficient of determination (R2), and Nash–Sutcliffe efficiency (NSE). Upon analyzing the results, it was found that the model generated using the XGBoost algorithm outperformed other machine learning and statistical techniques. Consequently, the models implemented in this study demonstrate a high level of accuracy in predicting potential streamflow in the river basin system.

The Grid Mean Algorithm is a computational approach designed to evaluate regression metrics such as coefficient of determination (R2), mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) directly on tabular data without the need to train machine learning (ML) models. This method enables researchers and practitioners to assess the potential of data for regression tasks, estimate the feasibility of ML projects, and make informed decisions about resource allocation. Additionally, the algorithm allows for estimating the approximate accuracy limit achievable with the given data, making it a valuable criterion for determining the optimality of a model. By addressing whether further research stages are necessary or redundant, it provides a practical tool for planning ML experiments and evaluating the economic viability of investing in such models. Experiments on synthetic datasets demonstrate the method's capability to produce accurate metric estimates across various functional forms and noise levels, making it a robust choice for initial data exploration and ML project planning.