Interventionally-guided representation learning for robust and interpretable AI models in cancer medicine

In this paper we propose a new class of dynamic mixture models (DAMMs) being able to sequentially adapt the mixture components as well as the mixture composition using information coming from the data. The information driven nature of the proposed class of models allows to exactly compute the full likelihood and to avoid computer intensive simulation schemes. Specific models for financial data are developed starting from the general specification. These models nest many specifications already available in the literature. The properties of the new class of models are discussed through the paper and a large-scale application in quantitative risk management using U.S. equity data is reported.

To evaluate the diagnostic performance of Machine Learning (ML) and Deep Learning (DL) models for predicting preoperative Lymph Node Metastasis (LNM) in Colorectal Cancer (CRC) patients. A systematic review and meta-analysis were conducted following PRISMA-DTA and AMSTAR-2 guidelines. We searched PubMed, Web of Science, Embase, and Cochrane Library databases until February 16, 2024. Study quality and risk of bias were assessed using the QUADAS-2 tool. Data were analyzed using STATA v18, applying random-effects models to all analyses. Twelve studies involving 8321 patients were included, with most published in 2021-2024 (9/12). The pooled AUC of ML models for predicting LNM in CRC patients was 0.87 (95% CI: 0.82-0.91, I2:86.17) with a sensitivity of 78% (95% CI: 69-87%) and a specificity of 77% (95% CI: 64%-90%). In addition, when assessing the AUC reported by radiologists, both junior and senior radiologists had similar performance, significantly lower than the ML models. (P < 0.001). Subgroup analysis revealed higher AUCs in prospective studies (0.95, 95% CI: 0.87-1) compared to retrospective studies (0.85, 95% CI: 0.81-0.89) (P = 0.03). Studies without external validation exhibited significantly higher AUCs than those with external validation (P < 0.01). While there was no significant difference in AUC and sensitivity between the T1-T2 and T2-T4 stages, specificity was significantly higher in the T2-T4 stages than the low stages of T1 and T2 (95%, 95% CI: 92-98% vs. 61%, 95% CI: 44-78%; P < 0.01). ML models demonstrate strong potential for preoperative LNM staging and treatment planning in CRC, potentially reducing the need for additional surgeries and related health and financial burdens. Further prospective multicenter studies, with standardized reporting of algorithms, modality parameters, and LNM staging, are needed to validate these findings.

The size of chemical compound space is too large to be probed exhaustively. This leads high-throughput protocols to drastically subsample and results in sparse and nonuniform datasets. Rather than arbitrarily selecting compounds, we systematically explore chemical space according to the target property of interest. We first perform importance sampling by introducing a Markov chain Monte Carlo scheme across compounds. We then train a machine learning (ML) model on the sampled data to expand the region of chemical space probed. Our boosting procedure enhances the number of compounds by a factor 2 to 10, enabled by the ML model's coarse-grained representation, which both simplifies the structure-property relationship and reduces the size of chemical space. The ML model correctly recovers linear relationships between transfer free energies. These linear relationships correspond to features that are global to the dataset, marking the region of chemical space up to which predictions are reliable; this is a more robust alternative to the predictive variance. Bridging coarse-grained simulations with ML gives rise to an unprecedented database of drug-membrane insertion free energies for 1.3 million compounds.

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

Distributionally robust optimization tackles out-of-sample issues like overfitting and distribution shifts by adopting an adversarial approach over a range of possible data distributions, known as the ambiguity set. To balance conservatism and accuracy, these sets must include realistic probability distributions by leveraging information from the nominal distribution. Assuming that nominal distributions arise from a structural causal model with a directed acyclic graph G and structural equations, previous methods such as adapted and G-causal optimal transport have only utilized causal graph information in designing ambiguity sets. In this work, we propose incorporating structural equations, which include causal graph information, to enhance ambiguity sets, resulting in more realistic distributions. We introduce structural causal optimal transport and its associated ambiguity set, demonstrating their advantages and connections to previous methods. A key benefit of our approach is a relaxed version, where a regularization term replaces the complex causal constraints, enabling an efficient algorithm via difference-of-convex programming to solve structural causal optimal transport. We also show that when structural information is absent and must be estimated, our approach remains effective and provides finite sample guarantees. Lastly, we address the radius of ambiguity sets, illustrating how our method overcomes the curse of dimensionality in optimal transport problems, achieving faster shrinkage with dimension-free order.

Federated learning (FL) has arisen as an alternative to collecting large amounts of data in a central place to train a machine learning (ML) model. FL is privacy-friendly, allowing multiple parties to collaboratively train an ML model without exchanging or transmitting their training data. For this purpose, an aggregator iteratively coordinates the training process among parties, and parties simply share with the aggregator model updates, which contain information pertinent to the model such as neural network weights. Besides privacy, generalization has been another key driver for FL: parties who do not have enough data to train a good performing model by themselves can now engage in FL to obtain an ML model suitable for their tasks. Products and real applications in the industry and consumer space have demonstrated the power of this learning paradigm. Recently, foundation models have taken the AI community by storm, promising to solve the shortage of labeled data. A foundation model is a powerful model that can be recycled for a variety of use cases by applying techniques such as zero-shot learning and full or parameter-efficient fine tuning. The premise is that the amount of data required to fine tune a foundation model for a new task is much smaller than fully training a traditional model from scratch. The reason why this is the case is that a good foundation model has already learned relevant general representations, and thus, adapting it to a new task only requires a minimal number of additional samples. This raises the question: Is FL still alive in the era of foundation models? In this talk, I will address this question. I will present some use cases where FL is very much alive. In these use cases, finding a foundation model with a desired representation is difficult if not impossible. With this pragmatic point of view, I hope to shed some light into a real use case where disparate private data is available in isolation at different parties and where labels may be located at a single party that doesn’t have any other information, making it impossible for a single party to train a model on its own. Furthermore, in some vertically-partitioned scenarios, cleaning data is not an option due to privacy-related reasons and it is not clear how to apply foundation models. Finally, I will also go over a few other requirements that are often overlooked, such as unlearning of data and its implications for the lifecycle management of FL and systems based on foundation models.

In this paper machine learning (ML) models have been developed for the application of big data load forecasting using parallel computation. The load forecasting models’ performance is directly linked to system execution capacity, memory, thread count, balancing the load, and available resources. This paper is focused on two main challenges. The first challenge is to reduce the execution time of the ML models and the second one is to choose the suitable tree-based model for effective load forecasting. The paper conducts a comprehensive evaluation of the load forecasting using real-world data on energy consumption. Comprehensive results are obtained to show that the performance of random search to tune the ML models exhibits competitive performances whilst not losing the accuracy of the models and gaining a competitive advantage on the run time.

Simple SummarySalivary gland cancer (SGC) is a rare and heterogeneous cancer for which limited treatment options are available in the palliative treatment setting. Characterization of the SGC genetic landscape to identify actionable aberrations is therefore important. This research aimed to comprehensively assess the prevalence of various types of actionable aberrations, including gene fusions, in a large cohort of patients with different SGC subtypes. The combined approach using RNA- and DNA-based targeted next-generation sequencing panels revealed the presence of gene fusions in half of the cases, including several fusions not previously described in SGC. Targets for genetically matched therapies were identified in 28.3–81.8% of cases, depending on the SGC subtype (overall 53.7% of the cases). This highlights the potential of molecular diagnostics to select systemic treatment in SGC.Introduction: Salivary gland cancer (SGC) is a rare cancer for which systemic treatment options are limited. Therefore, it is important to characterize its genetic landscape in search for actionable aberrations, such as NTRK gene fusions. This research aimed to identify these actionable aberrations by combining NGS-based analysis of RNA (gene fusions) and DNA (single and multiple nucleotide variants, copy number variants, microsatellite instability and tumor mutational burden) in a large cohort of SGC patients. Methods: RNA and DNA were extracted from archival tissue of 121 patients with various SGC subtypes. Gene fusion analysis was performed using a customized RNA-based targeted NGS panel. DNA was sequenced using a targeted NGS panel encompassing 523 cancer-related genes. Cross-validation of NGS-based NTRK fusion detection and pan-TRK immunohistochemistry (IHC) was performed. Results: Fusion transcripts were detected in 50% of the cases and included both known (MYB-NFIB, MYBL1-NFIB, CRTC1-MAML2) and previously unknown fusions (including transcripts involving RET, BRAF or RAD51B). Only one NTRK fusion transcript was detected, in a secretory carcinoma case. Pan-TRK IHC (clone EPR17341) was false positive in 74% of cases. The proportion of patients with targets for genetically matched therapies differed among subtypes (salivary duct carcinoma: 82%, adenoid cystic carcinoma 28%, mucoepidermoid carcinoma 50%, acinic cell carcinoma 33%). Actionable aberrations were most often located in PIK3CA (n = 18, 15%), ERBB2 (n = 15, 12%), HRAS and NOTCH1 (both n = 9, 7%). Conclusions: Actionable genetic aberrations were seen in 53.7% of all SGC cases on the RNA and DNA level, with varying percentages between subtypes.

PurposeWatch and wait strategy is a safe and effective alternative to surgery in patients with locally advanced rectal cancer (LARC) who have achieved pathological complete response (pCR) after neoadjuvant therapy (NAT); present restaging methods do not meet clinical needs. This study aimed to construct a machine learning (ML) model to predict pCR preoperatively.MethodsLARC patients who received NAT were included to generate an extreme gradient boosting-based ML model to predict pCR. The group was divided into a training set and a tuning set at a 7:3 ratio. The SHapley Additive exPlanations value was used to quantify feature importance. The ML model was compared with a nomogram model developed using independent risk factors identified by conventional multivariate logistic regression analysis.ResultsCompared with the nomogram model, our ML model improved the area under the receiver operating characteristics from 0.72 to 0.95, sensitivity from 43 to 82.2%, and specificity from 87.1 to 91.6% in the training set, the same trend applied to the tuning set. Neoadjuvant radiotherapy, preoperative carbohydrate antigen 125 (CA125), CA199, carcinoembryonic antigen level, and depth of tumor invasion were significant in predicting pCR in both models.ConclusionOur ML model is a potential alternative to the existing assessment tools to conduct triage treatment for patients and provides reference for clinicians in tailoring individual treatment: the watch and wait strategy is used to avoid surgical trauma in pCR patients, and non-pCR patients receive surgical treatment to avoid missing the optimal operation time window.

Well log curves, acquired from downhole logging tools during well logging, are pivotal for reservoir characterization and formation evaluation in oil and gas exploration and production. However, manual feature extraction from raw curves remains essential for constructing effective machine learning models, presenting time-consuming challenges and stringent labeling requirements. Concurrently, the transformer architecture, prevalent in NLP and computer vision, offers promise for representation learning. This paper proposes a self-supervised transformer based methodology for extracting well log curves representations, aiming to expedite downstream model development.While transformer models have gained prominence in handling text and image data, well log curves present a distinct challenge as they resemble time series data. Despite the nascent development of time series transformer models, we conducted an extensive review of current progress and adopted the best-performing time series transformer model for extracting representations from well log curves. Importantly, given the challenges posed by factors such as borehole conditions and instrument failure, certain types of well log curves may occasionally be missing or distorted. To address this issue, our proposed methodology introduces an adaptive masking mechanism, which selectively applies masking to patches of curves where data quality is poor, thereby effectively mitigating data quality concerns.Data from 2000 wells are utilized for model training, with an additional 100 wells reserved for validation purposes. Our study observed a consistent decrease in both training and test losses until convergence during the training stage. Initially, mean squared error (MSE) and mean absolute error (MAE) are employed to quantify reconstruction errors between reconstructed curves and raw curves, low values of MSE (0.08) and MAE (0.07) indicate effectiveness of the learned representations. Subsequently, a downstream task involving oil and gas identification is undertaken, wherein a classification model is developed based on representations learned by the transformer model. Performance comparison between models utilizing learned representation and those employing statistical features highlights the superior performance of the former (98% accuracy), emphasizing the efficacy of our representation learning methodology.&amp;#160;This paper introduces a novel self-supervised methodology based on transformer architecture for well log curve representation learning. The method automates information extraction without requiring logging expertise and substantially enhances downstream machine learning model performance.

Although there are 3 hospice care programs for terminal cancer patients in Taiwan, the medical utilization and expenses for these patients by programs have not been well-explored. The aim of this study was to examine the medical utilization and expenses of terminal cancer patients under different programs of hospice care in the last 90, 30, and 14 days of life.This was a retrospective observational study by secondary data analysis. By using the National Health Insurance claim database and Hospice Shared Care Databases. We identified cancer descents from these databases and classified them into nonhospice care and hospice care groups based on different combination of hospice care received. We then analyzed medical utilization including inpatient care, outpatient care, emergency room visits, and medical expenses by patient groups in the last 90, 30, and 14 days of life.Among 118,376 cancer descents, 46.9% ever received hospice care. Patients had ever received hospice care had significantly lower average medical utilization and expenses in their last 90, 30, and 14 days of life (all P < 0.001) compared to nonhospice care group. Each hospice care group had significantly less medical utilization and expenses in the last 90, 30, and 14 days of life (all P < 0.01).Different kinds of hospice care program have different effects on medical care utilization reduction and cost-saving at different stage of the end of life of terminal cancer patients.

AI foundation models hold considerable promise for leveraging the vast and diverse datasets available in atmospheric and geoscientific research. These models have the potential to advance scientific discovery by capturing complex spatial and temporal relationships inherent in earth system processes. However, the development and deployment of such models is often hindered by limited computational resources.Accurate reconstruction of fine-scale atmospheric features from coarse-resolution data is a critical challenge in geoscientific modeling, as well as a benchmark for understanding the performance of climate-related models. High-resolution atmospheric data are essential for capturing localized phenomena, such as convective systems, topographic effects, and land-atmosphere interactions, that influence weather patterns and climate processes. However, the generation and storage of high-resolution datasets are computationally expensive, necessitating methods that can infer fine-scale structures from lower-resolution observations.The primary objective of this study is to validate PrithviWxC [4], a ViT-based foundation model [1], for the task of downscaled image reconstruction. While Prithvi WxC was trained on 160 atmospheric variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2) dataset [2], we implement a smaller version of the initial model, with 21 million parameters, and pretrained on the same dataset with a set of six variables.&amp;#160;The evaluation process involves the assessment of the model's capacity to reconstruct fine-scale features through the process of downscaling atmospheric data. In this procedure, inputs at a high spatial resolution of 1 km from [5] are first coarsened to 25 km resolution, the same as European Centre of Medium-range Weather Forecasts Reanalysis v.5 (ERA5) [3], and subsequently upscaled to recover the original fine-grained structure. This process serves as a benchmark for assessing the model's capacity to learn and preserve spatial details during resolution transformations, which is an essential requirement for geoscientific modeling tasks.We fine-tune the model for downscaled image reconstruction on a set of 30784 128x128 patches, and validate its output, produced after learning on a limited temporal period, on tiles coming from the ERA5 dataset, which encompass all seasonality. In particular, we aim at highlighting the model's ability to generalize to data domains beyond its pretraining distribution, demonstrating its adaptability and the transferability of knowledge embedded within ViT architectures. By applying PrithviWxC to a knowledge domain that is distinct from its original training context, we demonstrate the potential for cross-domain learning in geoscientific applications.&amp;#160;REFERENCES[1] Dosovitskiy, Alexey. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020).[2] Gelaro, Ronald, et al. "The modern-era retrospective analysis for research and applications, version 2 (MERRA-2)." Journal of climate 30.14 (2017): 5419-5454.[3] Hersbach, Hans, et al. "The ERA5 global reanalysis." Quarterly Journal of the Royal Meteorological Society 146.730 (2020): 1999-2049.[4] Schmude, Johannes, et al. "Prithvi wxc: Foundation model for weather and climate." arXiv preprint arXiv:2409.13598 (2024).[5] Wedi, Nils P., et al. "A baseline for global weather and climate simulations at 1 km resolution." Journal of Advances in Modeling Earth Systems 12.11 (2020): e2020MS002192.

We studied the role of context in reacquisition of extinguished reward-seeking. Rats were trained to respond for alcoholic beer, then extinguished and retrained. Reacquisition was faster than acquisition regardless of whether retraining occurred in the original training context, the extinction context, a novel context, or a context with a mixed history of reinforcement. Reacquisition was also rapid after extended extinction training. Nonetheless, context did significantly influence reacquisition via affecting latency to first response: rats took significantly longer to initiate responding when tested in the extinction context. These results suggest that reacquisition of drug and reward seeking is determined by an inhibitory influence caused by the extinction context and a facilitatory influence caused by reintroduction of the reinforcer (Bouton, 1993).

P2.13-04 Outcomes of ALK-Positive Non-Small-Cell Lung Cancer (NSCLC) Patients Treated with Crizotinib: A Multicenter Cohort Retrospective Study.

Explanation leaks: Explanation-guided model extraction attacks