Bonn Aachen International Center for Information Technology

Chemical language models (CLMs), particularly encoder‐decoder transformers, have advanced generative molecular design. Transformer CLMs are able to learn a variety of molecular mappings for compound design that can be conditioned using context‐dependent rules. However, their black‐box nature complicates the interpretation of predictions. Current analysis methods mostly focus on attention weights of token relationships or attention flow in encoder and decoder modules and cannot explain predictions at the molecular level. Sequence‐based compound design was used as a model system to investigate transformer learning characteristics through systematic control calculations involving modifications of protein sequences and sequence‐compound pairs. The analysis revealed that compound reproducibility depended on similarity relationships between training and test data and on compound memorization, while specific sequence information was not learned. These findings indicate that predictions of transformer CLMs are driven by memorization effects and statistical correlations rather than by learning specific chemical or biological information. Understanding this learning behavior aids in avoiding over‐interpretation of model outputs and informs the appropriate application of transformer‐based CLMs in molecular design.

BackgroundDementia poses an increasingly significant global health challenge, with projections indicating a dramatic rise in the number of affected individuals over the coming decades. This growing prevalence highlights the urgent need for early detection and intervention strategies. The 2024 Lancet Commission on dementia prevention, intervention, and care identified key risk factors for dementia, emphasizing the importance of early identification of high‐risk individuals to facilitate timely interventions that could potentially alter the disease's trajectory. These common risk factors can typically be gathered during routine general practitioner (GP) visits and do not necessitate specialized testing.MethodWe developed and compared various machine learning models, including multi‐state‐models, random survival forest, CatBoost, and DeepHit, to predict the age dependent risk of a future dementia diagnosis (Figure 1). Our models were adjusted for the competing risk of deaths and trained on retrospective data from UK biobank, with external validation on data from the HUNT population study. All models incorporated common dementia risk factors as recommended by the Lancet Commission and were benchmarked against the previously published LIBRA score.ResultOur preliminary findings suggest that machine learning‐based risk models can accurately predict the likelihood of a future dementia diagnosis, achieving a 10‐fold cross‐validated area under the receiver operating characteristic curve (AUC) of 0.785 and 0.776 after 3 and 5 years, respectively.ConclusionThe models we developed can function as effective screening tools for estimating the risk of developing dementia 3 to 5 years in the future, utilizing easily accessible features at the GP level. The strength of these models lies in their incorporation of common risk factors that are routinely collected or readily available during standard GP visits. This approach represents a practical and scalable method for the early detection and prevention of dementia. By leveraging readily available data, these models could serve as valuable resources in primary care settings, potentially alleviating the global burden of dementia through timely interventions and personalized risk management strategies.AcknowledgmentThis Project Is Supported By The Innovative Health Initiative Joint Undertaking (IHIJU) Under Grant Agreement No 101132356. The JU Receives Support From The European Union's Horizon Europe Research And Innovation Programme. This Work Was Funded By UK Research AndInnovation (UKRI) Under The UK Government's Horizon Europe Funding Guarantee [UKRI Reference Number: 10083181]. In Switzerland The University Of Geneva Is Funded For PREDICTOM By The Swiss State Secretariat For Education Research And Innovation (SERI‐ Ref‐1131 52304).

Abstract Electronic Health Records (EHRs) offer rich data for machine learning, but model generalizability across institutions is hindered by statistical and coding biases. This study investigates domain adaptation (DA) techniques to improve model transfer, focusing on the Ex-Med-BERT transformer architecture for structured EHR data. We compare supervised and un-supervised DA methods in transferring predictive capabilities from the large-scale IBM Explorys database (U.S.) to the UK Biobank. Results across six clinical endpoints show that DA methods outperform fine-tuning, especially with limited target data. These findings emphasize selecting DA strategies based on target data availability and the benefit of incorporating source domain data for robust adaptation.

SummaryThe use of synthetic data is a widely discussed and promising solution for privacy-preserving medical research. Synthetic data may however not always be privacy preserving and can vary greatly in terms of data fidelity and utility.We systematically evaluate the trade-offs between privacy, fidelity, and utility across five synthetic data models and three patient-level datasets. We evaluate fidelity based on statistical similarity to the real data, utility on three machine learning use cases and privacy via membership inference, singling out, and attribute inference risks. Synthetic data without differential privacy (DP) maintained fidelity and utility without evident privacy breaches, whereas DP-enforced models significantly disrupted correlation structures. K-anonymity-based data sanitization, while preserving fidelity, introduced notable privacy risks. Our findings emphasize the need to advance methods that effectively balance privacy, fidelity, and utility in synthetic patient data generation.HighlightsDifferential Privacy (DP) had a detrimental effect on feature correlationsModels that did not implement DP showed good fidelity compared to real dataNon-DP synthetic models showed no strong evidence of privacy breachesk-anonymization produced high fidelity data but showed notable privacy risks

During the COVID-19 pandemic, many hospitals reached their capacity limits and could no longer guarantee treatment of all patients. At the same time, governments endeavored to take sensible measures to stop the spread of the virus while at the same time trying to keep the economy afloat. Many models extrapolating confirmed cases and hospitalization rate over short periods of time have been proposed, including several ones coming from the field of machine learning. However, the highly dynamic nature of the pandemic with rapidly introduced interventions and new circulating variants imposed non-trivial challenges for the generalizability of such models. In the context of this paper, we propose the use of ensemble models, which are allowed to change in their composition or weighting of base models over time and could thus better adapt to highly dynamic pandemic or epidemic situations. In that regard, we also explored the use of secondary metadata-Google searches-to inform the ensemble model. We tested our approach using surveillance data from COVID-19, Influenza, and hospital syndromic surveillance of severe acute respiratory infections (SARI). In general, we found ensembles to be more robust than the individual models. Altogether we see our work as a contribution to enhance the preparedness for future pandemic situations.

Deep neural networks (DNN) have fundamentally revolutionized the artificial intelligence (AI) field. The transformer model is a type of DNN that was originally used for the natural language processing tasks and has since gained more and more attention for processing various kinds of sequential data, including biological sequences and structured electronic health records. Along with this development, transformer-based models such as BioBERT, MedBERT, and MassGenie have been trained and deployed by researchers to answer various scientific questions originating in the biomedical domain. In this paper, we review the development and application of transformer models for analyzing various biomedical-related datasets such as biomedical textual data, protein sequences, medical structured-longitudinal data, and biomedical images as well as graphs. Also, we look at explainable AI strategies that help to comprehend the predictions of transformer-based models. Finally, we discuss the limitations and challenges of current models, and point out emerging novel research directions.

The drug discovery community faces high costs in bringing safe and effective medicines to market, in part due to the rising volume and complexity of data which must be generated during the research and development process. Fully utilising these expensively created experimental and computational data resources has become a key aim of scientists due to the clear imperative to leverage the power of artificial intelligence (AI) and machine learning-based analyses to solve the complex problems inherent in drug discovery. In turn, AI methods heavily rely on the quantity, quality, consistency, and scope of underlying training data. While pre-existing preclinical and clinical data cannot fully replace the need for de novo data generation in a project, having access to relevant historical data represents a valuable asset, as its reuse can reduce the need to perform similar experiments, therefore avoiding a “reinventing the wheel” scenario. Unfortunately, most suitable data resources are often archived within institutes, companies, or individual research groups and hence unavailable to the wider community. Hence, enabling the data to be Findable, Accessible, Interoperable, and Reusable (FAIR) is crucial for the wider community of drug discovery and development scientists to learn from the work performed and utilise the findings to enhance comprehension of their own research outcomes. In this mini-review, we elucidate the utility of FAIR data management across the drug discovery pipeline and assess the impact such FAIR data has made on the drug development process.

<div>AbstractPurpose:<p>Therapy resistance and fatal disease progression in glioblastoma are thought to result from the dynamics of intra-tumor heterogeneity. This study aimed at identifying and molecularly targeting tumor cells that can survive, adapt, and subclonally expand under primary therapy.</p>Experimental Design:<p>To identify candidate markers and to experimentally access dynamics of subclonal progression in glioblastoma, we established a discovery cohort of paired vital cell samples obtained before and after primary therapy. We further used two independent validation cohorts of paired clinical tissues to test our findings. Follow-up preclinical treatment strategies were evaluated in patient-derived xenografts.</p>Results:<p>We describe, in clinical samples, an archetype of rare ALDH1A1+ tumor cells that enrich and acquire AKT-mediated drug resistance in response to standard-of-care temozolomide (TMZ). Importantly, we observe that drug resistance of ALDH1A1+ cells is not intrinsic, but rather an adaptive mechanism emerging exclusively after TMZ treatment. In patient cells and xenograft models of disease, we recapitulate the enrichment of ALDH1A1+ cells under the influence of TMZ. We demonstrate that their subclonal progression is AKT-driven and can be interfered with by well-timed sequential rather than simultaneous antitumor combination strategy.</p>Conclusions:<p>Drug-resistant ALDH1A1+/pAKT+ subclones accumulate in patient tissues upon adaptation to TMZ therapy. These subclones may therefore represent a dynamic target in glioblastoma. Our study proposes the combination of TMZ and AKT inhibitors in a sequential treatment schedule as a rationale for future clinical investigation.</p></div>

The anticipation of progression of Alzheimer’s disease (AD) is crucial for evaluations of secondary prevention measures thought to modify the disease trajectory. However, it is difficult to forecast the natural progression of AD, notably because several functions decline at different ages and different rates in different patients. We evaluate here AD Course Map, a statistical model predicting the progression of neuropsychological assessments and imaging biomarkers for a patient from current medical and radiological data at early disease stages. We tested the method on more than 96,000 cases, with a pool of more than 4,600 patients from four continents. We measured the accuracy of the method for selecting participants displaying a progression of clinical endpoints during a hypothetical trial. We show that enriching the population with the predicted progressors decreases the required sample size by 38% to 50%, depending on trial duration, outcome, and targeted disease stage, from asymptomatic individuals at risk of AD to subjects with early and mild AD. We show that the method introduces no biases regarding sex or geographic locations and is robust to missing data. It performs best at the earliest stages of disease and is therefore highly suitable for use in prevention trials.

Purpose:Therapy resistance and fatal disease progression in glioblastoma are thought to result from the dynamics of intra-tumor heterogeneity. This study aimed at identifying and molecularly targeting tumor cells that can survive, adapt, and subclonally expand under primary therapy.Experimental Design:To identify candidate markers and to experimentally access dynamics of subclonal progression in glioblastoma, we established a discovery cohort of paired vital cell samples obtained before and after primary therapy. We further used two independent validation cohorts of paired clinical tissues to test our findings. Follow-up preclinical treatment strategies were evaluated in patient-derived xenografts.Results:We describe, in clinical samples, an archetype of rare ALDH1A1+ tumor cells that enrich and acquire AKT-mediated drug resistance in response to standard-of-care temozolomide (TMZ). Importantly, we observe that drug resistance of ALDH1A1+ cells is not intrinsic, but rather an adaptive mechanism emerging exclusively after TMZ treatment. In patient cells and xenograft models of disease, we recapitulate the enrichment of ALDH1A1+ cells under the influence of TMZ. We demonstrate that their subclonal progression is AKT-driven and can be interfered with by well-timed sequential rather than simultaneous antitumor combination strategy.Conclusions:Drug-resistant ALDH1A1+/pAKT+ subclones accumulate in patient tissues upon adaptation to TMZ therapy. These subclones may therefore represent a dynamic target in glioblastoma. Our study proposes the combination of TMZ and AKT inhibitors in a sequential treatment schedule as a rationale for future clinical investigation.