ReactFormer: A Reaction-Aware Graph Transformer for Barrier Height Prediction without Explicit Transition State Geometries

Supervised deep learning (DL) models are becoming ubiquitous in computational chemistry because they can efficiently learn complex input-output relationships and predict chemical properties at a cost significantly lower than methods based on quantum mechanics. The central challenge in many DL applications is the need to invest considerable computational resources in generating large ( N>1×105 ) training sets such that the resulting DL model can be generalized reliably to unseen systems. The lack of better alternatives has encouraged the use of low-cost and relatively inaccurate density-functional theory (DFT) methods to generate training data, leading to DL models that lack accuracy and reliability. In this article, we describe a robust and easily implemented approach based on property-specific atom-centered potentials (ACPs) that resolves this central challenge in DL model development. ACPs are one-electron potentials that are applied in combination with a computationally inexpensive but inaccurate quantum mechanical method (e.g. double-ζ DFT) and fitted against relatively few high-level data ( N≈1×103 – 1×104 ), possibly obtained from the literature. The resulting ACP-corrected methods retain the low cost of the double-ζ DFT approach, while generating high-level-quality data in unseen systems for the specific property for which they were designed. With this approach, we demonstrate that ACPs can be used as an intermediate method between high-level approaches and DL model development, enabling the calculation of large and accurate DL training sets for the chemical property of interest. We demonstrate the effectiveness of the proposed approach by predicting bond dissociation enthalpies, reaction barrier heights, and reaction energies with chemical accuracy at a computational cost lower than the DFT methods routinely used for DL training data set generation.

Lyme disease is one of the most common vector-borne infections. It typically causes cardiac illnesses, neurologic illnesses, musculoskeletal disorders, and dermatologic conditions. However, most of the time, it is poorly diagnosed due to many similarities with other diseases such as drug rash. Given the potentially serious consequences of unnecessary antimicrobial treatments, it is essential to understand frequent and uncommon diagnoses that explain symptoms in this population. Recently, deep learning models have been used for the diagnosis of various rash-related diseases. However, these models suffer from overfitting and color variation problems. To overcome these problems, an efficient stacked deep transfer learning model is proposed that can efficiently distinguish between patients infected with Lyme (+) or infected with other infections. 2nd order edge-based color constancy is used as a preprocessing approach to reduce the impact of multisource light from images acquired under different setups. The AlexNet pretrained learning model is used for building the Lyme disease diagnosis model. To prevent overfitting, data augmentation techniques are also used to augment the dataset. In addition, 5-fold cross-validation is also used. Comparative analysis indicates that the proposed model outperforms the existing models in terms of accuracy, f-measure, sensitivity, specificity, and area under the curve.

Exponential growth in data rate demands has driven efforts to develop novel beamforming techniques for realizing massive multiple-input and multiple-output (MIMO) systems in sixth-generation (6G) terabits per second wireless communications. Existing beamforming techniques rely on conventional optimization algorithms that are too computationally expensive for real-time applications and require complex digital processing yet to be achieved for phased array antennas at terahertz frequencies. Here, we develop an intelligent and self-adaptive beamforming scheme enabled by deep reinforcement learning, which can predict the spatial phase profiles required to produce arbitrary desired radiation patterns in real-time. Our deep learning model adaptively trains an artificial neural network in real-time by comparing the input and predicted intensity patterns via automatic differentiation of the phase-to-intensity function. As a proof of concept, we experimentally demonstrate two-dimensional beamforming by spatially modulating broadband terahertz waves using silicon metasurfaces designed with the aid of the deep learning model. Our work offers an efficient and robust deep learning model for real-time self-adaptive beamforming to enable multi-user massive MIMO systems for 6G terahertz wireless communications, as well as intelligent metasurfaces for other terahertz applications in imaging and sensing.

Incorporating self-interaction corrections (SIC) significantly improves chemical reaction barrier height predictions made using density functional theory methods. We present a detailed orbital-by-orbital analysis of these corrections for three semi-local density functional approximations (DFAs) situated on the three lowest rungs of Jacob's ladder of approximations. The analysis is based on Fermi-Löwdin Orbital Self-Interaction Correction (FLOSIC) calculations performed at several steps along the reaction pathway from the reactants (R) to the transition state (TS) to the products (P) for four representative reactions selected from the BH76 benchmark set. For all three functionals, the major contribution to self-interaction corrections of the barrier heights can be traced to stretched bond orbitals that develop near the TS configuration. The magnitude of the ratio of the self-exchange-correlation energy to the self-Hartree energy (XC/H) for a given orbital is introduced as an indicator of one-electron self-interaction error. XC/H = 1.0 implies that an orbital's self-exchange-correlation energy exactly cancels its self-Hartree energy and that the orbital, therefore, makes no contribution to the SIC in the FLOSIC scheme. For the practical DFAs studied here, XC/H spans a range of values. The largest values are obtained for stretched or strongly lobed orbitals. We show that significant differences in XC/H for corresponding orbitals in the R, TS, and P configurations can be used to identify the major contributors to the SIC of barrier heights and reaction energies. Based on such comparisons, we suggest that barrier height predictions made using the strongly constrained and appropriately normed meta-generalized gradient approximation may have attained the best accuracy possible for a semi-local functional using the Perdew-Zunger SIC approach.

The adverse effect of air pollution has always been a problem for human health. The presence of a high level of air pollutants can cause severe illnesses such as emphysema, chronic obstructive pulmonary disease (COPD), or asthma. Air quality prediction helps us to undertake practical action plans for controlling air pollution. The Air Quality Index (AQI) reflects the degree of concentration of pollutants in a locality. The average AQI was calculated for the various cities in China to understand the annual trends. Furthermore, the air quality index has been predicted for ten major cities across China using five different deep learning techniques, namely, Recurrent Neural Network (RNN), Bidirectional Gated Recurrent unit (Bi-GRU), Bidirectional Long Short-Term Memory (BiLSTM), Convolutional Neural Network BiLSTM (CNN-BiLSTM), and Convolutional BiLSTM (Conv1D-BiLSTM). The performance of these models has been compared with a machine learning model, eXtreme Gradient Boosting (XGBoost) to discover the most efficient deep learning model. The results suggest that the machine learning model, XGBoost, outperforms the deep learning models. While Conv1D-BiLSTM and CNN-BiLSTM perform well among the deep learning models in the estimation of the air quality index (AQI), RNN and Bi-GRU are the least performing ones. Thus, both XGBoost and neural network models are capable of capturing the non-linearity present in the dataset with reliable accuracy.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

This research paper explores the rising interest in utilising deep learning methods, specifically Convolution Neural Network (CNN), to improve the reconstruction of Compressively Sampled Magnetic Resonance Imaging (CS MRI) images from under-sampled data. By training deep learning architectures on extensive data sets of paired under sampled and fully sampled MR images, these models aim to capture intricate patterns and structures, ultimately enhancing the accuracy of MR image reconstructions. A novel deep learning model is proposed dubbed as “Deep Iterative Shrinkage Thresholding Algorithm-Compressed Sensing Network” (DISTA-CSNet), specifically designed for efficient recovery of CS MRI. Our model showcases impressive results with only 20 epochs and can be effortlessly trained on diverse datasets. To ensure robustness across different datasets, the dropouts are incorporated into the model, evident from testing results. The trained DISTA-CSNet exhibits remarkable performance in recovering CS MRI from various data sets, surpassing several advanced deep learning techniques with changing compression ratios consistently. The proposed model demonstrates significant enhancements in both Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) metrics, affirming the efficacy of our proposed model. The capability of our DISTA-CSNet in accurately reconstructing CS MRI images from 5-fold undersampled data shows promise in improving medical imaging applications and advancing the field of compressed sensing MRI.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Urban traffic passenger flows prediction is practically important to facilitate many real applications including transportation management and public safety. Recently, deep learning based approaches are proposed to learn the spatio-temporal characteristics of the traffic passenger flows. However, it is still very challenging to handle some complex factors such as hybrid transportation lines, mixed traffic, transfer stations, and some extreme weathers. Considering the multi-channel and irregularity properties of urban traffic passenger flows in different transportation lines, a more efficient and fine-grained deep spatio-temporal feature learning model is necessary. In this paper, we propose a deep irregular convolutional residual LSTM network model called DST-ICRL for urban traffic passenger flows prediction. We first model the passenger flows among different traffic lines in a transportation network into multi-channel matrices analogous to the RGB pixel matrices of an image. Then, we propose a deep learning framework that integrates irregular convolutional residential network and LSTM units to learn the spatial-temporal feature representations. To fully utilize the historical passenger flows, we sample both the short-term and long-term historical traffic data, which can capture the periodicity and trend of the traffic passenger flows. In addition, we also fuse other external factors further to facilitate a real-time prediction. We conduct extensive experiments on different types of traffic passenger flows datasets including subway, taxi and bus flows in Beijing as well as bike flows in New York. The results show that the proposed DST-ICRL significantly outperforms both traditional and deep learning based urban traffic passenger flows prediction methods.

Predicting the performance of deep learning (DL) models, such as execution time and resource utilization, is crucial for Neural Architecture Search (NAS), DL cluster schedulers, and other technologies that advance deep learning. The representation of a model is the foundation for its performance prediction. However, existing methods cannot comprehensively represent diverse model configurations, resulting in unsatisfactory accuracy. To address this, we represent a model as a graph that includes the topology, along with node, edge, and global features, all of which are crucial for effectively capturing the performance of the model. Based on this representation, we propose PerfSeer, a novel predictor that uses a Graph Neural Network (GNN)-based performance prediction model, SeerNet. SeerNet fully leverages the topology and various features, while incorporating optimizations such as Synergistic Max-Mean aggregation (SynMM) and Global-Node Perspective Boost (GNPB) to more effectively capture the critical performance information, enabling it to predict the performance of models accurately. Furthermore, SeerNet can be extended to SeerNet-Multi by using Project Conflicting Gradients (PCGrad), enabling efficient simultaneous prediction of multiple performance metrics without significantly affecting accuracy. We constructed a dataset containing performance metrics for 53k+ model configurations, including execution time, memory usage, and Streaming Multiprocessor (SM) utilization during both training and inference. The evaluation results show that PerfSeer outperforms nn-Meter, Brp-NAS, and DIPPM.

ATP13A2 is a gene encoding a protein of the P5B subfamily of ATPases and is a PARK gene. Molecular defects of the gene are mainly associated with variations of Parkinson's disease (PD). Despite the established importance of the protein in regulating neuronal integrity, the three-dimensional structure of the protein currently remains unresolved crystallographically. We have modeled the structure and reactivity of the full-length protein in its E1-ATP state. Using molecular dynamics (MD), quantum cluster, and quantum mechanical/molecular mechanical (QM/MM) methods, we aimed at describing the main catalytic reaction, leading to the phosphorylation of Asp513. Our MD simulations suggest that two positively charged Mg2+ cations are present at the active site during the catalytic reaction, stabilizing a specific triphosphate binding mode. Using QM/MM calculations, we subsequently calculated the reaction profiles for the phosphoryl transfer step in the presence of one and two Mg2+ cations. The calculated barrier heights in both cases are found to be ∼12.5 and 7.5 kcal mol-1, respectively. We elucidated details of the catalytically competent ATP conformation and the binding mode of the second Mg2+ cofactor. We also examined the role of the conserved Arg686 and Lys859 catalytic residues. We observed that by significantly lowering the barrier height of the ATP cleavage reaction, Arg686 had major effect on the reaction. The removal of Arg686 increased the barrier height for the ATP cleavage by more than 5.0 kcal mol-1 while the removal of key electrostatic interactions created by Lys859 to the γ-phosphate and Asp513 destabilizes the reactant state. When missense mutations occur in close proximity to an active site residue, they can interfere with the barrier height of the reaction, which can halt the normal enzymatic rate of the protein. We also found large binding pockets in the full-length structure, including a transmembrane domain pocket, which is likely where the ATP13A2 cargo binds.

The timely and accurate detection of potato pests is crucial for mitigating crop losses and supporting sustainable agriculture. This study presents an efficient deep learning model based on the VGG16 architecture for the automated classification of common potato leaf pests, including aphids, Colorado potato beetles, leafminers, and ladybird beetles. A curated dataset of 3,000 annotated images, comprising both pest-infected and healthy leaves, was compiled from authentic field images directly from rural agricultural areas. To enhance generalization, the model was fine-tuned using transfer learning and optimized with data augmentation techniques, including rotation, flipping, zooming, and contrast adjustment. Experimental results demonstrate that the proposed VGG16-based model achieves superior performance across accuracy (96.3%), precision (95.8%), recall (96.1%), and F1-score (95.9%) metrics, outperforming baseline architectures including ResNet50, InceptionV3, MobileNetV2, and EfficientNetB0. Additional analyses, such as confusion matrices, ROC-AUC curves, and class-wise evaluations, confirm the model's robustness in real-world scenarios. With an average inference time of 45 ms per image on a Kaggle environment (e.g., Tesla P100 or T4 GPU) and successful deployment potential on edge devices, the proposed model offers a scalable, lightweight and practical solution for real-time pest monitoring in precision agriculture. Future work will focus on expanding pest diversity, improving model interpretability, and integrating the system into IoT-enabled smart farming frameworks.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

We developed a multimodal ultrasound (US) deep learning (DL) fusion model to automatically classify early fibrosis in patients with chronic kidney disease (CKD). This prospective study included patients with CKD who underwent continuous gray-scale US, superb microvascular imaging, and strain elastography from May to November 2022. According to the pathological tubular atrophy and interstitial fibrosis score, patients were divided into minimal and mild groups (affected area ≤10% and 11 − 25% of the total cortical volume, respectively). The dataset was divided into training (70%) and test (30%) sets. A DL model combining the features of the three US modes was developed to predict early fibrosis in patients with CKD. We compared these findings with the area under the receiver operating characteristic curve (AUC) of the clinical model by analyzing the receiver operating characteristic curve in the test set. The AUC of single-mode DL based on gray-scale US, superb microvascular imaging, and strain elastography was 0.682, 0.745, and 0.648, respectively, while that of the multimodal US DL model was 0.86. The accuracy, specificity, and sensitivity of the multimodal US DL model were 0.779, 0.767, and 0.796, respectively, and the negative and positive predictive values were 0.842 and 0.706, respectively. The AUC of the multimodal US DL model was significantly better than that of the single-mode DL and clinical models. The DL algorithm developed using multimodal US images can effectively predict early fibrosis in patients with CKD with significantly greater accuracy than single-mode DL or clinical models.

We aimed at comparing intratumoral and peritumoral deep learning, radiomics, and fusion models in predicting KRAS mutations in rectal cancer using endorectal ultrasound imaging. This study included 304 patients with rectal cancer from Fujian Medical University Union Hospital. The patients were randomly divided into a training group (213 patients) and a test group (91 patients) at a 7:3 ratio. Radiomics and deep learning models were established using primary tumor and peritumoral images. In the optimally performing regions-of-interest, two fusion strategies, a feature-based and a decision-based model, were employed to build the fusion models. The Shapley additive explanation (SHAP) method was used to evaluate the significance of features in the optimal radiomics, deep learning, and fusion models. The performance of each model was assessed using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). In the test cohort, both the radiomics and deep learning models exhibited optimal performance with a 10-pixel patch extension, yielding AUC values of 0.824 and 0.856, respectively. The feature-based DLRexpand10_FB model attained the highest AUC (0.896) across all study sets. In addition, the DLRexpand10_FB model demonstrated excellent sensitivity, specificity, and DCA. SHAP analysis underscored the deep learning feature (DL_1) as the most significant factor in the hybrid model. The feature-based fusion model DLRexpand10_FB can be employed to predict KRAS gene mutations based on pretreatment endorectal ultrasound images of rectal cancer. The integration of peritumoral regions enhanced the predictive performance of both the radiomics and deep learning models.