EvoStore: Towards Scalable Storage of Evolving Learning Models

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

This paper presents a Data Science-oriented application for image classification tasks that is able to automatically: a) gather images needed for training Deep Learning (DL) models with a built-in search engine crawler; b) remove duplicate images; c) sort images using built-in pre-trained DL models or user’s own trained DL model; d) apply data augmentation; e) train a DL classification model; f) evaluate the performance of a DL model and system by using an accuracy calculator as well as the Accuracy Per Consumption (APC), Accuracy Per Energy Cost (APEC), Time to closest APC (TTCAPC) and Time to closest APEC (TTCAPEC) metrics calculators. Experimental results show that the proposed Computer Vision application has several unique features and advantages, proving to be efficient regarding execution time and much easier to use when compared to similar applications.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

Research in many fields has shown that transfer learning (TL) is well-suited to improve the performance of deep learning (DL) models in datasets with small numbers of samples. This empirical success has triggered interest in the application of TL to cognitive decoding analyses with functional neuroimaging data. Here, we systematically evaluate TL for the application of DL models to the decoding of cognitive states (e.g., viewing images of faces or houses) from whole-brain functional Magnetic Resonance Imaging (fMRI) data. We first pre-train two DL architectures on a large, public fMRI dataset and subsequently evaluate their performance in an independent experimental task and a fully independent dataset. The pre-trained DL models consistently achieve higher decoding accuracies and generally require less training time and data than model variants that were not pre-trained, while also outperforming linear baseline models trained from scratch, clearly underlining the benefits of pre-training. We demonstrate that these benefits arise from the ability of the pre-trained models to reuse many of their learned features when training with new data, providing deeper insights into the mechanisms giving rise to the benefits of pre-training. Yet, we also surface nuanced challenges for whole-brain cognitive decoding with DL models when interpreting the decoding decisions of the pre-trained models, as these have learned to utilize the fMRI data in unforeseen and counterintuitive ways to identify individual cognitive states.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

We aimed at comparing intratumoral and peritumoral deep learning, radiomics, and fusion models in predicting KRAS mutations in rectal cancer using endorectal ultrasound imaging. This study included 304 patients with rectal cancer from Fujian Medical University Union Hospital. The patients were randomly divided into a training group (213 patients) and a test group (91 patients) at a 7:3 ratio. Radiomics and deep learning models were established using primary tumor and peritumoral images. In the optimally performing regions-of-interest, two fusion strategies, a feature-based and a decision-based model, were employed to build the fusion models. The Shapley additive explanation (SHAP) method was used to evaluate the significance of features in the optimal radiomics, deep learning, and fusion models. The performance of each model was assessed using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). In the test cohort, both the radiomics and deep learning models exhibited optimal performance with a 10-pixel patch extension, yielding AUC values of 0.824 and 0.856, respectively. The feature-based DLRexpand10_FB model attained the highest AUC (0.896) across all study sets. In addition, the DLRexpand10_FB model demonstrated excellent sensitivity, specificity, and DCA. SHAP analysis underscored the deep learning feature (DL_1) as the most significant factor in the hybrid model. The feature-based fusion model DLRexpand10_FB can be employed to predict KRAS gene mutations based on pretreatment endorectal ultrasound images of rectal cancer. The integration of peritumoral regions enhanced the predictive performance of both the radiomics and deep learning models.

Accurate rainfall forecasting is crucial in activities such as agricultural planning, water resource management, and flood preparedness, so failure to accurately forecast rainfall might undermine these activities. This study aimed to improve the accuracy of daily rainfall prediction in Hai district, Kilimanjaro region, Tanzania, a region characterised by a bimodal rainfall pattern, by comparing rainfall prediction accuracies of Traditional Machine Learning (TML) and Deep Learning (DL) models. The 25-year daily rainfall dataset for Hai district was first pre-processed by cleaning, normalisation, and splitting into training (first 60%), validation (next 20%), and test (last 20%) sets. TML models (K-Nearest Neighbours (KNN) through its variant KNN Regressor and Support Vector Machine (SVM) through its variant Support Vector Regressor (SVR)) were developed, trained, and their performances compared with developed and trained DL models (Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU)). Each model’s test performance was evaluated using the Root Mean Squared Error (RMSE) metric. The findings revealed that the DL models, LSTM and GRU, achieved test RMSE scores of 6.35 mm and 6.49 mm, respectively, while the TML models, KNN and SVM, achieved test RMSE scores of 6.61 mm and 10.42 mm, respectively. The study demonstrated that DL models are more effective for accurate rainfall forecasting than TML models in regions with complex rainfall patterns. The insights gained will guide the development of more reliable, location-specific weather prediction systems, ultimately supporting informed decision-making in agriculture, environmental management, and flood risk reduction in Tanzania and other regions facing similar climatic challenges

Skin diseases impact millions of people around the world and pose a severe risk to public health. These diseases have a wide range of effects on the skin’s structure, functionality, and appearance. Identifying and predicting skin diseases are laborious processes that require a complete physical examination, a review of the patient’s medical history, and proper laboratory diagnostic testing. Additionally, it necessitates a significant number of histological and clinical characteristics for examination and subsequent treatment. As a disease’s complexity and quantity of features grow, identifying and predicting it becomes more challenging. This research proposes a deep learning (DL) model utilizing transfer learning (TL) to quickly identify skin diseases like chickenpox, measles, and monkeypox. A pre-trained VGG16 is used for transfer learning. The VGG16 can identify and predict diseases more quickly by learning symptom patterns. Images of the skin from the four classes of chickenpox, measles, monkeypox, and normal are included in the dataset. The dataset is separated into training and testing. The experimental results performed on the dataset demonstrate that the VGG16 model can identify and predict skin diseases with 93.29% testing accuracy. However, the VGG16 model does not explain why and how the system operates because deep learning models are black boxes. Deep learning models’ opacity stands in the way of their widespread application in the healthcare sector. In order to make this a valuable system for the health sector, this article employs layer-wise relevance propagation (LRP) to determine the relevance scores of each input. The identified symptoms provide valuable insights that could support timely diagnosis and treatment decisions for skin diseases.

Deep learning (DL) model training must address the memory bottleneck to continue scaling. Processing-in-memory approaches can be a viable solution as they move computations near or into the memory, reducing substantial data movement. However, to deploy applications on such hardware, end-to-end software support is crucial for efficient computation mapping and scheduling as well as extensible code generation, but no consideration has been made for DL training workloads. In this paper, we propose XLA-NDP, a compiler and runtime solution for NDPX, a near-data processing (NDP) architecture integrated with an existing DL training framework. XLA-NDP offloads NDPX kernels and schedules them to overlap with GPU kernels to maximize parallelism based on GPU and NDPX costs, while providing a template-based code generator with low-level optimizations. The experiments showed that XLA-NDP provides up to 1.41x speedup (1.24x on average) over the GPU baseline for four DL model training.

Deep learning breast cancer risk models demonstrate improved accuracy compared with traditional risk models but have not been prospectively tested. We compared the accuracy of a deep learning risk score derived from the patient's prior mammogram to traditional risk scores to prospectively identify patients with cancer in a cohort due for screening. We collected data on 119 139 bilateral screening mammograms in 57 617 consecutive patients screened at 5 facilities between September 18, 2017, and February 1, 2021. Patient demographics were retrieved from electronic medical records, cancer outcomes determined through regional tumor registry linkage, and comparisons made across risk models using Wilcoxon and Pearson χ2 2-sided tests. Deep learning, Tyrer-Cuzick, and National Cancer Institute Breast Cancer Risk Assessment Tool (NCI BCRAT) risk models were compared with respect to performance metrics and area under the receiver operating characteristic curves. Cancers detected per thousand patients screened were higher in patients at increased risk by the deep learning model (8.6, 95% confidence interval [CI] = 7.9 to 9.4) compared with Tyrer-Cuzick (4.4, 95% CI = 3.9 to 4.9) and NCI BCRAT (3.8, 95% CI = 3.3 to 4.3) models (P &lt; .001). Area under the receiver operating characteristic curves of the deep learning model (0.68, 95% CI = 0.66 to 0.70) was higher compared with Tyrer-Cuzick (0.57, 95% CI = 0.54 to 0.60) and NCI BCRAT (0.57, 95% CI = 0.54 to 0.60) models. Simulated screening of the top 50th percentile risk by the deep learning model captured statistically significantly more patients with cancer compared with Tyrer-Cuzick and NCI BCRAT models (P &lt; .001). A deep learning model to assess breast cancer risk can support feasible and effective risk-based screening and is superior to traditional models to identify patients destined to develop cancer in large screening cohorts.

To develop and evaluate static ultrasound images of ovaries-based radiomics and deep learning models for predicting antral follicle count (AFC) and anti-Müllerian hormone (AMH). A retrospective analysis was performed on 395 infertility patients who completed AFC through transvaginal ultrasonography and AMH test in two branches of the hospital. The data set was divided into training cohort (n = 210, Branch A), internal test cohort (n = 91, Branch A), and external test cohort (n = 94, Branch B). The AFC results of static ultrasound images (AFCstatic) and dynamic scan (AFCdynamic) were analysed for correlation and consistency. The radiomics, deep learning and combined models (Rad-DL model) were developed to quantitatively predict AFC and AMH respectively, and the ordered 3-class classification (AFCdynamic≤5,6≤AFCdynamic≤19 and 20≤AFCdynamic) prediction was further carried out. There was a high degree of correlation and consistency between AFCstatic and AFCdynamic in both branches. In the quantitative prediction of AFCdynamic, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.572, 0.541, 0.583, respectively. In the quantitative prediction of AMH, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.420, 0.352, 0.509, respectively. For the ordered 3-class classification prediction, the radiomics, deep learning and Rad-DL models demonstrated a good discrimination performance with overall AUCs of 0.816 (95%CI, 0.747–0.887), 0.843 (95%CI, 0.772–0.901), and 0.881 (95%CI, 0.816–0.931) in the external test cohort, respectively. Radiomics and deep learning models developed by using static ultrasound images of ovaries may quantitatively predict AFCdynamic to a certain extent, and accurately screen infertility patients with different ovarian reserve levels. This study lays the foundation for further break through the rigid antral follicle count and explore an intelligent non-invasive ovarian reserve evaluation method.

Brain tumors are among the most common and fatal types of cancer.Accurate and timely diagnosis of these tumors is essential for disease management and successful patient prognosis.Additionally, the precise identification of the tumor type is crucial in determining the treatment path.By recognizing the type of tumor, doctors can select the most appropriate treatment method, which may include surgery, radiation therapy, chemotherapy, or a combination of these methods.Furthermore, the tumor type helps predict disease progression and the quality of life post-treatment.In recent years, deep learning has been a powerful tool for various image-processing tasks, including brain tumor detection.In this paper, various deep learning models such as CNN, RNN, VGG16, InceptionV3, and ResNet101 are evaluated for classifying brain tumor types from the Figshare dataset based on MRI images of glioma, meningioma, pituitary tumors, and no tumor.Finally, a suitable deep model based on ResNet101 combined with transfer learning is proposed.Based on various metrics and statistical tests, the findings indicate that deep learning models can be effectively used for brain tumor detection.Among these, the ResNet101 model achieved an accuracy of 98.37% in classifying the four tumor classes.This study demonstrates that deep learning holds significant potential for improving brain tumor detection accuracy.

Artificial intelligence (AI) has the ability to predict rheological properties and constituent composition of 3D-printed materials with appropriately trained models. However, these models are not currently available for use. In this work, we trained deep learning (DL) models to (1) predict the rheological properties, such as the storage (G’) and loss (G”) moduli, of 3D-printed polyacrylamide (PAA) substrates, and (2) predict the composition of materials and associated 3D printing parameters for a desired pair of G’ and G”. We employed a multilayer perceptron (MLP) and successfully predicted G’ and G” from seven gel constituent parameters in a multivariate regression process. We used a grid-search algorithm along with 10-fold cross validation to tune the hyperparameters of the MLP, and found the R2 value to be 0.89. Next, we adopted two generative DL models named variational autoencoder (VAE) and conditional variational autoencoder (CVAE) to learn data patterns and generate constituent compositions. With these generative models, we produced synthetic data with the same statistical distribution as the real data of actual hydrogel fabrication, which was then validated using Student’s t-test and an autoencoder (AE) anomaly detector. We found that none of the seven generated gel constituents were significantly different from the real data. Our trained DL models were successful in mapping the input–output relationship for the 3D-printed hydrogel substrates, which can predict multiple variables from a handful of input variables and vice versa.

This paper takes the problems and ways of diversified curriculum construction of vocal music education in colleges and universities as the entry point and constructs a deep blended learning model of vocal music diversified curriculum based on the deep learning model. The deep evolutionary knowledge tracking model is constructed by combining the self-attention mechanism embedding model with vocal music knowledge evolution based on Transformer. To verify the effectiveness of the deep blended learning model constructed in this paper, model comparison experiments and empirical analysis were conducted. The results show that compared with the SAKT model, the DBL model in this paper has a 5.19% improvement in the average ROC-AUC value. The vocal posttest scores of the students in the experimental class were 5.09 points higher than those of the control class, with a two-tailed significance value of 0.027, which is less than 0.05 and a significant difference. This indicates that the deep learning model can effectively promote the innovative design of diversified curricula for vocal music education in colleges and universities and enhance students’ vocal music learning ability.

With the increasing integration of artificial intelligence (AI) in several scientific domains, there is a rising demand for advanced AI tools capable of addressing advanced research challenges. A challenge of paramount importance lies in accurately predicting the streamflow within river basins. Effective river flow prediction holds significant relevance, particularly given the substantial societal implications of river usage, encompassing areas such as transportation, agriculture, and power generation. The present study introduces a novel approach to streamflow prediction involving the development of a Deep Learning (DL) model that combines a convolutional neural network with Transfer Learning (TL) techniques to predict streamflow in river systems. With the aim of training the developed DL model, the study employed a time-series dataset containing hydrological data related to two distinct river basins, i.e., Paraíba do Sul, in Brazil, and Zambezi in the state of Mozambique. The developed DL models exhibited the capability to effectively predict the river flow with a one-day horizon, relying on the preceding three or seven days of historical data. To overcome the limited availability of training data and reduce the training time of DL models, TL was leveraged to incorporate two additional distinct time-series datasets, i.e., historical streamflow data from the São Francisco River in Brazil, and climate data from Delhi, India. The application of TL significantly reduced training time, leading only to a minimal decrease in prediction performance. Indeed, in the case DL models were trained on data collected from the Paraíba do Sul River, a substantial reduction in training time was observed - up to 27% - with a modest percentage decrease of 0.31% in test predictive performance (). Similarly, TL induced a significant reduction in training time of up to 48%, while resulting in a modest 2% reduction in test predictive performance () for the Zambezi dataset. The findings underscore the significance of TL as a strategic and viable approach to improve the efficiency of river flow prediction models in the context of basins with limited hydrological data available.