A deep multi-task learning approach for ECG data analysis

In machine learning, the relatedness across multiple tasks is usually complex and entangled. Due to dataset bias, the relatedness among tasks might be distorted and mislead the training of the models with solid learning ability, such as the multi-task neural networks. In this paper, we propose the idea of Relatedness Refinement Multi-Task Learning (RRMTDL) by introducing adversarial learning in the multi-task deep neural network to tackle the problem. The RRMTDL deep learning model restrains the misleading relatedness task by adversarial training and extracts information sharing across tasks with valuable relatedness. With RRMTDL, multi-task deep learning can enhance the task-specific representation for the major tasks by excluding the misleading relatedness. We design tests with various combinations of task-relatedness to validate the proposed model. Experimental results show that the RRMTDL model can effectively refine the task relatedness and prominently outperform other multi-task deep learning models in datasets with entangled task labels.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

Colorectal cancer (CRC) is the third most commonly diagnosed cancer and the second most lethal cancer in the world. The segmentation and staging of tumours in CT images are important for CRC diagnosis and treatment. Using deep learning to fulfil medical image diagnosis is a promising approach. Most current methods solve segmentation and staging tasks separately, while ignoring the associated information between tasks. Multi-task deep learning (MTL) can solve the two tasks simultaneously. In this study, we proposed a CRC_MTL framework applying multi-task deep learning to complete two tasks jointly for analysing CRC tumours in abdominal CT images. The CRC_MTL framework not only improves the accuracy of the classification task from 91.40% to 93.52% but also improves the dice coefficient of the segmentation task from 0.6900 to 0.7379.

Magnetotelluric (MT) forward modeling is a key technique in magnetotelluric sounding, and deep learning has been widely applied to MT forward modeling. In three-dimensional (3-D) problems, although existing methods can predict forward modeling results with high accuracy, they often use multiple networks to simulate multiple forward modeling parameters, resulting in low efficiency. We apply multi-task learning (MTL) to 3-D MT forward modeling to achieve simultaneous inference of apparent resistivity and impedance phase, effectively improving overall efficiency. Furthermore, through comparative analysis of feature map differences in various decoder layers of the network, we identify the optimal branching point for multi-task learning decoders. This enhances the feature extraction capabilities of the network and improves the prediction accuracy of forward modeling parameters. Additionally, we introduce an uncertainty-based loss function to dynamically balance the learning weights between tasks, addressing the shortcomings of traditional loss functions. Experiments demonstrate that compared with single-task networks and existing multi-task networks, the proposed network (MT-FeatureNet) achieves the best results in terms of Structural Similarity Index Measure (SSIM), Mean Relative Error (MRE), and Mean Absolute Error (MAE). The proposed multi-task learning model not only improves the efficiency and accuracy of 3-D MT forward modeling but also provides a novel approach to the design of multi-task learning network structures.

Purpose To develop and evaluate a novel multitask deep learning framework for automated detection and localization of endoleaks at aortic digital subtraction angiography (DSA) performed during real-world endovascular aneurysm repair (EVAR) procedures for abdominal aortic aneurysm. Materials and Methods This retrospective study analyzed intraoperative aortic DSA images from patients undergoing EVAR (January 2017-December 2021). An expert panel assessed each sequence for endoleaks. Each sequence was processed into three input channels: peak density, time to peak, and area under the time-density curve, generating three two-dimensional perfusion maps per patient. These maps served as input into a convolutional neural network for binary detection (classification) and localization (regression) of endoleaks through multitask learning. Fivefold cross-validation was performed, with patients split 80:20 into training and testing datasets for each fold. Performance metrics included area under the receiver operating characteristic curve, F1 score, precision, and recall and were compared with human experts. Results The study included 220 patients (median age, 74 years [IQR, 68-79]; 181 male). Endoleaks were visible in 111 of 220 (50.5%) patients. The model identified and localized endoleaks with an area under the receiver operating characteristic curve of 0.85 ± 0.0031 (SD), F1 score of 0.78 ± 0.21, 95% precision, and 73% recall. Compared with the procedural team (94% precision, 63% recall), it had higher values in both metrics, with an F1 score within the human observer range (0.75-0.85). Balancing regression and classification by multitask learning delivered optimal results. The interobserver agreement among human experts was moderate (Fleiss κ = 0.404). Conclusion A novel, fully automated deep learning method accurately detected and localized endoleaks at DSA imaging from EVAR procedures. Keywords: Endoleaks, Deep Learning, Digital Subtraction Angiography, Endovascular Aneurysm Repair, Abdominal Aortic Aneurysm, Interventional Suite, Hybrid Operating Room Supplemental material is available for this article. © RSNA, 2025.

Deep understanding of urban mobility is of great significance for many real-world applications, such as urban traffic management and autonomous driving. This thesis develops deep learning methodologies to extract vehicle counts from streaming realtime video captured by multiple low resolution web cameras and construct maps of traffic density in a city environment; in particular, we focus on cameras installed in the Manhattan borough of NYC. The large-scale videos from these web cameras have low spatial and temporal resolution, high occlusion, large perspective, and variable environment conditions, making most existing methods to lose their efficacy. To overcome these challenges, the thesis develops several techniques: 1. a block-level regression model with a rank constraint to map the dense image feature into vehicle densities; 2. a deep multi-task learning framework based on fully convolutional neural networks to jointly learn vehicle density and vehicle count; 3. deep spatio-temporal networks for vehicle counting to incorporate temporal information of the traffic flow; and 4. multi-source domain adaptation mechanisms with adversarial learning to adapt the deep counting model to multiple cameras. To train and validate the proposed system, we have collected a largescale webcam traffic dataset CityCam that contains 60 million frames from 212 webcams installed in key intersections of NYC. Of there, 60; 000 frames have been annotated with rich information, leading to about 900; 000 annotated objects. To the best of our knowledge, it is the first and largest webcam traffic dataset with such large number of elaborate annotations. The proposed methods are integrated into the CityScapeEye system that has been extensively evaluated and compared to existing techniques on different counting tasks and datasets, with experimental results demonstrating the effectiveness and robustness of CityScapeEye.

In this work we propose a method that integrates multi-task learning (MTL) and deep learning. Our method appends a MTL-like loss to a deep convolutional neural network, in order to learn the relations between tasks together at the same time, and also incorporates the label correlations between pairs of tasks. We apply the proposed method on a transfer learning scenario, where our objective is to fine-tune the parameters of a network that has been originally trained on a large-scale image dataset for concept detection, so that it be applied on a target video dataset and a corresponding new set of target concepts. We evaluate the proposed method for the video concept detection problem on the TRECVID 2013 Semantic Indexing dataset. Our results show that the proposed algorithm leads to better concept-based video annotation than existing state-of-the-art methods.

A deep multi-task representation learning method for time series classification and retrieval

To achieve more effective solution for large-scale image classification (i.e., classifying millions of images into thousands or even tens of thousands of object classes or categories), a deep multi-task learning algorithm is developed by seamlessly integrating deep CNNs with multi-task learning over the concept ontology, where the concept ontology is used to organize large numbers of object classes or categories hierarchically and determine the inter-related learning tasks automatically. Our deep multi-task learning algorithm can integrate the deep CNNs to learn more discriminative high-level features for image representation, and it can also leverage multi-task learning and inter-level relationship constraint to train more discriminative tree classifier over the concept ontology and control the inter-level error propagation effectively. In our deep multi-task learning algorithm, we can use back propagation to simultaneously refine both the relevant node classifiers (at different levels of the concept ontology) and the deep CNNs according to a joint objective function. The experimental results have demonstrated that our deep multi-task learning algorithm can achieve very competitive results on both the accuracy and the cost of feature extraction for large-scale image classification.

Congestion control is a fundamental network task that modulates the data transmission rates of traffic sources to efficiently utilize network capacity. With the advent of machine learning, congestion control based on deep reinforcement learning is the subject of extensive attention. At present, research on machine-learning-based congestion control is mainly focused on single-task scenarios -- to design a control strategy that is only used to reduce congestion. This article studies the congestion control problem based on machine learning in multi-task scenarios. Specifically, we propose a congestion control model based on multi-task deep reinforcement learning. The model takes congestion control as the main task and load balancing as the auxiliary task. Compared to the single-task method, our model can better represent the network environment by learning the shared representation of congestion features and load balancing features. Moreover, network traffic control may involve both congestion control and load balancing. Therefore, learning multiple tasks jointly while exploiting commonalities and differences across tasks can help reduce the cost of task coordination. We use software defined networking to decouple the data and control planes, making network control strategies more flexible than traditional networks. To the best of our knowledge, this is the first time multi-task learning has been applied to network traffic control. Experimental results show that the method is efficient.

Early detection and timely surgical intervention are crucial in reducing mortality rates associated with clinically significant prostate cancer (csPCa). Currently, clinical diagnostics primarily depend on magnetic resonance imaging (MRI) and nuclear medicine, with the potential diagnostic value of abdominal computed tomography (CT) remaining underexplored. This study aims to evaluate the effectiveness of multi-task deep learning neural networks in identifying early-stage prostate cancer using CT scans. In this study, we enrolled 539 patients from the Department of Radiology (N=461) and Nuclear Medicine (N=78). We utilized a multi-task deep learning network model (MTDL), based on the 3DUnet architecture, to segment and analyze the collected abdominal plain CT images. The predictive performance of this model was compared with a radiomics model and a single-task deep learning model using ResNet18. A diagnostic nomogram was then developed using the multi-task deep learning approach, incorporating prediction results and PSAD, age. The diagnostic performance of the different models was evaluated using the receiver operating characteristic (ROC) curve and the area under the curve (AUC). The 461 patients from the Department of Radiology were divided into training and test sets at a ratio of 6:4, while the patients from the Department of Nuclear Medicine formed the validation set. Our MTDL nomogram demonstrated AUCs of 0.941 (95% confidence interval [CI]: 0.905valceedi 0.912 (95% CI: 0.904valceedi and 0.932 (95% CI: 0.883valceed in the training, test, and validation cohorts, respectively. This study indicates that combining abdominal CT with a multi-task neural network model effectively diagnoses csPCa, offering superior diagnostic performance compared to clinical models. Additionally, the multi-task neural network model outperformed both the single-task neural network model and the radiomics model in diagnostic accuracy. Our study demonstrated that the MTDL nomogram can accurately predict the presence of prostate cancer using abdominal CT scans, offering significant value for the early diagnosis of prostate cancer.

To develop and validate a multi-task deep learning (MTDL) model using multiphase contrast-enhanced CT (CECT) for simultaneously assessing histological subtypes, clinical stages, and anatomical complexity grades of solid malignant renal tumors. This two-center retrospective study included patients with solid malignant renal tumors and their preoperative kidney CECT images. A progressive layered extraction (PLE)-based MTDL model was trained and externally tested. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA), compared with the results of five radiologists. Among 798 patients (mean age, 54 ± 12 years; 279 females; Center A: n = 620, Center B: n = 178), 597 (74.8%) had clear cell renal cell carcinomas (ccRCC), 150 (18.8%) were clinical staging III/IV, and 187 (23.4%) had high anatomical complexity. On the external test set, the MTDL model achieved AUCs of 0.89 (95% CI: 0.82, 0.94) for distinguishing ccRCC from non-ccRCC, 0.87 (95% CI: 0.81, 0.93) for clinical staging (I/II vs. III/IV), and 0.87 (95% CI: 0.82, 0.92) for anatomical complexity grading (low-intermediate vs. high). The MTDL model outperformed single-task deep learning (STDL) in clinical staging (AUC: 0.87 vs. 0.82, p = 0.022), showed higher net benefit on DCA, and demonstrated better diagnostic performance than junior radiologists in histological subtyping and clinical staging. Additionally, it used 68% less memory and was 60% faster than STDL models. The CECT-based MTDL model demonstrated robust performance in simultaneously predicting histological subtypes, clinical stages, and anatomical complexity grades of malignant renal tumors. Question Accurate preoperative description of the histological subtyping, clinical staging, and anatomical complexity of malignant renal tumors is crucial for treatment decision-making. Findings By sharing features, the multi-task deep learning algorithm model enhances clinical staging performance and significantly improves computational efficiency in predicting all three tasks simultaneously. Clinical relevance The multi-task deep learning algorithm model enables rapid and accurate comprehensive preoperative evaluation of renal tumors, which assists surgeons in optimizing surgical plans and promotes the advancement of renal tumor management toward precision and efficiency.

Conventional methods for solving Navier–Stokes (NS) equations to analyze flow fields and aerodynamic forces of airfoils with trailing edge flaps (TEFs) are known for their significant time cost. This study presents a Multi-Task Swin Transformer (MT-Swin-T) deep learning framework tailored for swift prediction of velocity fields and aerodynamic coefficients of TEF-equipped airfoils. The proposed model combines a Swin Transformer (Swin-T) for flow field prediction with a multi-layer perceptron (MLP) dedicated to lift coefficient prediction. Both networks undergo gradient updates through the shared encoder component of the Swin Transformer. Such a trained network model for computational fluid dynamics simulations is both effective and robust, significantly improving the efficiency of complex aerodynamic shape design optimization and flow control. The study further investigates the impact of integrating multi-task learning loss functions, skip connections, and the network’s structural design on prediction accuracy. Additionally, the effectiveness of deep learning in improving the aerodynamic simulation efficiency of airfoils with TEF is examined. Results demonstrate that the multi-task deep learning approach provides accurate predictions for TEF airfoil flow fields and lift coefficients. The strategic combination of these tasks during network training, along with the optimal selection of loss functions, significantly enhances prediction accuracy compared with the single-task network. In a specific case study, the MT-Swin-T model demonstrated a prediction time that was 1/7214 of the time necessitated by CFD simulation.

The aim of this study was to determine the invasiveness of ground-glass nodules (GGNs) using a 3D multi-task deep learning network. We propose a novel architecture based on 3D multi-task learning to determine the invasiveness of GGNs. In total, 770 patients with 909 GGNs who underwent lung CT scans were enrolled. The patients were divided into the training (n = 626) and test sets (n = 144). In the test set, invasiveness was classified using deep learning into three categories: atypical adenomatous hyperplasia (AAH) and adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), and invasive pulmonary adenocarcinoma (IA). Furthermore, binary classifications (AAH/AIS/MIA vs. IA) were made by two thoracic radiologists and compared with the deep learning results. In the three-category classification task, the sensitivity, specificity, and accuracy were 65.41%, 82.21%, and 64.9%, respectively. In the binary classification task, the sensitivity, specificity, accuracy, and area under the ROC curve (AUC) values were 69.57%, 95.24%, 87.42%, and 0.89, respectively. In the visual assessment of GGN invasiveness of binary classification by the two thoracic radiologists, the sensitivity, specificity, and accuracy of the senior and junior radiologists were 58.93%, 90.51%, and 81.35% and 76.79%, 55.47%, and 61.66%, respectively. The proposed multi-task deep learning model achieved good classification results in determining the invasiveness of GGNs. This model may help to select patients with invasive lesions who need surgery and the proper surgical methods. • The proposed multi-task model has achieved good classification results for the invasiveness of GGNs. • The proposed network includes a classification and segmentation branch to learn global and regional features, respectively. • The multi-task model could assist doctors in selecting patients with invasive lesions who need surgery and choosing appropriate surgical methods.

Accurate estimation of emotions in SNS posts plays an essential role in a wide variety of real world applications such as intelligent dialogue systems, review analysis for recommendations and so on. In this paper, we focus on developing accurate models for estimating types of emotions and their intensities in Japanese tweets by using multi-task deep learning. More concretely, three deep learning models for estimating intensities of emotions were extended to be able to predict the emotional types and their intensities at a time. The effectiveness of the developed models was confirmed through experiments using the database of Japanese tweets annotated with intensity scores of four types of emotions using best-worst scaling.