Tencent Healthcare (China)

Most existing federated learning (FL) methods for medical image analysis only considered intramodal heterogeneity, limiting their applicability to multimodal imaging applications. In practice, it is not uncommon that some FL participants only possess a subset of the complete imaging modalities, posing inter-modal heterogeneity as a challenge to effectively training a global model on all participants’ data. In addition, each participant would expect to obtain a personalized model tailored for its local data characteristics from the FL in such a scenario. In this work, we propose a new FL framework with federated modality-specific encoders and multimodal anchors (FedMEMA) to simultaneously address the two concurrent issues. Above all, FedMEMA employs an exclusive encoder for each modality to account for the inter-modal heterogeneity in the first place. In the meantime, while the encoders are shared by the participants, the decoders are personalized to meet individual needs. Specifically, a server with full-modal data employs a fusion decoder to aggregate and fuse representations from all modality-specific encoders, thus bridging the modalities to optimize the encoders via backpropagation reversely. Meanwhile, multiple anchors are extracted from the fused multimodal representations and distributed to the clients in addition to the encoder parameters. On the other end, the clients with incomplete modalities calibrate their missing-modal representations toward the global full-modal anchors via scaled dot-product cross-attention, making up the information loss due to absent modalities while adapting the representations of present ones. FedMEMA is validated on the BraTS 2020 benchmark for multimodal brain tumor segmentation. Results show that it outperforms various up-to-date methods for multimodal and personalized FL and that its novel designs are effective. Our code is available.

Using deep learning (DL)-based technique, we identify risk factors and create a prediction model for refractory neovascular age-related macular degeneration (nAMD) characterized by persistent disease activity (PDA) in spectral domain optical coherence tomography (SD-OCT) images. A total of 671 typical B-scans were collected from 186 eyes of 186 patients with nAMD. Spectral domain optical coherence tomography images were analyzed using a classification convolutional neural network (CNN) and a fully convolutional network (FCN) algorithm to extract six features involved in nAMD, including ellipsoid zone (EZ), external limiting membrane (ELM), intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelium detachment (PED), and subretinal hyperreflective material (SHRM). Random forest models were probed to predict 1-year disease activity (stable, PDA, and cured) based on the quantitative features computed from automated segmentation and evaluated with cross-validation. The algorithm to segment six SD-OCT features achieved the mean accuracy of 0.930 (95% CI: 0.916-0.943), dice coefficients of 0.873 (95% CI: 0.847-0.899), a sensitivity of 0.873 (95% CI: 0.844-0.910), and a specificity of 0.922 (95% CI: 0.905-0.940). The six-metric model including EZ and ELM achieved the optimal performance to predict 1-year disease activity, with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.980, the accuracy of 0.930, the sensitivity of 0.920, and the specificity of 0.962. The integrity of EZ and ELM significantly improved the performance of the six-metric model than that of the four-metric model. The prediction model reveals the potential to predict PDA in nAMD eyes. The integrity of EZ and ELM constituted the strongest predictive factor for PDA in nAMD eyes in real-world clinical practice. The results of this study are a significant step toward image-guided prediction of long-term disease activity in the management of nAMD and highlight the importance of the automatic identification of photoreceptor layers.

Medical image segmentation tasks hitherto have achieved excellent progresses with large-scale datasets, which empowers us to train potent deep convolutional neural networks (DCNNs). However, labeling such large-scale datasets is laborious and error-prone, which leads the noisy (or incorrect) labels to be an ubiquitous problem in the real-world scenarios. In addition, data collected from different sites usually exhibit significant data distribution shift (or domain shift). As a result, noisy label and domain shift become two common problems in medical imaging application scenarios, especially in medical image segmentation, which degrade the performance of deep learning models significantly. In this paper, we identify a novel problem hidden in medical image segmentation, which is unsupervised domain adaptation on noisy labeled data, and propose a novel algorithm named "Self-Cleansing Unsupervised Domain Adaptation" (S-CDUA) to address such issue. S-CUDA sets up a realistic scenario to solve the above problems simultaneously where training data (i.e., source domain) not only shows domain shift w.r.t. unsupervised test data (i.e., target domain) but also contains noisy labels. The key idea of S-CUDA is to learn noise-excluding and domain invariant knowledge from noisy supervised data, which will be applied on the highly corrupted data for label cleansing and further data-recycling, as well as on the test data with domain shift for supervised propagation. To this end, we propose a novel framework leveraging noisy-label learning and domain adaptation techniques to cleanse the noisy labels and learn from trustable clean samples, thus enabling robust adaptation and prediction on the target domain. Specifically, we train two peer adversarial networks to identify high-confidence clean data and exchange them in companions to eliminate the error accumulation problem and narrow the domain gap simultaneously. In the meantime, the high-confidence noisy data are detected and cleansed in order to reuse the contaminated training data. Therefore, our proposed method can not only cleanse the noisy labels in the training set but also take full advantage of the existing noisy data to update the parameters of the network. For evaluation, we conduct experiments on two popular datasets (REFUGE and Drishti-GS) for optic disc (OD) and optic cup (OC) segmentation, and on another public multi-vendor dataset for spinal cord gray matter (SCGM) segmentation. Experimental results show that our proposed method can cleanse noisy labels efficiently and obtain a model with better generalization performance at the same time, which outperforms previous state-of-the-art methods by large margin. Our code can be found at https://github.com/zzdxjtu/S-cuda.

Intracerebral hemorrhage (ICH) is the deadliest type of stroke. Early prediction of stroke lesion growth is crucial in assisting physicians towards better stroke assessments. Existing stroke lesion prediction methods are mainly for ischemic stroke. In ICH, most methods only focus on whether the hematoma will expand but not how it will develop. This paper explored a new, unknown topic of predicting ICH growth at the image-level based on the baseline non-contrast computerized tomography (NCCT) image and its hematoma mask. We propose a novel end-to-end prediction framework based on the displacement vector fields (DVF) with the following advantages. 1) It can simultaneously predict CT image and hematoma mask at follow-up, providing more clinical assessment references and surgery indication. 2) The DVF regularization enforces a smooth spatial deformation, limiting the degree of the stroke lesion changes and lowering the requirement of large data. 3) A multi-modal fusion module learns high-level associations between global clinical features and spatial image features. Experiments on a multi-center dataset demonstrate improved performance compared to several strong baselines. Detailed ablation experiments are conducted to highlight the contributions of various components.

Retinal artery/vein (A/V) classification lays the foundation for the quantitative analysis of retinal vessels, which is associated with potential risks of various cardiovascular and cerebral diseases. The topological connection relationship, which has been proved effective in improving the A/V classification performance for the conventional graph based method, has not been exploited by the deep learning based method. In this paper, we propose a Topology Ranking Generative Adversarial Network (TR-GAN) to improve the topology connectivity of the segmented arteries and veins, and further to boost the A/V classification performance. A topology ranking discriminator based on ordinal regression is proposed to rank the topological connectivity level of the ground-truth, the generated A/V mask and the intentionally shuffled mask. The ranking loss is further back-propagated to the generator to generate better connected A/V masks. In addition, a topology preserving module with triplet loss is also proposed to extract the high-level topological features and further to narrow the feature distance between the predicted A/V mask and the ground-truth. The proposed framework effectively increases the topological connectivity of the predicted A/V masks and achieves state-of-the-art A/V classification performance on the publicly available AV-DRIVE dataset.

Medical images are generally labeled by multiple experts before the final ground-truth labels are determined. Consensus or disagreement among experts regarding individual images reflects the gradeability and difficulty levels of the image. However, when being used for model training, only the final ground-truth label is utilized, while the critical information contained in the raw multi-rater gradings regarding the image being an easy/hard case is discarded. In this paper, we aim to take advantage of the raw multi-rater gradings to improve the deep learning model performance for the glaucoma classification task. Specifically, a multi-branch model structure is proposed to predict the most sensitive, most specifical and a balanced fused result for the input images. In order to encourage the sensitivity branch and specificity branch to generate consistent results for consensus labels and opposite results for disagreement labels, a consensus loss is proposed to constrain the output of the two branches. Meanwhile, the consistency/inconsistency between the prediction results of the two branches implies the image being an easy/hard case, which is further utilized to encourage the balanced fusion branch to concentrate more on the hard cases. Compared with models trained only with the final ground-truth labels, the proposed method using multi-rater consensus information has achieved superior performance, and it is also able to estimate the difficulty levels of individual input images when making the prediction.

e18052 {\rtf1\ansi\ansicpg936\cocoartf1347\cocoasubrtf570 {\fonttbl\f0\froman\fcharset0 Times-Roman;\f1\fnil\fcharset0 Calibri;\f2\fswiss\fcharset0 Helvetica; \f3\fnil\fcharset134 STHeitiSC-Light;\f4\froman\fcharset0 TimesNewRomanPSMT;} {\colortbl;\red255\green255\blue255;\red0\green0\blue0;} \paperw11900\paperh16840\margl1440\margr1440\vieww25400\viewh13600\viewkind0 \deftab720 \pard\pardeftab720 \f0\fs24 \cf2 \expnd0\expndtw0\kerning0 \outl0\strokewidth0 \strokec2 AI platform in mammagraphy to screening breast cancer: a sole center experence. \f1\fs21 \cf0 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 \ \pard\pardeftab720\ri714\qj \cf0 Background: The aim of this study was to use deep convolutional neural network (DCNN) to relieve radiologists ' burden and to minimize misses and interpretation errors of suspicious lesions at digital mammography. Methods: We developed and trained the DCNN model on the training set, then validated it in an internal validation set. The labeled data were used for training the DCNN model, and the mass detection, calcification detection and benign and malignant diagnosis were interpreted respectively. After verifying the feasibility of the DCNN model in the diagnosis of breast cancer, we gradually increased the number of cases in the training set and established the verification set to test the training results. The latest training sets include 21100 atlas of our hospital between April 2010 and October 2017, among which the malignant case atlas is 1774. Each atlas contains 2 ipsilateral mammography images. The verification set includes 1307 atlas, among which the malignant case atlas is 248. Results: The DCNN model achieved high performance in identifying breast cancer patients in the validation sets tested, with sensitivity of 98.02%, specificity of 91.86% \f3 \'a3\'ac \f2 and area under the curve values of 0.9813. As for mass detection, we reached a recall rate of 89.6% on the premise of lowing the false positive to 0.168%. For the latest clinical data between October 2018 and November 2018 including 1043 atlas from 576 patients, the sensitivity was 78.6% (95%CI 62.8-89.1) versus 90.5% (95%CI 76.5 \f4 - \f2 96.9; p=0.021) and specificity was 93.4% (95%CI 91.6 \f4 - \f2 94.8) versus 82.7% (95%CI 80.2 \f4 - \f2 85 \f4 . \f2 0; p<0.0001). Conclusions: The DCNN model showed improved specificity in diagnosis breast cancer compared with a group of skilled radiologists. As the number of cases in the training set increases, the capacity of DCNN model will be further improved.