A Performance Evaluation of 3D Deep Learning Algorithms for Crime Classification

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

MACHINE LEARNING PREDICTION OF IN-HOSPITAL DISEASE PROGRESSION IN COMMUNITY-ACQUIRED PNEUMONIA: DERIVATION AND VALIDATION OF CLINICAL DETERIORATION IN PNEUMONIA (CDIP)

Active surveillance (AS) has excellent short to medium term outcomes in well-selected prostate cancer patients. Traditional biopsy-based selection criteria have been criticized for inaccurate determination of cancer grade and extent. We evaluated the incremental benefit of multiparametric magnetic resonance imaging (mpMRI) in patient selection using various AS criteria. We retrospectively evaluated men who received mpMRI before radical prostatectomy between 2011 and 2014. Patients were classified as suitable for AS using four criteria: (1) Epstein, (2) National Comprehensive Cancer Network (NCCN) low-risk or (3) extended criteria (Gleason ⩽3+4, PSA ⩽15 ng/ml, clinical stage ⩽T2b) using clinical parameters. The incremental value of mpMRI was evaluated against the referent standard of surgical pathology in determining suitability for AS using sensitivity, specificity, likelihood ratios (LRs) and area under receiver operating curves (AUCs). We evaluated 208 men. Only one man fulfilled Epstein criteria (1) at pathology, who was neither identified using clinical criteria nor mpMRI. Using (2), clinical criteria had a sensitivity of 80%, specificity 75%, LR+ 3.3, LR- 0.3, AUC 0.78, while combined clinical-mpMRI criteria achieved a sensitivity of 80%, specificity 99.5% (P<0.01), LR+ 162, LR- 0.2 and AUC 0.90 (P<0.01 compared to clinical). Using (3), clinical criteria had a sensitivity of 74%, specificity 47%, LR+ 1.4, LR- 0.6, AUC 0.60, while combined clinical-mpMRI criteria achieved a sensitivity of 26% (P<0.01), specificity 97% (P<0.01), LR+ 8.3, LR- 0.8 and AUC 0.62 (P=0.85). Addition of mpMRI significantly improved selection of men for AS using NCCN low-risk criteria. For selecting men with limited prognostic grade group 2, mpMRI significantly improved specificity at the expense of sensitivity.

BackgroundThe process for assessing cognitive symptoms in clinic is time consuming, expensive and usually requires multiple clinicians. There is an urgent need for accessible, quick and low‐cost methods to aid identification of people at the early stages of the dementia continuum. Emerging evidence shows that simple hand movement tests could aid risk stratification. We evaluated the classification accuracy of the new online TAS Test alternate tapping test in the ISLAND Cognitive Clinic in Tasmania, Australia, for dementia, mild cognitive impairment (MCI) and subjective cognitive decline (SCD) from control subjects.Method212 participants (69.0 ± 9.3 years old) were recruited: 130 from the Clinic (50 diagnosed with dementia, 54 with MCI, 26 with SCD) and 82 cognitively healthy controls. All Clinic participants had a gold‐standard interdisciplinary consensus diagnosis after detailed neuropsychological tests and completed a 60‐second alternate key tapping test on the TAS Test website during their standard clinic visit. Healthy controls completed the Addenbrookes Cognitive Examination with a score of 95 or higher and TAS Test in clinic. Frequency, variability, key press duration and accuracy scores were calculated. Regression models comprising all motor features, adjusted for age and sex, were compared to their respective null models, comprising age and sex, with area under ROC curves (AUC).ResultCompared to the baseline model, tapping motor data improved classification of people with dementia (AUC = 0.89, 95% CI = [0.84,0.95], p=0.025), MCI (AUC = 0.8, 95% CI = [0.73,0.87], p=0.002) and SCD (AUC = 0.75, 95% CI = [0.63,0.86], p = 0.004) from cognitively healthy controls. The motor features also helped distinguish dementia (AUC = 0.9, 95% CI = [0.84,0.97], p = 0.024) and MCI (AUC = 0.79, 95% CI = [0.67,0.91], p = 0.022) from SCD, but not dementia from MCI (p = 0.221).ConclusionA 60‐second simple keyboard tapping test helped distinguish people with SCD, MCI, or dementia from cognitively healthy controls and holds potential as a new low‐cost and clinically‐accessible motor biomarker.

BackgroundThe Inflammatory Bowel Disease Symptom Inventory (IBDSI) is a validated patient self-reported measure used to assess IBD disease activity.PurposeWe aimed to assess the prevalence of symptoms, and examine which symptoms are most associated with disease activity as measured by a symptom index and objective measure of inflammation.MethodThe Manitoba Living with IBD Study is a prospective study of 156 participants with confirmed IBD who completed bi-weekly Inflammatory Bowel Disease Symptom Inventory (IBDSI) surveys. Relative risks (RR) (with 95% confidence interval (CI)), positive and negative predictive values (PPV, NPV), and area under receiver operator curve (AUC) were reported for each symptom to predict active disease defined as: (1) active IBDSI, (2) self-reported flare, and (3) elevated fecal calprotectin (FCAL) (>250µg/g). Analyses were undertaken following stratification based on sex, and disease type (Crohn’s disease (CD) and ulcerative colitis (UC)).Result(s)In total, 69.2% of participants were female; 64.7% had CD. Fatigue was the most prevalent symptom in both inactive and active disease, across all 3 disease measures (IBDSI: 24.5% and 75.1%, self-reported flare: 42.2% and 72.2%, FCAL: 46.0% and 60.6%). Absence of fatigue had a high NPV for active IBDSI and self-reporting a flare in both CD and UC. Urgency had a consistently strong NPV, RR and AUC across all three disease measures in both IBD subtypes and sexes. The number of loose/liquid bowel movements predicted elevated FCAL in UC (RR in men = 3.5, 95% CI 1.2-9.9, RR in women = 1.8, 95% CI 1.2-2.7, AUC in men = 0.73, 95% CI 0.59-0.84, AUC in women = 0.65, 95% CI 0.56-0.72), as did blood in stool in females with UC (RR = 1.8, 95% CI 1.2-2.7). In men with CD, excessive bowel gas (RR = 2.0, 95% CI 1.2-3.4) and urgency (RR = 3.9, 95% CI 1.6-9.3) best predicted an elevated FCAL. No symptom was strongly predictive of elevated FCAL in females with CD.Conclusion(s)Fatigue was the most prevalent symptom irrespective of disease activity. Urgency stood out as being consistently associated with disease activity, irrespective of the disease measure. Individual symptoms have different impacts on subjective (IBDSI) and objective (FCAL) measures in IBD.Please acknowledge all funding agencies by checking the applicable boxes belowNoneDisclosure of InterestNone Declared

Exploring the interrelationships between microbes and disease can help microbiologists make decisions and plan treatments. Predicting new microbe-disease associations currently relies on biological experiments and domain knowledge, which is time-consuming and inefficient. Automated algorithms are used to uncover the intrinsic link between microbes and disease. However, due to data noise and inadequate understanding of relevant biology, the efficient prediction of microbe-disease associations is still crucial. This study develops a multi-view graph augmentation convolutional network (MVGCNMDA) to predict potential disease-associated microbes. First, we use two data augmentation methods, edge perturbation and node dropping, to remove the data noise in the preprocessing stage. Second, we calculate Gaussian interaction profile kernel similarity and cosine similarity. Therefore, the Graph Convolutional Network(GCN) can fully use multi-view features. Then, the multi-view features are fed into the multi-attention block to learn the weights of different features adaptively. Finally, the embedding results are obtained using a Convolutional Neural Network (CNN) combiner, and the matrix completion is used to predict the relationship between potential microbes and diseases. We test our model on the Human microbe-disease Association Database (HMDAD), Disbiome, and the Combined Dataset (Peryton and MicroPhenoDB). The area under PR curve (AUPR), area under ROC curve (AUC), F1 score, and RECALL value are calculated to evaluate the performance of the developed MVGCNMDA. The AUPR is 0.9440, AUC is 0.9428, F1 score is 0.9383, and RECALL value is 0.8858. The experiments show that our model can accurately predict potential microbe-disease associations compared with the state-of-the-art works on the global Leave-One-Out-Cross-Validation (LOOCV) and the fivefold Cross-Validation (fivefold CV). To further verify the effectiveness of the proposed graph data augmentation, we designed five different settings in the ablation study. Furthermore, we present two case studies that validate the prediction of the potential association between microbes and diseases by MVGCNMDA.

Background Moyamoya disease (MMD) is a serious intracranial cerebrovascular disease. Cerebral hemorrhage caused by MMD will bring life risk to patients. Therefore, MMD detection is of great significance in the prevention of cerebral hemorrhage. In order to improve the accuracy of digital subtraction angiography (DSA) in the diagnosis of ischemic MMD, in this paper, a deep network architecture combined with 3D convolutional neural network (3D CNN) and bidirectional convolutional gated recurrent unit (BiConvGRU) is proposed to learn the spatiotemporal features for ischemic MMD detection. Methods Firstly, 2D convolutional neural network (2D CNN) is utilized to extract spatial features for each frame of DSA. Secondly, the long-term spatiotemporal features of DSA sequence are extracted by BiConvGRU. Thirdly, the short-term spatiotemporal features of DSA are further extracted by 3D convolutional neural network (3D CNN). In addition, different features are extracted when gray images and optical flow images pass through the network, and multiple features are extracted by features fusion. Finally, the fused features are utilized to classify. Results The proposed method was quantitatively evaluated on a data sets of 630 cases. The experimental results showed a detection accuracy of 0.9788, sensitivity and specificity were 0.9780 and 0.9796, respectively, and area under curve (AUC) was 0.9856. Compared with other methods, we can get the highest accuracy and AUC. Conclusions The experimental results show that the proposed method is stable and reliable for ischemic MMD detection, which provides an option for doctors to accurately diagnose ischemic MMD.

We assessed the role of urine lipoarabinomannan (LAM) grade and a second LAM test for HIV-associated pulmonary tuberculosis (TB) screening in outpatient clinics in South Africa. We enrolled newly diagnosed HIV-infected adults (≥18 years) at 4 clinics, excluding those on TB therapy. Participants provided sputum for acid-fast bacilli (AFB) microscopy and culture. Nurses conducted 2 rapid urine LAM tests at the point-of-care and graded positive results from low (faint) to high (5+). Culture-confirmed pulmonary TB was the gold standard. We used area under receiver operating curves (AUROC) to compare screening strategies. Among 320 HIV-infected adults, median CD4 was 248 cells per cubic millimeter (interquartile range, 107-379/mm); 54 (17%) were TB culture positive. Fifty-two (16%) of all participants were LAM positive by either test; correlation between LAM tests was high. Among 10 "faint" positive results, 2 (20%) had culture-positive TB. Using ≥1+ LAM grade as positive, 1 LAM test had sensitivity of 41% [95% confidence interval (CI): 28% to 55%] and specificity of 92% (95% CI: 88% to 95%). A 2 LAM test strategy had a sensitivity of 43% (95% CI: 29% to 57%). One LAM test ≥1+ grade (AUROC = 0.66; 95% CI: 0.60 to 0.73) was significantly better than sputum AFB alone. The optimal strategy was sequentially performing 1 LAM test followed by sputum AFB if LAM grade <1+ (AUROC = 0.70; 95% CI: 0.63 to 0.77), which had sensitivity of 48% (95% CI: 34% to 62%) and specificity of 91% (95% CI: 87% to 94%). In this clinic-based study, "faint" line was a false-positive second urine LAM test added no value, and an optimal screening strategy was 1 LAM test followed by sputum AFB microscopy for urine LAM-negative people.

Diabetes, also known as chronic illness, is a group of metabolic diseases due to a high level of sugar in the blood over a long period. The risk factor and severity of diabetes can be reduced significantly if the precise early prediction is possible. The robust and accurate prediction of diabetes is highly challenging due to the limited number of labeled data and also the presence of outliers (or missing values) in the diabetes datasets. In this literature, we are proposing a robust framework for diabetes prediction where the outlier rejection, filling the missing values, data standardization, feature selection, K-fold cross-validation, and different Machine Learning (ML) classifiers (k-nearest Neighbour, Decision Trees, Random Forest, AdaBoost, Naive Bayes, and XGBoost) and Multilayer Perceptron (MLP) were employed. The weighted ensembling of different ML models is also proposed, in this literature, to improve the prediction of diabetes where the weights are estimated from the corresponding Area Under ROC Curve (AUC) of the ML model. AUC is chosen as the performance metric, which is then maximized during hyperparameter tuning using the grid search technique. All the experiments, in this literature, were conducted under the same experimental conditions using the Pima Indian Diabetes Dataset. From all the extensive experiments, our proposed ensembling classifier is the best performing classifier with the sensitivity, specificity, false omission rate, diagnostic odds ratio, and AUC as 0.789, 0.934, 0.092, 66.234, and 0.950 respectively which outperforms the state-of-the-art results by 2.00 % in AUC. Our proposed framework for the diabetes prediction outperforms the other methods discussed in the article. It can also provide better results on the same dataset which can lead to better performance in diabetes prediction. Our source code for diabetes prediction is made publicly available.

Backgrounds: Sentinel lymph node (SLN) biopsy is a highly accurate predictor of axillary status and has become the surgical axillary standard in breast cancer patients. About 50–70 % of patients with involved SLN have no additional non sentinel node (NSN) involved, suggesting that it be possible to avoid ALND in selected patients. Many tools have been developed to help surgeons in NSLN evaluation but they all need pathological data from tumor and SLN and can't be used during surgery. Developed for intraoperative detection of SLN macro or micrometastasis involvement, the semi-automated molecular one step nucleic acid amplification (OSNA), as accurate as pathology, is available. Two simple nomograms have been developed to predict NSN involvement based on the number of CK19 mRNA copy determined by OSNA: · Nomogram developed by Peg V (Eur J Surg Oncol 2013): based on total tumoral load (TTL). TTL is defined as the addition of CK19 mRNA copies of each positive SLN (copies/μL). A TTL≥1.2 × 10(5) copies/ml (specificity=85.3%, negative predictive value (NPV) = 80%) can predict NSN involvement. · Nomogram developed by Di Filippo F (Journal of Experimental &amp; Clinical Cancer Research 2015): based on the number of CK19 mRNA copies and ultrasound tumor size. These two variables are categorized using quartiles with a score for each and the addition of both corresponds to a probability of NSN involvement (sensitivity = 98.1%, NPV = 92.5 %). Patients and Methods: this is a retrospective study of 299 patients. Each patient had SLN involvement (macro or micrometastasis) and underwent a complementary ALND. The main objective was to evaluate the performance of each nomogram using a discrimination ability model, assessed by ROC analysis. Predictive accuracy was measured by the area under ROC curves (AUC) reported with its 95 % confidence interval. The second objective was to compare the two nomograms using Hanley &amp; McNeil method, to test the statistical significance of the difference between the AUC. Analysis was performed using stata 13.1 SE. Results: The mean age was 59, 1 year. Most patients were treated for an infiltrating ductal carcinoma (80.3%, 240/299). The mean ultrasound tumor size was 13 mm and the mean pathological tumor size was 15 mm. The median number of examined SLN was 2 with a macro-metastasis in 67, 6%, 202/299). 70 patients had involved nodes in ALND (23%). The discrimination of N Peg, quantified with AUC was 0.685 (p&amp;lt;0, 00001). The discrimination of N Di Filippo, quantified with AUC was 0.72 (p&amp;lt;0, 00001). Hanley &amp; McNeil method shows that Di Filippo nomogram is significantly superior to Peg nomogram (p=0,048). Conclusion: The current study shows that these two nomograms are reliable and can be used to predict NSLN involvement. The combination of molecular data and ultrasound tumor size seems to be more efficient than molecular data alone. These results are similar to results of nomogram studies based on pathological analysis but only these nomograms integrating molecular data can be used during the surgery. Citation Format: Bordes V, Campion L, Jezequel P, Lefrancois A, Boiffard F, Brillaud-Meflah V, Dravet F, Jaffre I, Classe J-M. Non-sentinel lymph nodes involvement in early breast cancer patients: Performance of two predictive nomograms integrating the analysis of sentinel nodes by one step nucleic acid amplification in a cohort of 299 patients [abstract]. In: Proceedings of the 2016 San Antonio Breast Cancer Symposium; 2016 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2017;77(4 Suppl):Abstract nr P2-01-33.

Deep Learning-Based Electrocardiogram Analysis Predicts Biventricular Dysfunction and Dilation in Congenital Heart Disease

To evaluate the diagnostic value of circulating microRNA-1 (miR-1) in early coronary artery plaque rupture in patients with stable coronary artery disease (SCAD). A prospective cohort study was conducted. Sixty-seven patients with SCAD admitted to the department of cardiology of the Third Affiliated Hospital of Soochow University from January to June in 2019 were enrolled. All patients had completed coronary angiography (CAG), percutaneous coronary intervention (PCI) single stent implantation or only CAG was performed according to the CAG results. Blood samples were collected before (0 hour) and 3 hours after the procedure. The expression of plasma miR-1 was detected by real-time quantitative reverse transcription-polymerase chain reaction (RT-PCR), and electrocardiogram was used to detect cardiac troponin I (cTnI) levels. The difference of miR-1 and cTnI levels in PCI or CAG patients before and after procedure were compared, and the value for early diagnosis of coronary artery plaque rupture in SCAD patients was evaluated. The diagnostic efficacy was evaluated by the receiver operating characteristic curve (ROC curve). There were 38 CAG patients and 29 PCI patients. There were no significant differences in gender, age, previous history (without hypertension history) and baseline data of cardiac function between the two groups. The expression of miR-1 after PCI was significantly higher than that before PCI [2-ΔΔCt: 2.11 (1.56, 2.73) vs. 1.26 (1.07, 1.92), P < 0.01], and there was no significant difference in cTnI level before and after PCI [μg/L: 0.00 (0.00, 0.02) vs. 0.00 (0.00, 0.02), P > 0.05]. There were no significant differences in miR-1 and cTnI levels before and after procedure in the CAG group [miR-1 (2-ΔΔCt): 1.09 (1.00, 1.40) vs. 1.21 (1.00, 1.71), cTnI (μg/L): 0.00 (0.00, 0.02) vs. 0.00 (0.00, 0.02), both P > 0.05]. ROC curve analysis showed that the area under ROC curve (AUC) and 95% confidence interval (95%CI) of miR-1 in the diagnosis of coronary plaque rupture were 0.794 (0.687-0.900), P < 0.01, the sensitivity was 82.8%, the specificity was 68.4%, and the optimal cut-off value was 1.51. The AUC and 95%CI of the difference of miR-1 before and after operation (ΔmiR-1) were 0.704 (0.567-0.842), P = 0.004, the sensitivity was 62.1%, the specificity was 84.2%, and the optimal cut-off value was 0.39. The efficancy of miR-1 and ΔmiR-1 after procedure to diagnose coronary plaque rupture in patients with SCAD was similar (Z = 1.287, P = 0.198). However, baseline miR-1 might not predict whether patients with SCAD need PCI or not (AUC = 0.630, P > 0.05). Multivariate binary Logistic regression analysis showed that increased postoperative miR-1 expression was an independent risk factor for coronary plaque rupture in SCAD patients [odds ratios (OR) = 2.887, 95%CI was 1.044-7.978, P = 0.041]. Circulating miR-1 might have the value for early diagnosis of coronary artery plaque rupture in SCAD patients.

BackgroundConvolutional neural networks (CNNs) have outperformed dermatologists in classifying pigmented skin lesions under artificial conditions. We investigated, for the first time, the performance of three-dimensional (3D) and two-dimensional (2D) CNNs and dermatologists in the early detection of melanoma in a real-world setting. MethodsIn this prospective study, 1690 melanocytic lesions in 143 patients with high-risk criteria for melanoma were evaluated by dermatologists, 2D-FotoFinder-ATBM and 3D-Vectra WB360 total body photography (TBP). Excision was based on the dermatologists' dichotomous decision, an elevated CNN risk score (study-specific malignancy cut-off: FotoFinder >0.5, Vectra >5.0) and/or the second dermatologist's assessment with CNN support. The diagnostic accuracy of the 2D and 3D CNN classification was compared with that of the dermatologists and the augmented intelligence based on histopathology and dermatologists’ assessment. Secondary end-points included reproducibility of risk scores and naevus counts per patient by medical staff (gold standard) compared to automated 3D and 2D TBP CNN counts. ResultsThe sensitivity, specificity, and receiver operating characteristics area under the curve (ROC-AUC) for risk-score-assessments compared to histopathology of 3D-CNN with 95% confidence intervals (CI) were 90.0%, 64.6% and 0.92 (CI 0.85–1.00), respectively. While dermatologists and augmented intelligence achieved the same sensitivity (90%) and comparable classification ROC-AUC (0.91 [CI 0.80–1.00], 0.88 [CI 0.77–1.00]) with 3D-CNN, their specificity was superior (92.3% and 86.2%, respectively). The 2D-CNN (sensitivity: 70%, specificity: 40%, ROC-AUC: 0.68 [CI 0.46–0.90]) was outperformed by 3D CNN and dermatologists. The 3D-CNN showed a higher correlation coefficient for repeated measurements of 246 lesions (R = 0.89) than the 2D-CNN (R = 0.79). The mean naevus count per patient varied significantly (gold standard: 210 lesions; 3D-CNN: 469; 2D-CNN: 1324; p < 0.0001). ConclusionsOur study emphasises the importance of validating the classification of CNNs in real life. The novel 3D-CNN device outperformed the 2D-CNN and achieved comparable sensitivity with dermatologists. The low specificity of CNNs and the lack of automated counting of TBP nevi currently limit the use of augmented intelligence in clinical practice.

BackgroundCoronary computed tomography angiography stenosis score (CCTA-SS) is a proposed diagnosis score that considers the plaque characteristics, myocardial function, and the diameter reduction rate of the lesions. This study aimed to evaluate the diagnostic performance of the CCTA-SS in seeking coronary artery disease (CAD).MethodsThe 228 patients with suspected CAD who underwent CCTA and invasive coronary angiography (ICA) procedures were under examination. The diagnostic performance was evaluated with the receiver operating curve (ROC) for CCTA-SS in detecting CAD (defined as a diameter reduction of ≥ 50%) and severe CAD (defined as a diameter reduction of ≥ 70%).ResultsThe area under ROC (AUC) of CCTA-SS was 0.909 (95% CI: 0.864–0.943), which was significantly higher than that of CCTA (AUC: 0.826; 95% CI: 0.771–0.873; P = 0.0352) in diagnosing of CAD with a threshold of 50%. The optimal cutoff point of CCTA-SS was 51% with a sensitivity of 90.66%, specificity of 95.65%, positive predictive value of 98.80%, negative predictive value of 72.13%, and accuracy of 91.67%, whereas the optimal cutoff point of CCTA was 55%, and the corresponding values were 87.36%, 93.48%, 98.15%, 65.15%, and 88.60%, respectively. With a threshold of 70%, the performance of CCTA-SS with an AUC of 0.927 (95% CI: 0.885–0.957) was significantly higher than that of CCTA with an AUC of 0.521 (95% CI: 0.454–0.587) (P < 0.0001).ConclusionsCCTA-SS significantly improved the diagnostic accuracy of coronary stenosis, including CAD and severe CAD, compared with CCTA.

Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For 4D time-series tomograms, this is usually handled by segmenting the constituent tomograms independently through time with 3D convolutional neural networks. Inter-volume information is therefore not utilized, potentially leading to temporal incoherence. In this paper, we attempt to resolve this by proposing two hidden Markov model variants that refine 4D segmentation labels made by 3D convolutional neural networks working on each time point. Our models utilize not only inter-volume information, but also the prediction confidence generated by the 3D segmentation convolutional neural networks themselves. To the best of our knowledge, this is the first attempt to refine 4D segmentations made by 3D convolutional neural networks using hidden Markov models. During experiments we test our models, qualitatively, quantitatively and behaviourally, using prespecified segmentations. We demonstrate in the domain of time series tomograms which are typically undersampled to allow more frequent capture; a particularly challenging problem. Finally, our dataset and code is publicly available.