Multi-task Cross-modal Learning for Chest X-ray Image Retrieval.

The coronavirus disease 2019 (COVID-19) led to a dramatic increase in the number of cases of patients with pneumonia worldwide. In this study, we aimed to develop an AI-assisted multistrategy image enhancement technique for chest X-ray (CXR) images to improve the accuracy of COVID-19 classification. Our new classification strategy consisted of 3 parts. First, the improved U-Net model with a variational encoder segmented the lung region in the CXR images processed by histogram equalization. Second, the residual net (ResNet) model with multidilated-rate convolution layers was used to suppress the bone signals in the 217 lung-only CXR images. A total of 80% of the available data were allocated for training and validation. The other 20% of the remaining data were used for testing. The enhanced CXR images containing only soft tissue information were obtained. Third, the neural network model with a residual cascade was used for the super-resolution reconstruction of low-resolution bone-suppressed CXR images. The training and testing data consisted of 1,200 and 100 CXR images, respectively. To evaluate the new strategy, improved visual geometry group (VGG)-16 and ResNet-18 models were used for the COVID-19 classification task of 2,767 CXR images. The accuracy of the multistrategy enhanced CXR images was verified through comparative experiments with various enhancement images. In terms of quantitative verification, 8-fold cross-validation was performed on the bone suppression model. In terms of evaluating the COVID-19 classification, the CXR images obtained by the improved method were used to train 2 classification models. Compared with other methods, the CXR images obtained based on the proposed model had better performance in the metrics of peak signal-to-noise ratio and root mean square error. The super-resolution CXR images of bone suppression obtained based on the neural network model were also anatomically close to the real CXR images. Compared with the initial CXR images, the classification accuracy rates of the internal and external testing data on the VGG-16 model increased by 5.09% and 12.81%, respectively, while the values increased by 3.51% and 18.20%, respectively, for the ResNet-18 model. The numerical results were better than those of the single-enhancement, double-enhancement, and no-enhancement CXR images. The multistrategy enhanced CXR images can help to classify COVID-19 more accurately than the other existing methods.

Pneumonia, a lung inflammation and consolidation disorder, poses diagnostic challenges necessitating accurate detection. This paper introduces an innovative automated approach using segmented lung morphology and texture attributes from Chest X-ray (CXR) images. Unlike conventional methods analyzing the entire CXR, our focus narrows to segmented lung regions. Discriminative ranking of extracted features enhances the categorization of CXR images into pneumonia and normal cases. Diverse machine learning classifiers are evaluated, yielding a compelling 86\% accuracy—validating our method's efficacy in distinguishing pneumonia from normal cases. This study offers a robust and efficient diagnostic avenue for improved pneumonia differentiation.

Development of a computer-aided tool for detection of COVID-19 pneumonia from CXR images using machine learning algorithm

The world is witnessing an unprecedented catastrophe as a result of the COVID-19 (Coronavirus Disease) epidemic, which has spread to approximately 216 nations and territories throughout the globe. A COVID-19 infection may progress to pneumonia, which can be diagnosed by CXR (Chest X-Ray) examination and should be treated as soon as possible after diagnosis. This work is intended to examine the use of Artificial Intelligence (AI) in speedy & precise diagnosis of COVID-19 pneumonia utilizing digital CXR pictures, as well as to construct a robust computer-aided application for automated classification of COVID-19 pneumonia from other pneumonia as well as normal images. In this research, we use a standard machine learning (ML) technique that is effective. The SVM (Support Vector Machine) classification technique was used in the development of the model. The purpose of this research has been to determine the role of machine learning, image processing, image segmentation, and feature extraction in fast or accurate identification of COVID19 chest X-ray or CT images. We assessed the performance of ML techniques on chest X-ray pictures as well as CT scans to COVID-19 diagnosis in this paper. The model's performance was assessed using relevant classification measures, such as accuracy, precision, recall, & F1 score, among others. The model is capable of identifying COVID-19 patients from CXRs with training accuracy of 100 percent. We believe this high-accuracy & reasonably fast Computer-Aided Diagnosis (CAD) technique might be extremely beneficial in the containment of the pandemic.

In the last century, we have passed two severe pandemics; the 1957 influenza (Asian flu) pandemic and the 1918 influenza (Spanish flu) pandemic with a high fatality rate. In the last few months, we have been again facing a new epidemic (COVID-19), which is a frighteningly high-risk disease and is globally threatening human lives. Among all attempts and presented solutions to tackle the COVID-19, a publicly available dataset of radiological imaging using chest radiography, also called chest X-ray (CXR) images, could efficiently accelerate the detection process of patients infected with COVID-19 through presented abnormalities in their chest radiography images. In this study, we have proposed a deep neural network (DNN), namely RAM-Net, a new combination of MobileNet with Dilated Depthwise Separable Convolution (DDSC), Residual blocks, and Attention augmented convolution. The network has been learned and validated using the COVIDx dataset, one of the most popular public datasets comprising the chest X-ray (CXR) images. Using this model, we could accurately identify the positive cases of COVID-19 viral infection while a new suspicious chest X-ray image is shown to the network. Our network's overall accuracy on the COVIDx test dataset was 95.33%, with a sensitivity and precision of 92% and 99% for COVID-19 cases, respectively, which are the highest results on the COVIDx dataset to date, to the best of our knowledge. Finally, we performed an audit on RAM-Net based on the Grad-CAM's interpretation to demonstrate that our proposed architecture detects SARS-CoV-2 (COVID-19) viral infection by focusing on vital factors rather than relying on irrelevant information.

The occurrence of COVID-19 disease and the pandemic has caused severe detrimental effects on global health, economy, and well-being. Ignorance and disobeying precautionary measures result in the rapid spread of this contagious infection and subsequent illnesses. There have been more than forty crore cases and fifty lakh deaths reported worldwide. Though the RT-PCR manual testing is widely carried out for diagnosing the presence of the virus in the human body, it is less reliable than chest CT and X-Ray imaging techniques. These tests are swift and assist in determining the severity of the infection with affordable costs. Hence, recent advances involve the application of artificial intelligence and deep learning techniques for automatic detection of this disease. Various datasets of X-Ray images of the chest are available online which are utilized for training and validation of results. In this paper, a dense layer Convolutional Neural Network with one twenty one layers model is used to predict the pneumonia type from the Chest X-Ray images. The results acquired from the implementation show that the model has the highest accuracy of 0.9738 while the specificity is 93.6%, and it outperforms the other similar models. Hence, the possibility of a false positive case occurring is lower and can facilitate easy diagnosis of the infection.

Tuberculosis (TB) is one of the most common diseases in developing countries such as Indonesia. TB disease oftenly attacks the human lungs due to its high oxygen pressure. The basic characteristics of TB-infected lungs are in the form of wounds, which change its texture and shape. The common detection method is by comparing the X-ray thoracic image with the tuberculin skin test (TST) and fast acid staining methods. This diagnosis method is often constrained by the availability of radiologists in the health facilities. To overcome such situation, a computer-aided diagnosis (CAD) tool is required to assist the doctor read the X-ray images easily. In this paper, a complementary detection method is proposed by using a chest X-ray (CXR) image to assist the diagnosis process. In current method, the image is segmented to reduce unnecessary parts prior to preprocessing step. We use a texture-based feature approach for detection purpose because most of the common examination methods for abnormality detection on CXRs is by analyzing the change of textural (content) CXR. In our texture approach, we use texture-based statistical properties on the histogram’s intensity. Some features that are used in this approach including Mean, Standard Deviation, Smoothness, Entropy, Root Mean Square (RMS), Variance, Kurtosis, Skewness, and Inverse Difference Moment (IDM). As a result of simulation, it indicates that CXR images can be well segmented to localize the focus area, allowing to the features can be extracted from the CXR image for further detection process. The obtained accuracy is about 93.94% and 92.86%, respectively for Shenzhen-Chest X-ray dataset and Montgomery County-Chest X-ray dataset.

Novel coronavirus disease (COVID-19) is an extremely contagious and quickly spreading coronavirus infestation. Severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS), which outbreak in 2002 and 2011, and the current COVID-19 pandemic are all from the same family of coronavirus. This work aims to classify COVID-19, SARS, and MERS chest X-ray (CXR) images using deep convolutional neural networks (CNNs). To the best of our knowledge, this classification scheme has never been investigated in the literature. A unique database was created, so-called QU-COVID-family, consisting of 423 COVID-19, 144 MERS, and 134 SARS CXR images. Besides, a robust COVID-19 recognition system was proposed to identify lung regions using a CNN segmentation model (U-Net), and then classify the segmented lung images as COVID-19, MERS, or SARS using a pre-trained CNN classifier. Furthermore, the Score-CAM visualization method was utilized to visualize classification output and understand the reasoning behind the decision of deep CNNs. Several deep learning classifiers were trained and tested; four outperforming algorithms were reported: SqueezeNet, ResNet18, InceptionV3, and DenseNet201. Original and preprocessed images were used individually and all together as the input(s) to the networks. Two recognition schemes were considered: plain CXR classification and segmented CXR classification. For plain CXRs, it was observed that InceptionV3 outperforms other networks with a 3-channel scheme and achieves sensitivities of 99.5%, 93.1%, and 97% for classifying COVID-19, MERS, and SARS images, respectively. In contrast, for segmented CXRs, InceptionV3 outperformed using the original CXR dataset and achieved sensitivities of 96.94%, 79.68%, and 90.26% for classifying COVID-19, MERS, and SARS images, respectively. The classification performance degrades with segmented CXRs compared to plain CXRs. However, the results are more reliable as the network learns from the main region of interest, avoiding irrelevant non-lung areas (heart, bones, or text), which was confirmed by the Score-CAM visualization. All networks showed high COVID-19 detection sensitivity (> 96%) with the segmented lung images. This indicates the unique radiographic signature of COVID-19 cases in the eyes of AI, which is often a challenging task for medical doctors.

The Coronavirus was detected in Wuhan, China in late 2019 and then led to a\npandemic with a rapid worldwide outbreak. The number of infected people has\nbeen swiftly increasing since then. Therefore, in this study, an attempt was\nmade to propose a new and efficient method for automatic diagnosis of Corona\ndisease from X-ray images using Deep Neural Networks (DNNs). In the proposed\nmethod, the DensNet169 was used to extract the features of the patients' Chest\nX-Ray (CXR) images. The extracted features were given to a feature selection\nalgorithm (i.e., ANOVA) to select a number of them. Finally, the selected\nfeatures were classified by LightGBM algorithm. The proposed approach was\nevaluated on the ChestX-ray8 dataset and reached 99.20% and 94.22% accuracies\nin the two-class (i.e., COVID-19 and No-findings) and multi-class (i.e.,\nCOVID-19, Pneumonia, and No-findings) classification problems, respectively.\n

Radiologists' eye movements during medical image interpretation reflect their perceptual-cognitive processes of diagnostic decisions. The eye movement data can be modeled to represent clinically relevant regions in a medical image and potentially integrated into an artificial intelligence (AI) system for automatic diagnosis in medical imaging. In this article, we first conduct a large-scale eye-tracking study involving 13 radiologists interpreting 191 chest X-ray (CXR) images, establishing a best-of-its-kind CXR visual saliency benchmark. We then perform analysis to quantify the reliability and clinical relevance of saliency maps (SMs) generated for CXR images. We develop CXR image saliency prediction method (CXRSalNet), a novel saliency prediction model that leverages radiologists' gaze information to optimize the use of unlabeled CXR images, enhancing training and mitigating data scarcity. We also demonstrate the application of our CXR saliency model in enhancing the performance of AI-powered diagnostic imaging systems.

An official announcement by the United Nations agency has confirmed an enduring deadly disease, namely coronavirus (COVID-19). While comparing to traditional methods such as blood tests and analysis, chest X-ray (CXR) images are the most important evidential techniques to find whether the patient is normal or COVID-19. In order to distinguish the COVID-19 patient among these images there is a classification method via convolution neural network (NN) techniques adapted. In this chapter, the experimental outcomes in which histogram equalization approach is utilized to process the input CXR images. To find accurate results deep learning (DL)-based convolution neural network (CNN) achieves accuracy as 50%, subsequently reshaping the input images into 224×224 along with batch size 32 undergoes training phase and it consists of 10 epochs to generate enhanced accuracy as 96.4% by tuning hyperparameters. Moreover, we applied optimization algorithm such as Adam optimizer, RMSprop optimizer and stochastic gradient descent (SGD) in which Adam and SGD achieves validation accuracy as 96.67% yields better performance in classifying images as normal and Coronavirus disease patients.

COVID-19 has engulfed over 200 nations through human-to-human transmission, either directly or indirectly. Reverse Transcription-polymerase Chain Reaction (RT-PCR) has been endorsed as a standard COVID-19 diagnostic procedure but has caveats such as low sensitivity, the need for a skilled workforce, and is time-consuming. Coronaviruses show significant manifestation in Chest X-Ray (CX-Ray) images and, thus, can be a viable option for an alternate COVID-19 diagnostic strategy. An automatic COVID-19 detection system can be developed to detect the disease, thus reducing strain on the healthcare system. This paper discusses a real-time Convolutional Neural Network (CNN)based system for COVID-19 illness prediction from CX-Ray images on the cloud. The implemented CNN model displays exemplary results, with training accuracy being 99.94% and validation accuracy reaching 98.81%. The confusion matrix was utilized to assess the models' outcome and achieved 99% precision, 98% recall, 99% F1 score, 100% training area under the curve (AUC) and 98.3% validation AUC. The same CX-Ray dataset was also employed to predict the COVID-19 disease with deep Convolution Neural Networks (DCNN), such as ResNet50, VGG19, InceptonV3, and Xception. The prediction outcome demonstrated that the present CNN was more capable than the DCNN models. The efficient CNN model was deployed to the Platform as a Service (PaaS) cloud.

Pneumonia is a respiratory infection caused by bacteria, virus or fungi. This infection affects the lungs by filling the air sacs(alveoli) with fluid or pus. Chest X-ray(CXR) imaging is the preferred diagnostic examination to locate the inflammation in the lungs. Examining the chest X-rays with high diagnostic accuracy is very difficult for the medical practitioners. To counteract this, computer-aided diagnosis system is used in this study for classifying the Chest X-ray (CXR) images into pneumonia and normal lungs. This research uses transfer learning based stacked ensemble CNN snapshots for the pneumonia prediction from the chest X-ray images. The DenseNet-121 architecture is used as the base learner. To reduce the computational time, the model is trained once to yield several CNN snapshots. The created snapshots give various details regarding the features taken from the CXR. The decision scores of multiple snapshots are combined using the stacking ensemble technique through a metalearner(Random Forest). The proposed ensemble model obtains an accuracy score of 98.36% on the publicly available Kermany dataset.

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray (CXR) images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced. Through comprehensive experiments, it is shown that networks pertained on CXR images outperform networks pretrained on natural image datasets such as ImageNet. Qualitatively and quantitatively evaluations also reveal that the predictive uncertainty estimates are statistically higher for erroneous predictions than correct predictions. Accordingly, uncertainty quantification methods are capable of flagging risky predictions with high uncertainty estimates. We also observe that ensemble methods more reliably capture uncertainties during the inference. DNN-based solutions for COVID-19 detection have been mainly proposed without any principled mechanism for risk mitigation. Previous studies have mainly focused on on generating single-valued predictions using pretrained DNNs. In this paper, we comprehensively apply and comparatively evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced for the first time. Using these new uncertainty performance metrics, we quantitatively demonstrate when we could trust DNN predictions for COVID-19 detection from chest X-rays. It is important to note the proposed novel uncertainty evaluation metrics are generic and could be applied for evaluation of probabilistic forecasts in all classification problems.