Chest X-Ray Image Annotation based on Spatial Relationship Feature Extraction

The coronavirus disease 2019 (COVID-19) led to a dramatic increase in the number of cases of patients with pneumonia worldwide. In this study, we aimed to develop an AI-assisted multistrategy image enhancement technique for chest X-ray (CXR) images to improve the accuracy of COVID-19 classification. Our new classification strategy consisted of 3 parts. First, the improved U-Net model with a variational encoder segmented the lung region in the CXR images processed by histogram equalization. Second, the residual net (ResNet) model with multidilated-rate convolution layers was used to suppress the bone signals in the 217 lung-only CXR images. A total of 80% of the available data were allocated for training and validation. The other 20% of the remaining data were used for testing. The enhanced CXR images containing only soft tissue information were obtained. Third, the neural network model with a residual cascade was used for the super-resolution reconstruction of low-resolution bone-suppressed CXR images. The training and testing data consisted of 1,200 and 100 CXR images, respectively. To evaluate the new strategy, improved visual geometry group (VGG)-16 and ResNet-18 models were used for the COVID-19 classification task of 2,767 CXR images. The accuracy of the multistrategy enhanced CXR images was verified through comparative experiments with various enhancement images. In terms of quantitative verification, 8-fold cross-validation was performed on the bone suppression model. In terms of evaluating the COVID-19 classification, the CXR images obtained by the improved method were used to train 2 classification models. Compared with other methods, the CXR images obtained based on the proposed model had better performance in the metrics of peak signal-to-noise ratio and root mean square error. The super-resolution CXR images of bone suppression obtained based on the neural network model were also anatomically close to the real CXR images. Compared with the initial CXR images, the classification accuracy rates of the internal and external testing data on the VGG-16 model increased by 5.09% and 12.81%, respectively, while the values increased by 3.51% and 18.20%, respectively, for the ResNet-18 model. The numerical results were better than those of the single-enhancement, double-enhancement, and no-enhancement CXR images. The multistrategy enhanced CXR images can help to classify COVID-19 more accurately than the other existing methods.

Development of a computer-aided tool for detection of COVID-19 pneumonia from CXR images using machine learning algorithm

Radiologists' eye movements during medical image interpretation reflect their perceptual-cognitive processes of diagnostic decisions. The eye movement data can be modeled to represent clinically relevant regions in a medical image and potentially integrated into an artificial intelligence (AI) system for automatic diagnosis in medical imaging. In this article, we first conduct a large-scale eye-tracking study involving 13 radiologists interpreting 191 chest X-ray (CXR) images, establishing a best-of-its-kind CXR visual saliency benchmark. We then perform analysis to quantify the reliability and clinical relevance of saliency maps (SMs) generated for CXR images. We develop CXR image saliency prediction method (CXRSalNet), a novel saliency prediction model that leverages radiologists' gaze information to optimize the use of unlabeled CXR images, enhancing training and mitigating data scarcity. We also demonstrate the application of our CXR saliency model in enhancing the performance of AI-powered diagnostic imaging systems.

Tuberculosis (TB) is one of the most common diseases in developing countries such as Indonesia. TB disease oftenly attacks the human lungs due to its high oxygen pressure. The basic characteristics of TB-infected lungs are in the form of wounds, which change its texture and shape. The common detection method is by comparing the X-ray thoracic image with the tuberculin skin test (TST) and fast acid staining methods. This diagnosis method is often constrained by the availability of radiologists in the health facilities. To overcome such situation, a computer-aided diagnosis (CAD) tool is required to assist the doctor read the X-ray images easily. In this paper, a complementary detection method is proposed by using a chest X-ray (CXR) image to assist the diagnosis process. In current method, the image is segmented to reduce unnecessary parts prior to preprocessing step. We use a texture-based feature approach for detection purpose because most of the common examination methods for abnormality detection on CXRs is by analyzing the change of textural (content) CXR. In our texture approach, we use texture-based statistical properties on the histogram’s intensity. Some features that are used in this approach including Mean, Standard Deviation, Smoothness, Entropy, Root Mean Square (RMS), Variance, Kurtosis, Skewness, and Inverse Difference Moment (IDM). As a result of simulation, it indicates that CXR images can be well segmented to localize the focus area, allowing to the features can be extracted from the CXR image for further detection process. The obtained accuracy is about 93.94% and 92.86%, respectively for Shenzhen-Chest X-ray dataset and Montgomery County-Chest X-ray dataset.

In the last century, we have passed two severe pandemics; the 1957 influenza (Asian flu) pandemic and the 1918 influenza (Spanish flu) pandemic with a high fatality rate. In the last few months, we have been again facing a new epidemic (COVID-19), which is a frighteningly high-risk disease and is globally threatening human lives. Among all attempts and presented solutions to tackle the COVID-19, a publicly available dataset of radiological imaging using chest radiography, also called chest X-ray (CXR) images, could efficiently accelerate the detection process of patients infected with COVID-19 through presented abnormalities in their chest radiography images. In this study, we have proposed a deep neural network (DNN), namely RAM-Net, a new combination of MobileNet with Dilated Depthwise Separable Convolution (DDSC), Residual blocks, and Attention augmented convolution. The network has been learned and validated using the COVIDx dataset, one of the most popular public datasets comprising the chest X-ray (CXR) images. Using this model, we could accurately identify the positive cases of COVID-19 viral infection while a new suspicious chest X-ray image is shown to the network. Our network's overall accuracy on the COVIDx test dataset was 95.33%, with a sensitivity and precision of 92% and 99% for COVID-19 cases, respectively, which are the highest results on the COVIDx dataset to date, to the best of our knowledge. Finally, we performed an audit on RAM-Net based on the Grad-CAM's interpretation to demonstrate that our proposed architecture detects SARS-CoV-2 (COVID-19) viral infection by focusing on vital factors rather than relying on irrelevant information.

Novel coronavirus disease (COVID-19) is an extremely contagious and quickly spreading coronavirus infestation. Severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS), which outbreak in 2002 and 2011, and the current COVID-19 pandemic are all from the same family of coronavirus. This work aims to classify COVID-19, SARS, and MERS chest X-ray (CXR) images using deep convolutional neural networks (CNNs). To the best of our knowledge, this classification scheme has never been investigated in the literature. A unique database was created, so-called QU-COVID-family, consisting of 423 COVID-19, 144 MERS, and 134 SARS CXR images. Besides, a robust COVID-19 recognition system was proposed to identify lung regions using a CNN segmentation model (U-Net), and then classify the segmented lung images as COVID-19, MERS, or SARS using a pre-trained CNN classifier. Furthermore, the Score-CAM visualization method was utilized to visualize classification output and understand the reasoning behind the decision of deep CNNs. Several deep learning classifiers were trained and tested; four outperforming algorithms were reported: SqueezeNet, ResNet18, InceptionV3, and DenseNet201. Original and preprocessed images were used individually and all together as the input(s) to the networks. Two recognition schemes were considered: plain CXR classification and segmented CXR classification. For plain CXRs, it was observed that InceptionV3 outperforms other networks with a 3-channel scheme and achieves sensitivities of 99.5%, 93.1%, and 97% for classifying COVID-19, MERS, and SARS images, respectively. In contrast, for segmented CXRs, InceptionV3 outperformed using the original CXR dataset and achieved sensitivities of 96.94%, 79.68%, and 90.26% for classifying COVID-19, MERS, and SARS images, respectively. The classification performance degrades with segmented CXRs compared to plain CXRs. However, the results are more reliable as the network learns from the main region of interest, avoiding irrelevant non-lung areas (heart, bones, or text), which was confirmed by the Score-CAM visualization. All networks showed high COVID-19 detection sensitivity (> 96%) with the segmented lung images. This indicates the unique radiographic signature of COVID-19 cases in the eyes of AI, which is often a challenging task for medical doctors.

The occurrence of COVID-19 disease and the pandemic has caused severe detrimental effects on global health, economy, and well-being. Ignorance and disobeying precautionary measures result in the rapid spread of this contagious infection and subsequent illnesses. There have been more than forty crore cases and fifty lakh deaths reported worldwide. Though the RT-PCR manual testing is widely carried out for diagnosing the presence of the virus in the human body, it is less reliable than chest CT and X-Ray imaging techniques. These tests are swift and assist in determining the severity of the infection with affordable costs. Hence, recent advances involve the application of artificial intelligence and deep learning techniques for automatic detection of this disease. Various datasets of X-Ray images of the chest are available online which are utilized for training and validation of results. In this paper, a dense layer Convolutional Neural Network with one twenty one layers model is used to predict the pneumonia type from the Chest X-Ray images. The results acquired from the implementation show that the model has the highest accuracy of 0.9738 while the specificity is 93.6%, and it outperforms the other similar models. Hence, the possibility of a false positive case occurring is lower and can facilitate easy diagnosis of the infection.

We developed a rich dataset of Chest X-Ray (CXR) images to assist investigators in artificial intelligence. The data were collected using an eye-tracking system while a radiologist reviewed and reported on 1,083 CXR images. The dataset contains the following aligned data: CXR image, transcribed radiology report text, radiologist’s dictation audio and eye gaze coordinates data. We hope this dataset can contribute to various areas of research particularly towards explainable and multimodal deep learning/machine learning methods. Furthermore, investigators in disease classification and localization, automated radiology report generation, and human-machine interaction can benefit from these data. We report deep learning experiments that utilize the attention maps produced by the eye gaze dataset to show the potential utility of this dataset.

PurposeTuberculosis (TB) is a severe bacterial infectious lung disease. Millions of people die or experience severe health complications due to TB each year. Accurate, automated, and effective detection of TB is key to curing and preventing person-to-person transmission. In this regard, deep learning (DL) and machine learning (ML) techniques applied to chest X-ray (CXR) images have proved effective. Here, we present our DL- and ML-based approach for TB detection using CXR images.Material and methodsWe implemented convolutional neural network (CNN)-based pre-trained DL models, such as DenseNet121, DenseNet169, DenseNet201, ResNet152, and VGG19, as feature extractors. ML models, including support vector machine (SVM), XGBoost, logistic regression, and a DL-based custom model, were used as classifiers. A total of 2,391 CXR images from three publicly available datasets were considered.ResultsWe found three models achieving the highest value in different evaluation metrics: accuracy (99.91%) with ResNet152 and SVM; recall (99.22%), precision (99.23%), and F1-score (99.22%) with DenseNet169 and the custom classifier; and area under the curve (99.99%) with DenseNet201 and the custom classifier. Among these models, we propose DenseNet169 with the custom classifier as the best performer for potential clinical application, as it reflected a high and well-balanced performance across all evaluation metrics.ConclusionsThis study evaluated pre-trained CNN-based DL and ML models on pre-processed CXR images for the detection of TB. The DenseNet169 model with a custom classifier stands out with its high and well-balanced performance, offering a significant contribution to the effective and automated detection of TB.

Pneumonia is a respiratory infection caused by bacteria, virus or fungi. This infection affects the lungs by filling the air sacs(alveoli) with fluid or pus. Chest X-ray(CXR) imaging is the preferred diagnostic examination to locate the inflammation in the lungs. Examining the chest X-rays with high diagnostic accuracy is very difficult for the medical practitioners. To counteract this, computer-aided diagnosis system is used in this study for classifying the Chest X-ray (CXR) images into pneumonia and normal lungs. This research uses transfer learning based stacked ensemble CNN snapshots for the pneumonia prediction from the chest X-ray images. The DenseNet-121 architecture is used as the base learner. To reduce the computational time, the model is trained once to yield several CNN snapshots. The created snapshots give various details regarding the features taken from the CXR. The decision scores of multiple snapshots are combined using the stacking ensemble technique through a metalearner(Random Forest). The proposed ensemble model obtains an accuracy score of 98.36% on the publicly available Kermany dataset.

The world is witnessing an unprecedented catastrophe as a result of the COVID-19 (Coronavirus Disease) epidemic, which has spread to approximately 216 nations and territories throughout the globe. A COVID-19 infection may progress to pneumonia, which can be diagnosed by CXR (Chest X-Ray) examination and should be treated as soon as possible after diagnosis. This work is intended to examine the use of Artificial Intelligence (AI) in speedy & precise diagnosis of COVID-19 pneumonia utilizing digital CXR pictures, as well as to construct a robust computer-aided application for automated classification of COVID-19 pneumonia from other pneumonia as well as normal images. In this research, we use a standard machine learning (ML) technique that is effective. The SVM (Support Vector Machine) classification technique was used in the development of the model. The purpose of this research has been to determine the role of machine learning, image processing, image segmentation, and feature extraction in fast or accurate identification of COVID19 chest X-ray or CT images. We assessed the performance of ML techniques on chest X-ray pictures as well as CT scans to COVID-19 diagnosis in this paper. The model's performance was assessed using relevant classification measures, such as accuracy, precision, recall, & F1 score, among others. The model is capable of identifying COVID-19 patients from CXRs with training accuracy of 100 percent. We believe this high-accuracy & reasonably fast Computer-Aided Diagnosis (CAD) technique might be extremely beneficial in the containment of the pandemic.

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

Chest X-ray imaging is a crucial diagnostic tool in medical practice, used for assessing various respiratory, cardiovascular, and musculoskeletal conditions. Recent advances in machine learning and computer vision have led to the development of algorithms capable of analyzing chest X-ray images to identify signs of disease or abnormalities with high accuracy. This study focuses on the application of a customized WideResNet model, a deep learning architecture, to classify chest X-ray images quality and diagnose various diseases. Two datasets were used in this study: an in-distribution dataset containing 5000 chest X-ray images classified into four classes (Infiltration, Effusion, Atelectasis, and Mass) and an out-of-distribution dataset consisting of chest X-Rays images also classified into four classes (Cardiomegaly, Consolidation, Emphysema, No Findings). Both datasets were preprocessed using a Scharr filter to enhance image quality and reduce noise. The Scharr filter is known for its accuracy in detecting small and diagonal edges, making it a suitable choice for preprocessing chest X-ray images. The WideResNet model, an extension of the ResNet architecture, was employed for image classification. The model's architecture consists of multiple residual blocks stacked together, each containing convolutional layers, batch normalization layers, and ReLU activation functions. The skip connections within the residual blocks address the vanishing gradient problem, allowing the model to learn the identity function and preserve input information. Wide residual blocks, a key feature of the WideResNet model, were used to improve the network's capacity to learn complex features while reducing its depth. The customized WideResNet model was trained and tested on the in-distribution and the out-of-distribution dataset to evaluate its ability to generalize to other medical conditions beyond chest X-rays. The model's performance was assessed by fine-tuning hyperparameters such as network depth, residual block width, and the number of filters in each convolutional layer. This study demonstrates the potential of the WideResNet model, combined with effective preprocessing techniques, in improving the accuracy and speed of chest X-ray imaging for disease diagnosis. The application of deep learning models such as WideResNet in medical imaging can lead to better patient outcomes through faster and more accurate detection of diseases. Furthermore, the use of out-of-distribution datasets for testing showcases the model's robustness and potential applicability in various medical fields beyond chest X-ray imaging.

IEViT: An enhanced vision transformer architecture for chest X-ray image classification