Interactive defect segmentation in X-Ray images based on deep learning

X-ray imaging has been broadly adopted as a nondestructive testing method for product quality inspection. Deep learning has demonstrated powerful image scene understanding capabilities. In this article, U-Net with resnet101 is taken as the baseline for defect segmentation. First, there exist gray inhomogeneous and low-contrast regions in X-ray images, which can hardly be segmented. Contrast-limited adaptive histogram equalization (CLAHE) could be used to improve the contrast and consistency of the X-ray image. A two-stream convolutional neural network (CNN) is proposed that takes the original image and CLAHE processed image as inputs to address this issue. And then in CNN, low-level feature maps are lacking semantic information, which may lead to worse results. A gated multilayer fusion module is proposed to adaptively fuse the high-level features into low-level features. Furthermore, loss functions (such as cross entropy) in semantic segmentation are usually pixel level, ignoring the regional information. A weighted intersection over union (IOU) loss function is proposed to introduce IOU information to guide the model to focus on the objects that are easy to mine. The experimental results prove that the three proposed methods have better performance than the baseline for our dataset, achieving 42.2 in mIoU, 59.2 in Dice, and 54.5%, 74.9%, and 86.3% in small, middle, and large object recall rate, respectively.

MotivationOver the last decades, image processing and analysis have become one of the key technologies in systems biology and medicine. The quantification of anatomical structures and dynamic processes in living systems is essential for understanding the complex underlying mechanisms and allows, i.e. the construction of spatio-temporal models that illuminate the interplay between architecture and function. Recently, deep learning significantly improved the performance of traditional image analysis in cases where imaging techniques provide large amounts of data. However, if only a few images are available or qualified annotations are expensive to produce, the applicability of deep learning is still limited.ResultsWe present a novel approach that combines machine learning-based interactive image segmentation using supervoxels with a clustering method for the automated identification of similarly colored images in large image sets which enables a guided reuse of interactively trained classifiers. Our approach solves the problem of deteriorated segmentation and quantification accuracy when reusing trained classifiers which is due to significant color variability prevalent and often unavoidable in biological and medical images. This increase in efficiency improves the suitability of interactive segmentation for larger image sets, enabling efficient quantification or the rapid generation of training data for deep learning with minimal effort. The presented methods are applicable for almost any image type and represent a useful tool for image analysis tasks in general.Availability and implementationThe presented methods are implemented in our image processing software TiQuant which is freely available at tiquant.hoehme.com.Supplementary informationSupplementary data are available at Bioinformatics online.

Revision joint replacements are challenging surgical tasks. Knowing the exact type of primary prosthesis is essential to avoid long preoperative organisation, long operation times, and especially loss of bone and soft-tissue during operation. In daily routine there is often no information about the primary prosthesis. We are developing methods for identifying implanted prostheses from x-ray images by means of matching template images generated from prosthesis CAD data. The application is separated into three major components: The "Template Image Generation" adds 3d models of endoprostheses to a database. The "X-ray Image Segmentation" extracts endoprostheses from provided sets of x-ray images. The "Template Matching" finds the best matching prosthesis types in the data base. At the current stage, one prosthesis model (Corin, Knee ProthesisUniglide) was used for evaluating these algorithms. Very accurate identifications with accuracies of about 90% for lateral and over 70% for frontal images could be achieved. The current results of this feasibility study are very promising. A reliable and fast prosthesis identification process seems realistic to support the surgeon when planning and performing revision arthroplasty. Further improvements of segmentation accuracies and extending the prosthesis data base are intended next steps towards this goal.

In the task of interactive image segmentation, the Inside-Outside Guidance (IOG) algorithm has demonstrated superior segmentation performance leveraging Inside-Outside Guidance information. Nevertheless, we observe that the inconsistent input between training and testing when selecting the inside point will result in significant performance degradation. In this paper, a deep reinforcement learning framework, named Inside Point Localization Network (IPL-Net), is proposed to infer the suitable position for the inside point to help the IOG algorithm. Concretely, when a user first clicks two outside points at the symmetrical corner locations of the target object, our proposed system automatically generates the sequence of movement to localize the inside point. We then perform the IOG interactive segmentation method for precisely segmenting the target object of interest. The inside point localization problem is difficult to define as a supervised learning framework because it is expensive to collect image and their corresponding inside points. Therefore, we formulate this problem as Markov Decision Process (MDP) and then optimize it with Dueling Double Deep Q-Network (D3QN). We train our network on the PASCAL dataset and demonstrate that the network achieves excellent performance.

MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning.

Interactive segmentation (IS) using minimal prompts like points and bounding boxes facilitates rapid image annotation, which is crucial for enhancing data-driven deep learning methods. Traditional IS methods, however, process only one target per interaction, leading to inefficiency when annotating multiple identical-class objects in remote sensing imagery (RSI). To address this issue, we present a new task—identical-class object detection (ICOD) for rapid IS in RSI. This task aims to only identify and detect all identical-class targets within an image, guided by a specific category target in the image with its mask. For this task, we propose an ICOD network (ICODet) with a two-stage object detection framework, which consists of a backbone, feature similarity analysis module (S3QFM), and an identical-class object detector. In particular, the S3QFM analyzes feature similarities from images and support objects at both feature-space and semantic levels, generating similarity maps. These maps are processed by a region proposal network (RPN) to extract target-level features, which are then refined through a simple feature comparison module and classified to precisely identify identical-class targets. To evaluate the effectiveness of this method, we construct two datasets for the ICOD task: one containing a diverse set of buildings and another containing multicategory RSI objects. Experimental results show that our method outperforms the compared methods on both datasets. This research introduces a new method for rapid IS of RSI and advances the development of fast interaction modes, offering significant practical value for data production and fundamental applications in the remote sensing community.

SummaryWe developed and compared deep learning models to detect hip osteoarthritis on clinical CT. The CT-based summation images, CT-AP, that resemble X-ray radiographs can detect radiographic hip osteoarthritis and in the absence of large training data, a reliable deep learning model can be optimized by combining CT-AP and X-ray images.IntroductionIn this study, we aimed to investigate the applicability of deep learning (DL) to assess radiographic hip osteoarthritis (rHOA) on computed tomography (CT).MethodsThe study data consisted of 94 abdominopelvic clinical CTs and 5659 hip X-ray images collected from Cohort Hip and Cohort Knee (CHECK). The CT slices were sequentially summed to create radiograph-like 2-D images named CT-AP. X-ray and CT-AP images were classified as rHOA if they had osteoarthritic changes corresponding to Kellgren-Lawrence grade 2 or higher. The study data was split into 55% training, 30% validation, and 15% test sets. A pretrained ResNet18 was optimized for a classification task of rHOA vs. no-rHOA. Five models were trained using (1) X-rays, (2) downsampled X-rays, (3) combination of CT-AP and X-ray images, (4) combination of CT-AP and downsampled X-ray images, and (5) CT-AP images.ResultsAmongst the five models, Model-3 and Model-5 performed best in detecting rHOA from the CT-AP images. Model-3 detected rHOA on the test set of CT-AP images with a balanced accuracy of 82.2% and was able to discriminate rHOA from no-rHOA with an area under the receiver operating characteristic curve (ROC AUC) of 0.93 [0.75–0.99]. Model-5 detected rHOA on the test set at a balanced accuracy of 82.2% and classified rHOA from no-rHOA with an ROC AUC of 0.89 [0.67–0.97].ConclusionCT-based summation images that resemble radiographs can be used to detect rHOA. In addition, in the absence of large training data, a reliable DL model can be optimized by combining CT-AP and X-ray images.

A deep unsupervised saliency model for lung segmentation in chest X-ray images

Pes planus is a common postural deformity related to the medial longitudinal arch of the foot. Radiographic examinations are important for reproducibility and objectivity; the most commonly used methods are the calcaneal inclusion angle and Mery angle. However, there may be variations in radiographic measurements due to human error and inexperience. In this study, a deep learning (DL)-based solution is proposed to solve this problem. Lateral radiographs of the right and left foot of 289 patients were taken and saved. The study population is a homogeneous group in terms of age and gender, and does not provide sufficient heterogeneity to represent the general population. These radiography (X-ray) images were measured by 2 different experts and the measurements were recorded. According to these measurements, each X-ray image is labeled as pes planus or non-pes planus. These images were then filtered and resized using Gaussian blurring and median filtering methods. As a result of these processes, 2 separate data sets were created. Generally accepted DL models (AlexNet, GoogleNet, SqueezeNet) were reconstructed to classify these images. The 2-category (pes planus/no pes planus) data in the 2 preprocessed and resized datasets were classified by fine-tuning these reconstructed transfer learning networks. The GoogleNet and SqueezeNet models achieved 100% accuracy, while AlexNet achieved 92.98% accuracy. These results show that the predictions of the models and the measurements of expert radiologists overlap to a large extent. DL-based diagnostic methods can be used as a decision support system in the diagnosis of pes planus. DL algorithms enhance the consistency of the diagnostic process by reducing measurement variations between different observers. DL systems accelerate diagnosis by automatically performing angle measurements from X-ray images, which is particularly beneficial in busy clinical settings by saving time. DL models integrated with smartphone cameras can facilitate the diagnosis of pes planus and serve as a screening tool, especially in regions with limited access to healthcare.

The context of this work is related to the vertebra segmentation. The method we propose is based on the active shape model (ASM). An original approach taking advantage of the edge polygonal approximation was developed to locate the vertebra positions in a X-ray image. Despite the fact that segmentation results show good efficiency, the time is a key variable that has always to be optimized in a medical context. Therefore, we present how vertebra extraction can efficiently be performed in exploiting the full computing power of parallel(GPU) and heterogeneous (multi-CPU/multi-GPU) architectures. We propose a parallel hybrid implementation of the mostintensive steps enabling to boost performance. Experimentationshave been conducted using a set of high-resolution X-ray medicalimages, showing a global speedup ranging from 3 to 22, bycomparison with the CPU implementation. Data transfer timesbetween CPU and GPU memories were included in the executiontimes of our proposed implementation.

COVID-19, also known as the coronavirus disease 2019 is a highly contagious respiratory illness caused by SARS-COV-2 virus which spread form china globally resulting in pandemic. With advances in diagnostic tools, radiologic imaging is widely used for COVID-19 pneumonia diagnosis other than usual clinical and laboratory testing. In this paper, several deep learning and machine learning enhanced techniques are applied to X-Ray and CT-Scan medical images for the detection of covid-19 and also a clinical data for the prediction of the covid19. The images are preprocessed and trained using U-Net model, a popular architecture for image segmentation tasks. The accuracy and F1-score were found to be above 98% in the diagnosis of COVID-19 using CT-scan images. Further, transfer learning techniques were applied to overcome the insufficient data and to improve the training time. The binary and multi-class classification of X-ray images tasks were performed by utilizing enhanced CNN deep transfer learning architecture. An accuracy of 99% was achieved by enhanced CNN in the detection of X-ray images from COVID-19 and pneumonia.Further,using clinical dataset we compared the performance of supervised machine learning algorithms: logistic regression, random forest classifier, and XGBoost classifier. These algorithms were trained and evaluated on the preprocessed and feature-selected dataset to predict COVID-19 cases. The results showed that XGBoost classifier outperformed logistic regression and random forest classifier in predicting COVID-19 cases based on symptoms, age, gender, and test indications.

In recent years, deep reinforcement learning has combined the advantages of reinforcement learning and deep learning, and has made great progress in decision-making tasks. However, the training of deep reinforcement learning requires frequent interactions between the agent and the environment and repeated experiments. Adversaries have chances to poison the sample data collected by the agent by attacking the experimental environment in the training process, thereby bringing security risks and serious consequences to the training process of reinforcement learning. This work is committed to addressing the security risks in the field of deep reinforcement learning. However, this work improves the algorithm from the perspective of sample data filtering, and improves the security performance of deep reinforcement learning algorithm. There are two contributions in this work: one is to defend against adversarial attacks against deep reinforcement learning through cluster analysis and sample value evaluation; the other is to propose a deep reinforcement learning algorithm based on sample value evaluation on the basis of deterministic strategy gradient algorithm. The algorithm uses the clustering method to classify the sample pool, and measures the contribution value and security risk of the sample to the model training through the sample value evaluation. The classic game experiments show that the proposed algorithm is safe and effective. It reduces the threat of the agent falling into the adversarial sample attack and improves the training performance of deep reinforcement learning.

Recent works on click-based interactive segmentation have demonstrated state-of-the-art results by using various inference-time optimization schemes. These methods are significantly more computationally expensive than feedforward approaches, as they run backward gradient passes during inference. Moreover, backward passes are not supported in popular mobile frameworks, which complicates the deployment of such methods on embedded devices. In this paper, we study design choices for interactive segmentation and discover that state-of-the-art results can be obtained without any additional optimization schemes. We propose a simple feedforward model for click-based interactive segmentation that employs the segmentation masks from previous steps. It allows not only segmenting an entirely new object but also correcting an existing mask. We analyze the performance of models trained on different datasets and observe that the choice of a training dataset has a large impact on the quality of interactive segmentation. We find that the models trained on a combination of COCO and LVIS with diverse and high-quality annotations outperform all existing models. The code and trained models are available at https://github.com/saic-vul/ritm_interactive_segmentation.

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Background: The recent outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the disease corresponding to it (coronavirus disease 2019; COVID-19) has been declared a pandemic by the World Health Organization. COVID-19 has become a global crisis, shattering health care systems, and weakening economies of most countries. The current methods of testing that are employed include reverse transcription polymerase chain reaction (RT-PCR), rapid antigen testing, and lateral flow testing with RT-PCR being used as the golden standard despite its accuracy being at a mere 63%. It is a manual process which is time consuming, taking about an average of 48 hours to obtain the results. Alternative methods employing deep learning techniques and radiologic images are up and coming. Methods: In this paper, we used a dataset consisting of COVID-19 and non-COVID-19 folders for both X-Ray and CT images which contained a total number of 17,599 images. This dataset has been used to compare 3 (non-pre-trained) CNN models and 5 pre-trained models and their performances in detecting COVID-19 under various parameters like validation accuracy, training accuracy, validation loss, training loss, prediction accuracy, sensitivity and the training time required, with CT and X-Ray images separately. Results: Xception provided the highest validation accuracy (88%) when trained with the dataset containing the X- ray images while VGG19 provided the highest validation accuracy (81.2%) when CT images are used for training. Conclusions: The model, VGG16, showed the most consistent performance, with a validation accuracy of 76.6% for CT images and 87.76% for X-ray images. When comparing the results between the modalities, models trained with the X-ray dataset showed better performances than the same models trained with CT images. Hence, it can be concluded that X-ray images provide a higher accuracy in detecting COVID-19 making it an effective method for detecting COVID-19 in real life.