Applications of generative adversarial networks in medical image processing

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

Generative Adversarial Networks (GANs) have increasingly broken records in solving challenging medical image analyses problems such as medical image de-noising, segmentation, detection, classification or reconstruction. However, to the best of our knowledge, they have not been used for female pelvic multi-organ segmentation. Accurate segmentation of uterine cervical cancer (UCC) organs (i.e., bladder, vagina and tumor) from magnetic resonance imaging (MRI) is crucial for effective UCC staging. However, it is a highly challenging task due to 1) noisy MR images, 2) within-subject large variability in structure and intensity of UCC organs, and 3) across-subject variability. More importantly, there have been very limited works on how to aggregate different interactions across MRI views using GANs for multi-organ segmentation while providing context information. In this work, we propose a novel synergetic multiplex network (SMN) using multi-stage deep learning architecture based on cycle-GAN to segment pelvic multi-organ using complementary multi-view MRI, introducing three major contributions in multi-organ segmentation literature: (1) Modeling the interactions across data views using a novel multiplex architecture composed of multiple layers. Each SMN layer nests a cascade of view-specific context-aware cycle-GANs and synergistically communicates context information to other paralleled view-specific layers via multiplex coupling links. (2) SMN captures shared and complementary information between different views to segment UCC in different MRI views. (3) It enforces the spatial consistency between neighboring pixels within the same tissue for UCC segmentation. Specifically, in a gradual and deep manner, the proposed method improves the segmentation results by iteratively providing more refined context information from other views to train the next segmentation cycle-GAN in the SMN layer. We evaluated our SMN framework using 15 T2w-MR sequences with axial and sagittal views. We show that SMN is robust for the UCC segmentation task by significantly (\(p<0.05\)) outperforming comparison segmentation methods.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Applications of generative adversarial networks in neuroimaging and clinical neuroscience

This article explores the application of Generative Adversarial Networks (GANs) in improving subsurface rock analysis. GANs generate realistic images of rock features, enhancing the accuracy and efficiency of estimating rock properties. The objective is to advance the field of subsurface rock analysis, offering insights into the potential of GAN techniques. The article reviews recent developments in data-driven rock reconstruction technology, specifically focusing on GANs. It highlights GANs’ ability to generate and estimate rock features like porosity and permeability. These GAN methods are complemented by discriminators to select or reject features, making them integral to the reconstruction model. The article provides a framework for engineers and researchers to effectively utilize these techniques for rock reconstruction, offering an assessment of their strengths and limitations. Digital rock reconstruction, aided by GANs, has promising implications for site selection and recovery. GANs enhance image resolution, improving the field of view and dynamism of parameters in rock analysis. However, GANs heavily rely on high-quality data, which poses a limitation. To address this, conditional GANs have been proposed. The article comprehensively reviews the latest developments in GAN methods for digital rock reconstruction, offering valuable insights for engineers and researchers using GANs for accurate subsurface rock analysis. It also proposes ways to enhance these methods and advance the field. The novelty of this article lies in its exploration of Generative Adversarial Networks (GANs) in the context of subsurface rock analysis. It emphasizes the potential of GAN techniques to generate realistic rock images and estimate properties like porosity and permeability. Additionally, the article discusses the integration of GANs into the digital rock concept, highlighting their role in improving predictive models and supporting decision-making in the oil and gas industry's resource extraction and production strategies.

Generative adversarial network (GAN) has been extensively applied in many fields since it was first proposed in 2014. There have been ongoing attempts at improving the model and developing its extensions. The basic idea behind GAN stems from the two-player zero-sum game. Composed of a generator and a discriminator, GAN is trained through a “battle” between the two networks, where the generator tries to fool the discriminator. In contrast, the discriminator tries not to be deceived. The purpose is to estimate the potential distribution of data samples and generate new data samples. GAN is currently being studied in computer vision, language processing, and information security and is still far from reaching its potential. Since there are limited existing resources on GAN reviews, this paper provides a comprehensive review of a few of the most important GAN variants. We divided the variants into three categories: foundation, extension, and advanced, gave an overview of each architecture, and analyzed their advantages and disadvantages. We also summarized some of the most popular applications of GAN, and finally, we brought up the problems of GAN-based architectures that have yet to be solved and pointed out the future direction of GAN research and development.

Mechanical fault diagnosis is crucial for ensuring the normal operation of mechanical equipment. With the rapid development of deep learning technology, the methods based on big data-driven provide a new perspective for the fault diagnosis of machinery. However, mechanical equipment operates in the normal condition most of the time, resulting in the collected data being imbalanced, which affects the performance of mechanical fault diagnosis. As a new approach for generating data, generative adversarial network (GAN) can effectively address the issues of limited data and imbalanced data in practical engineering applications. This paper provides a comprehensive review of GAN for mechanical fault diagnosis. Firstly, the development of GAN-based mechanical fault diagnosis, the basic theory of GAN and various GAN variants (GANs) are briefly introduced. Subsequently, GANs are summarized and categorized from the perspective of labels and models, and the corresponding applications are outlined. Lastly, the limitations of current research, future challenges, future trends and selecting the GAN in the practical application are discussed.

As a promising generative modeling method, Generative Adversarial Networks are a deep-learning-based generative model, in which two networks, namely the generative network and the discriminant network, contest with each other in a game during which the generator “learns” and get trained to be able to fool the discriminator of believing a specific image is (superficially) authentic. It is widely used to generate pictures based on existing data. Based on the MNIST data set, this paper studies and analyzes the application of Generative Adversarial Networks in the generation of handwritten digital images. In addition, by studying the Deep Convolutional Generative Adversarial Network and the Conditional Generation Adversarial Network model, we compared the connections and differences between these three models in training the MNIST dataset to generate handwritten digits algorithms, and gave an optimization method for generating handwritten digits.

Adopting an accurate anomaly detection mechanism is crucial for industrial software systems in order to prevent system outages that can deteriorate system availability. However, employing a supervised machine learning technique to detect anomalies in large production scale industrial software systems is highly impractical due to the requirement of annotated data. This raises the need for comprehensive semi-supervised and unsupervised anomaly detection mechanisms. This paper presents the application of Generative Adversarial Network (GAN) based models to detect system anomalies using semi-supervised one-class learning. We show that the use of a variant of GAN known as bidirectional GAN (BiGAN) gives augmented results when compared to the traditional GAN based anomaly detection, for the selected industrial system. Moreover, the experiments clearly show that the performance of the BiGAN has a direct correlation with the dimensions of the dataset used for training. The BiGAN even tends to outperform the well-established semi-supervised One-class SVM classifier and a prominent generative network for semi-supervised anomaly detection, Variational Autoencoders (VAEs) when the size of the feature space increases.

ABSTRACTData mining and natural language processing (NLP) are fundamental fields that interact in many ways. Text mining shares many topics, such as sentiment analysis and content understanding. Combining these two fields enables more efficient mining of text data and the extraction of valuable information. In particular, the GAN (Generative Adversarial Network) architecture has achieved success in image generation and has started to be used on text data. However, training GANs is fraught with difficulties due to the complexity of text data. Linguistic studies show important differences between languages. Language is characterized by fluidity, ambiguity, and context‐sensitive interpretations, and text‐generating GAN models can struggle to deal with these complexities. The interaction between data quality, language structure, and complex interpretation can lead to inconsistency and ambiguity in the text production of GAN models. These problems are particularly pronounced when complexities such as semantic subtleties, idiomatic expressions, and context‐dependent usages come into play. Text generation is an area of GAN models used in NLP to generate language and enrich text‐based applications. Work in this area can contribute to analyzing, classifying, and processing text data. Many methods and techniques have been proposed to improve the performance of text GANs. However, some problems may be encountered in the optimization of these methods. Therefore, it is essential to use optimized methods. In conclusion, GANs can be an important tool to improve text generation in NLP. Still, they require continuous research and innovation to deal with factors such as language complexity and data quality.

Liver cancer remains one of the most common and deadly forms of cancer worldwide, posing significant diagnostic challenges due to factors like low image contrast, noise, and the complex anatomical structure of the liver. This study explores the application of Generative Adversarial Networks (GANs) for enhancing computed tomography (CT) images to improve liver cancer detection. By leveraging the capabilities of GANs, the proposed system enhances low-contrast images, reduces noise, and preserves critical structural details, facilitating better tumour segmentation and localization. The GAN-enhanced imaging system is integrated into a web-based platform that supports real-time image analysis and classification, distinguishing between malignant and normal liver tissues. This approach addresses the limitations of traditional diagnostic methods, potentially revolutionizing liver cancer diagnosis by enabling earlier detection and more accurate treatment planning. The results from applying GANs demonstrate significant improvements in image quality and diagnostic accuracy, offering a scalable and accessible solution for medical professionals worldwide. This research highlights the potential of AI-driven technologies in transforming liver cancer diagnostics, suggesting broader implications for various medical imaging applications.

Generative adversarial network in medical imaging: A review.

Multimodal medical imaging of human pathological tissues provides comprehensive information to assist in clinical diagnosis. However, due to the high cost of imaging, physiological incompatibility, and the harmfulness of radioactive tracers, multimodal medical image data remains scarce. Currently, cross-modal medical synthesis methods can generate desired modal images from existing modal images. However, most existing methods are limited to specific domains. This paper proposes an Adaptive Domain Medical Image Synthesis Method based on Generative Adversarial Networks (ADGAN) to address this issue. ADGAN achieves multidirectional medical image synthesis and ensures pathological consistency by constructing a single generator to learn the latent shared representation of multiple domains. The generator employs dense connections in shallow layers to preserve edge details and incorporates auxiliary information in deep layers to retain pathological features. Additionally, spectral normalization is introduced into the discriminator to control discriminative performance and indirectly enhance the image synthesis ability of the generator. Theoretically, it can be proved that the proposed method can be trained quickly, and spectral normalization contributes to adaptive and multidirectional synthesis. In practice, comparing with recent state-of-the-art methods, ADGAN achieves average increments of 4.7% SSIM, 6.7% MSIM, 7.3% PSNR, and 9.2% VIF.

In this paper, we propose a novel application of Generative Adversarial Networks (GAN) to the synthesis of cells imaged by fluorescence microscopy. Compared to natural images, cells tend to have a simpler and more geometric global structure that facilitates image generation. However, the correlation between the spatial pattern of different fluorescent proteins reflects important biological functions, and synthesized images have to capture these relationships to be relevant for biological applications. We adapt GANs to the task at hand and propose new models with casual dependencies between image channels that can generate multi-channel images, which would be impossible to obtain experimentally. We evaluate our approach using two independent techniques and compare it against sensible baselines. Finally, we demonstrate that by interpolating across the latent space we can mimic the known changes in protein localization that occur through time during the cell cycle, allowing us to predict temporal evolution from static images.

The outbreak of novel coronavirus disease 2019, also called COVID-19, in Wuhan, China, began in December 2019. Since its outbreak, infectious disease has rapidly spread across the globe. The testing methods adopted by the medical practitioners gave false negatives, which is a big challenge. Medical imaging using deep learning can be adopted to speed up the testing process and avoid false negatives. This work proposes a novel approach, COVID-19 GAN, to perform coronavirus disease classification using medical image synthesis by a generative adversarial network. Detecting coronavirus infections from the chest X-ray images is very crucial for its early diagnosis and effective treatment. To boost the performance of the deep learning model and improve the accuracy of classification, synthetic data augmentation is performed using generative adversarial networks. Here, the available COVID-19 positive chest X-ray images are fed into the styleGAN2 model. The styleGAN model is trained, and the data necessary for training the deep learning model for coronavirus classification is generated. The generated COVID-19 positive chest X-ray images and the normal chest X-ray images are fed into the deep learning model for training. An accuracy of 99.78% is achieved in classifying chest X-ray images using CNN binary classifier model.