Add Sharpening to the Super-Resolution Image Processing

Security is one of the most important things in our daily lives and the security business is a big industry. Security cameras are set in many areas in order to keep us safe. When a crime occurs, we often hear that police analyze the security camera footage. It takes long time and it is often the case that the footage is not helpful in the investigation at all. The reason is very simple. Many of the security camera images do not have sufficient resolution. Currently there are many super resolution (SR) technologies. In detective dramas technicians solve crimes using SR technologies turning blurry images taken with a security camera into sharp high resolution ones with a click of a button. However, in real life it is not so easy. In this paper we discuss the current issues and limitations of the SR technologies.

To identify the feasibility of deep learning-based diagnostic models for detecting and assessing lower-extremity fatigue fracture severity on plain radiographs. This retrospective study enrolled 1151 X-ray images (tibiofibula/foot: 682/469) of fatigue fractures and 2842 X-ray images (tibiofibula/foot: 2000/842) without abnormal presentations from two clinical centers. After labeling the lesions, images in a center (tibiofibula/foot: 2539/1180) were allocated at 7:1:2 for model construction, and the remaining images from another center (tibiofibula/foot: 143/131) for external validation. A ResNet-50 and a triplet branch network were adopted to construct diagnostic models for detecting and grading. The performances of detection models were evaluated with sensitivity, specificity, and area under the receiver operating characteristic curve (AUC), while grading models were evaluated with accuracy by confusion matrix. Visual estimations by radiologists were performed for comparisons with models. For the detection model on tibiofibula, a sensitivity of 95.4%/85.5%, a specificity of 80.1%/77.0%, and an AUC of 0.965/0.877 were achieved in the internal testing/external validation set. The detection model on foot reached a sensitivity of 96.4%/90.8%, a specificity of 76.0%/66.7%, and an AUC of 0.947/0.911. The detection models showed superior performance to the junior radiologist, comparable to the intermediate or senior radiologist. The overall accuracy of the diagnostic model was 78.5%/62.9% for tibiofibula and 74.7%/61.1% for foot in the internal testing/external validation set. The deep learning-based models could be applied to the radiological diagnosis of plain radiographs for assisting in the detection and grading of fatigue fractures on tibiofibula and foot. • Fatigue fractures on radiographs are relatively difficult to detect, and apt to be misdiagnosed. • Detection and grading models based on deep learning were constructed on a large cohort of radiographs with lower-extremity fatigue fractures. • The detection model with high sensitivity would help to reduce the misdiagnosis of lower-extremity fatigue fractures.

The violence detection is mostly achieved through handcrafted feature descriptors, while some researchers have also employed deep learning-based representation models for violent activity recognition. Deep learning-based models have achieved encouraging results for fight activity recognition on benchmark data sets such as hockey and movies. However, these models have limitations in learning discriminating features for violence activity classification with abrupt camera motion. This research work investigated deep representation models using transfer learning for handling the issue of abrupt camera motion. Consequently, a novel deep multi-net (DMN) architecture based on AlexNet and GoogleNet is proposed for violence detection in videos. AlexNet and GoogleNet are top-ranked pre-trained models for image classification with distinct pre-learnt potential features. The fusion of these models can yield superior performance. The proposed DMN unleashed the integrated potential by concurrently coalescing both networks. The results confirmed that DMN outperformed state-of-the-art methods by learning finest discriminating features and achieved 99.82% and 100% accuracy on hockey and movies data sets, respectively. Moreover, DMN has faster learning capability i.e. 1.33 and 2.28 times faster than AlexNet and GoogleNet, which makes it an effective learning architecture on images and videos.

To increase the productivity, it is important to manage yield and reduce defects in the semiconductor industry. One of the efforts is to identify defect patterns and control the cause factors that affects the defects. Many engineers inspect the quality of each chip and check the defect pattern on the wafer bin maps. To get the accurate and consistent classification results regardless of the level for domain knowledge or experience of engineers, deep learning-based models have recently been studied. Since most previous studies aim to classify the single-type defect patterns, it is needed to consider the mixed-type defect patterns together. Also, they require a lot of labeled data to train the deep learning-based classification model. However, defects occur extremely rarely in actual manufacturing process. Therefore, the method securing the higher accuracy in a situation where enough labeled data are not given is needed. This paper proposes a deep convolutional generative adversarial network for wafer map synthesis (DCGAN-WS) which generates the mixed-type patterns by synthesizing the single-type pattern and adding the pixel-wise summation. To maintain the characteristics of the binary pixel of the wafer bin maps, a thresholding technique is added. MixedWM38 dataset is used for the experiments, and it was verified that the mixed-type patterns were synthesized well. It helps to construct more robust model for single-type pattern classification and to generate the mixed-type patterns that have not occurred before. In the future, it is expected that this model addresses the problem of the lack of labeled data for defect pattern classification models.

Accurate and effective classification of plant diseases is critical for increasing yield and quality in agricultural production, minimizing economic losses through early detection of diseases, and implementing sustainable agriculture approaches. In this study, an approach for the detection and classification of plant leaf diseases is presented. We compare the performance of machine learning and deep learning-based models, and we use GAN-based data synthesis methods on a dataset we created to improve the model performance. For machine learning methods, ResNet-based feature extraction is performed and XGBoost, Random Forest, SVM, and InceptionV3 models are evaluated. In contrast, AlexNet, VGG16, VGG19, DenseNet, and ResNet models are examined within the scope of deep learning. The study was analyzed in three classes Phytophthora Infestans, Potassium Deficiency, and Healthy, and tested on data obtained from 21 different plant species. According to the model performances obtained, the deep learning-based ResNet model showed the highest success in all performance metrics and achieved 99% accuracy, showing superior performance compared to other methods. In the study, a comprehensive evaluation of multiple classification, GAN-based data synthesis, machine learning, and deep learning models was carried out. A valuable contribution was made to the existing studies in the literature.

Cutaneous melanoma is a highly lethal form of cancer. Developing a medical image segmentation model capable of accurately delineating melanoma lesions with high robustness and generalization presents a formidable challenge. This study draws inspiration from cellular functional characteristics and natural selection, proposing a novel medical segmentation model named the vital characteristics cellular neural network. This model incorporates vital characteristics observed in multicellular organisms, including memory, adaptation, apoptosis, and division. Memory module enables the network to rapidly adapt to input data during the early stages of training, accelerating model convergence. Adaptation module allows neurons to select the appropriate activation function based on varying environmental conditions. Apoptosis module reduces the risk of overfitting by pruning neurons with low activation values. Division module enhances the network's learning capacity by duplicating neurons with high activation values. Experimental evaluations demonstrate the efficacy of this model in enhancing the performance of neural networks for medical image segmentation. The proposed method achieves outstanding results across numerous publicly available datasets, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery. The proposed method achieves outstanding results across numerous publicly available datasets, with an F1 score of 0.901, Intersection over Union of 0.841, and Dice coefficient of 0.913, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery.

A novel multiresolution approach is presented to segment Brain MRI images using fuzzy clustering. This approach is based on the fact that the image segmentation results should be optimized simultaneously in different scales. A new fuzzy inter-scale constraint based on antistrophic diffusion linkage model is introduced, which builds an efficient linkage relationship between the high resolution images and low resolution ones. Meanwhile, this paper develops two new fuzzy distances and then embeds them into the fuzzy clustering algorithm. The distances describe the fuzzy similarity in adjacent scales effectively. Moreover, a new multiresolution framework combining the inter- and intra-scale constraints is presented. The proposed framework is robust to noise images and low contrast ones, such as medical images. Segmentation of a number of images is illustrated. The experiments show that the proposed approach can extract the objects accurately.

Generative adversarial network (GAN) has been widely applied to produce super-resolution (SR) image with real perception and texture details. In most existing SR approaches, the training objective typically measures a pixel-wise average distance between the SR and high-resolution (HR) images. However, as the degradation function of different images from HR to low resolution (LR) is generally different, optimizing such metrics often leads to certain unpleasant artificial traces. Unlike the prevalent GAN inversion methods that require expensive image-specific optimization at runtime, we present an alternative formulation by directly leveraging latent representation produced by a pretrained AutoEncoder. We call this improved method reduce dimension super-resolved GAN (RD-SRGAN). RD-SRGAN first obtains the latent feature representation of LR image by a pretrained AutoEncoder as input to the generator network. This process not only reduces noise effects but also decreases the overall computational complexity. On the other hand, the residual between the ground truth and the produced images replaces the produced images as input to the discriminator network, and a 2D zero mean Gaussian noise with controllable low variance replaces the real images as another input to the discriminator network. By leveraging the feature representation and properties of the 2D zero mean Gaussian noise, we restrict the optimization space to produce an SR image. Therefore, the residual of the generated SR images tend to approximates to a Gaussian noise, which introduces useless deviation information as little as possible. Experimental results show that RD-SRGAN can benefit from these strategies and achieve improved fidelity and naturalness comparison to existing methods. Switching the pretrained AutoEncoder allows the method to deal with images from diverse categories, e.g., remote sensing satellite imaging, medical imaging, and astronomy.

The identification of plant leaf diseases is crucial in precision agriculture, playing a pivotal role in advancing the modernization of agriculture. Timely detection and diagnosis of leaf diseases for preventive measures significantly contribute to enhancing both the quantity and quality of agricultural products, thereby fostering the in-depth development of precision agriculture. However, despite the rapid development of research on plant leaf disease identification, it still faces challenges such as insufficient agricultural datasets and the problem of deep learning-based disease identification models having numerous training parameters and insufficient accuracy. This paper proposes a plant leaf disease identification method based on improved SinGAN and improved ResNet34 to address the aforementioned issues. Firstly, an improved SinGAN called Reconstruction-Based Single Image Generation Network (ReSinGN) is proposed for image enhancement. This network accelerates model training speed by using an autoencoder to replace the GAN in the SinGAN and incorporates a Convolutional Block Attention Module (CBAM) into the autoencoder to more accurately capture important features and structural information in the images. Random pixel Shuffling are introduced in ReSinGN to enable the model to learn richer data representations, further enhancing the quality of generated images. Secondly, an improved ResNet34 is proposed for plant leaf disease identification. This involves adding CBAM modules to the ResNet34 to alleviate the limitations of parameter sharing, replacing the ReLU activation function with LeakyReLU activation function to address the problem of neuron death, and utilizing transfer learning-based training methods to accelerate network training speed. This paper takes tomato leaf diseases as the experimental subject, and the experimental results demonstrate that: (1) ReSinGN generates high-quality images at least 44.6 times faster in training speed compared to SinGAN. (2) The Tenengrad score of images generated by the ReSinGN model is 67.3, which is improved by 30.2 compared to the SinGAN, resulting in clearer images. (3) ReSinGN model with random pixel Shuffling outperforms SinGAN in both image clarity and distortion, achieving the optimal balance between image clarity and distortion. (4) The improved ResNet34 achieved an average recognition accuracy, recognition precision, recognition accuracy (redundant as it's similar to precision), recall, and F1 score of 98.57, 96.57, 98.68, 97.7, and 98.17%, respectively, for tomato leaf disease identification. Compared to the original ResNet34, this represents enhancements of 3.65, 4.66, 0.88, 4.1, and 2.47%, respectively.

Lightweight image super-resolution with feature enhancement residual network

Heart failure is a clinical syndrome that occurs when the heart is too weak or stiff and cannot pump enough blood that our body needs. It is one of the most expensive diseases due to frequent hospitalizations and emergency room visits. Reducing unnecessary rehospitalizations is also an important and challenging task that has the potential of saving healthcare costs, enabling discharge planning, and identifying patients at high risk. Therefore, this paper proposes a deep learning-based prediction model of heart failure rehospitalization during 6, 12, 24-month follow-ups after hospital discharge in patients with acute myocardial infarction (AMI). We used the Korea Acute Myocardial Infarction-National Institutes of Health (KAMIR-NIH) registry which included 13,104 patient records and 551 features. The proposed deep learning-based rehospitalization prediction model outperformed traditional machine learning algorithms such as logistic regression, support vector machine, AdaBoost, gradient boosting machine, and random forest. The performance of the proposed model was accuracy, the area under the curve, precision, recall, specificity, and F1 score of 99.37%, 99.90%, 96.86%, 98.61%, 99.49%, and 97.73%, respectively. This study showed the potential of a deep learning-based model for cardiology, which can be used for decision-making and medical diagnosis tool of heart failure rehospitalization in patients with AMI.

Colorization for medical images helps make medical visualizations more engaging, provides better visualization in 3D reconstruction, acts as an image enhancement technique for tasks such as segmentation, and makes it easier for non-specialists to perceive tissue changes and texture details in medical images in diagnosis and teaching. However, colorization algorithms have been hindered by limited semantic understanding. In addition, current colorization methods still rely on paired data, which is often not available for specific fields such as medical imaging. To address the texture detail of medical images and the scarcity of paired data, we propose a self-supervised colorization framework based on CycleGAN(Cycle-Consistent Generative Adversarial Networks), treating the colorization problem of medical images as a cross-modal domain transfer problem in color space. The proposed framework focuses on global edge features and semantic information by introducing edge-aware detectors, multi-modal discriminators, and a semantic feature fusion module. Experimental results demonstrate that our method can generate high-quality color medical images.

One of the major challenges in medical imaging is the limited availability of high-quality datasets. To address this, Generative Artificial Intelligence (Generative AI) offers a promising solution by generating synthetic medical images to augment existing datasets. This study explores the application of Deep Convolutional Generative Adversarial Networks (DCGAN) for data augmentation in CIN imaging. Two training scenarios were implemented: DCGAN with manual data augmentation and another without manual augmentation. The image quality was evaluated using the Fréchet Inception Distance (FID). The results indicate that incorporating data augmentation improves the stability of training and enhances the quality of generated images FID scores of 2.21. In contrast, training DCGAN without manual augmentation resulted in a higher FID score of 2.52, indicating lower image quality. These findings highlight the effectiveness of DCGAN in medical image augmentation and its potential to enhance deep learning-based diagnostic models for cervical cancer detection or classification.

Study design of deep learning based automatic detection of cerebrovascular diseases on medical imaging: a position paper from Chinese Association of Radiologists

Abstract. Recent advancements in healthcare underscore the critical need for innovative strategies to enhance patient outcomes. Traditional health information systems often falter when managing the immense and intricate volumes of data generated in medical settings. Deep Learning technologies provide an innovative approach, enabling the extraction of valuable insights from intricate datasets while requiring minimal human effort. Neural Health emerges as a trailblazing approach, seamlessly integrating deep learning into health information systems. Neural Health collects data from various sources, including electronic health records, medical imaging, genetic profiles and wearable devices, then processes and structures it for deep learning applications. Leveraging recurrent neural networks (RNNs), it delivers actionable insights across key healthcare domains, including medical diagnostics, treatment planning, predictive analytics, and personalized medicine. Preliminary research and clinical trials reveal that Neural Health significantly enhances healthcare outcomes. It excels in early disease direction, risk prediction, and improving patient satisfaction. Thanks to its scalable and adaptable deep learning architecture, Neural Health is poised to revolutionize health information technology, making it a versatile solution for diverse healthcare environments. Keywords: Neural Health, Deep Learning, Electronic Health Records (EHRs), Healthcare Informatics, Medical Imaging.