An Efficient and Low-Complexity Transformer-Based Deep Learning Framework for High-Dynamic-Range Image Reconstruction.

Artifact remains a long-standing challenge in High Dynamic Range (HDR) reconstruction. Existing methods focus on model designs for artifact mitigation but ignore explicit detection and suppression strategies. Because artifact lacks clear boundaries, distinct shapes, and semantic consistency, and there is no existing dedicated dataset for HDR artifact, progress in direct artifact detection and recovery is impeded. To bridge the gap, we propose a unified HDR reconstruction framework that integrates artifact detection and model optimization. Firstly, we build the first HDR artifact dataset (HADataset), comprising 1,213 diverse multi-exposure Low Dynamic Range (LDR) image sets and 1,765 HDR image pairs with per-pixel artifact annotations. Secondly, we develop an effective HDR artifact detector (HADetector), a robust artifact detection model capable of accurately localizing HDR reconstruction artifact. HADetector plays two pivotal roles: (1) enhancing existing HDR reconstruction models through fine-tuning, and (2) serving as a non-reference image quality assessment (NR-IQA) metric, the Artifact Score (AS), which aligns closely with human visual perception for reliable quality evaluation. Extensive experiments validate the effectiveness and generalizability of our framework, including the HADataset, HADetector, fine-tuning paradigm, and AS metric. The code and datasets are available at: https://github.com/xinyueliii/hdr-artifact-detect-optimize.

In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively. Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition DOI: 10.25165/j.ijabe.20181104.4475 Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

Deep leaning in food safety and authenticity detection: An integrative review and future prospects

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.

MRI image classification of brain tumors is critical for accurate and early diagnosis. New developments in deep learning have revealed that inserting attention mechanisms into convolutional neural networks can greatly improve classification performance. This work assesses the effectiveness of Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) integrated with the ResNet50 model for brain tumor classification. The SE module recalibrates channel-wise feature responses, while CBAM enhances feature representation by combining channel and spatial attention. The SE and CBAM are combined with ResNet50, greatly increasing classification accuracy, precision, and recall when compared to the basic model, according to experimental results on MRI datasets. The suggested model employs attention mechanisms to focus valuable information selectively and suppress irrelevant information. The experiments on publicly available brain tumor datasets display great improvements over basic CNN models, with precision, recall, accuracy, as well as F1 score at 0.9914, 0.9903, 0.9945, and 0.9908, respectively. From these results, the importance of attention mechanisms in deep learning models for medical imaging is highlighted, which suggests that SE and CBAM modules can be available as more dependable and effective instruments for brain tumor classification in clinical settings. Future studies should investigate transformer-based and hybrid attention techniques to enhance automated brain tumor categorization.

Existing single image high dynamic range (HDR) reconstruction methods attempt to expand the range of illuminance. They are not effective in generating plausible textures and colors in the reconstructed results, especially for high-density pixels in ultra-high-definition (UHD) images. To address these problems, we propose a new HDR reconstruction network for UHD images by collaboratively learning color and texture details. First, we propose a dual-path network to extract the content and chromatic features at a reduced resolution of the low dynamic range (LDR) input. These two types of features are used to fit bilateral-space affine models for real-time HDR reconstruction. To extract the main data structure of the LDR input, we propose to use 3D Tucker decomposition and reconstruction to prevent pseudo edges and noise amplification in the learned bilateral grid. As a result, the high-quality content and chromatic features can be reconstructed capitalized on guided bilateral upsampling. Finally, we fuse these two full-resolution feature maps into the HDR reconstructed results. Our proposed method can achieve real-time processing for UHD images (about 160 fps). Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art HDR reconstruction approaches on public benchmarks and real-world UHD images.

Image style transformation techniques are an important research area in the field of computer vision, aiming to combine the content of one image with the artistic style of another image to create new images with the content of the input image and the style of the artistic style image. The development of this field has benefited from the rapid development of deep learning algorithms, especially the application of techniques such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN). At the heart of image style transformation is how to capture both the content and artistic style of the input image. This is typically achieved by using different levels of activation of the convolutional neural network as content and style representations. The content representation captures the object and structural information in the image, while the style representation captures the texture and colour information of the image. To capture the style of an image, a Gram matrix is typically used to measure the correlation between features at a particular level. This matrix representation allows us to capture texture information and colour distribution, thus enabling style migration. The goal of the image style transformation model is to minimise both content loss and style loss to produce synthetic images. Content loss typically uses Euclidean distance to measure the difference between content representations, while style loss involves the distance between Gram matrices. GANs have become an important tool in image style transformation. The generator network is responsible for generating synthetic images and the discriminator network evaluates the realism of the generated images. Through adversarial training, the generator and discriminator networks constantly compete to improve the quality of the generated images. Finally, this study highlights the potential applications of image style conversion techniques in the fields of art creation, image editing and virtual reality, and proposes directions for future research on deep learning algorithms and underlying principles to further improve the efficiency and quality of image style conversion techniques. The continuous development of this field will bring new opportunities for image processing and creative applications.

The World Health Organization (WHO) grading system of pancreatic neuroendocrine tumor (PNET) plays an important role in the clinical decision. The rarity of PNET often negatively affects the radiological application of deep learning algorithms due to the low availability of radiological images. We tried to investigate the feasibility of predicting WHO grades of PNET on contrast-enhanced magnetic resonance (MR) images by deep learning algorithms. Ninety-six patients with PNET underwent preoperative contrast-enhanced MR imaging. Fivefold cross-validation was used in which five iterations of training and validation were performed. Within every iteration, on the training set augmented by synthetic images generated from generative adversarial network (GAN), a convolutional neural network (CNN) was trained and its performance was evaluated on the paired internal validation set. Finally, the trained CNNs from cross-validation and their averaged counterpart were separately assessed on another ten patients from a different external validation set. Averaging the results across the five iterations in the cross-validation, for the CNN model, the average accuracy was 85.13% ± 0.44% and micro-average AUC was 0.9117 ± 0.0053. Evaluated on the external validation set, the average accuracy of the five trained CNNs ranges between 79.08 and 82.35%, and the range of micro-average AUC was between 0.8825 and 0.8932. The average accuracy and micro-average AUC of the averaged CNN were 81.05% and 0.8847, respectively. Synthetic images generated from GAN could be used to alleviate the difficulty of radiological image collection for uncommon disease like PNET. With the help of GAN, the CNN showed the potential to predict the WHO grades of PNET on contrast-enhanced MR images.

A promising way for low-illumination (LI) remote sensing images high-dynamic range (HDR) reconstruction is to model the mapping function from the input LI images to the corresponding high-quality counterpart using deep convolution neural networks. Due to various image contents, the key for achieving pleasing performance lies on comprehensively exploit the image-specific long-rang (e.g., non-local similarity, low-rank) and short-range (e.g., local similarity, texture etc.) structures in the LI images using appropriate network architecture. However, most existing methods can only exploit either short-range or long-range structures that are contentagnostic shared across all images, thus limiting their generalization capacity. To tackle this problem, we propose a dynamic long-short range structure learning framework for LR remote sensing images HDR reconstruction. In contrast to existing methods, we introduce a novel two-branch network architecture including a pixel-aware dynamic module that can adaptively exploit the pixel-aware short-range structure surrounding each pixel depending on its feature representation, and a long-range transformer module that dynamically exploit the long-range correlation between image patchesin the deep feature space. Then, the learned long-short range structures are integrated and cast into pixel-wise scaling factors of an illumination enhance module to restore the LI image. It empowers us to effectively exploit the image-specific long-short range structures of each input IL images for accurate HDR reconstruction. Experimental results on remote sensing images with different levels of IL demonstrate the effectiveness of the proposed method.

Glacial cirque identification based on Convolutional Neural Networks

Despite significant advancements in image classification, deep learning models struggle to accurately discern fine details in images, producing overly confident and imbalanced predictions for certain classes. These models typically employ feature recalibration techniques but do not account for the underlying uncertainty in predictions—particularly in complex sequential tasks like image classification. These uncertainties can significantly impact the reliability of subsequent analyses, potentially compromising accuracy across various applications. To address these limitations, we introduce the Stochastic Differential Equation Recalibration Module (SDERM), a novel approach designed to dynamically adjust the channel‐wise feature responses in convolutional neural networks. It integrates a stochastic differential equation (SDE) framework into a feature recalibration module to capture the inherent uncertainties in the data and its model predictions. To the best of our knowledge, our study is the first to explore the integration of SDE‐based feature recalibration modules in image classification. We build SDERM based on two interconnected networks—drift and diffusion network. The drift network serves as a deterministic component that approximates the predictive function of the model that systematically influences recalibrations of the predictions without considering the randomness. Concurrently, the diffusion network uses the Wiener process that captures the inherent uncertainties within the data and the network's predictions. We tested the classification accuracy of SDERM in ResNet50, ResNet101, and ResNet152 against other recalibration modules, including Squeeze‐Excitation (SE), Convolutional Block Attention Module (CBAM), Gather and Excite (GE), and Position‐Aware Recalibration Module (PARM), as well as the original Bottleneck architecture. Public image classification datasets were used, including CIFAR‐10, SVHN, FashionMNIST, and HAM10000, and their classification accuracies were evaluated using the F1 score. The proposed ResNetSDE architecture achieved state‐of‐the‐art F1 scores across four of five benchmark datasets. On Fashion‐MNIST, ResNetSDE attained an F1 score of 0.937 (CI: 0.932–0.941), outperforming all baseline recalibration methods by margins of 0.9%–1.3%. For CIFAR‐10 and CIFAR‐100, ResNetSDE achieved 0.886 (CI: 0.879–0.892) and 0.962 (CI: 0.958–0.965), respectively, surpassing ResNet‐GE and ResNet‐CBAM by 3.5% and 1.3%, respectively. ResNetSDE dominated SVHN with an F1 of 0.956 (CI: 0.953–0.958), a significant improvement over ResNet‐CBAM's 0.948 (CI: 0.945–0.951). While ResNet‐CBAM led on the class‐imbalanced HAM10000 (0.770, CI: 0.758–0.782), ResNetSDE remained competitive (0.768, CI: 0.749–0.786) since its consistent superiority—evidenced by narrow confidence intervals—validates its efficacy as a feature recalibration framework. Our experiments demonstrate that SDERM can outperform existing feature recalibration modules in image classification. The integration of SDERM to ResNet enables leveraging adaptability toward the stochasticity of each dataset at various depths of the architecture in image classification where uncertainty plays a fundamental role.

Deep learning has revolutionized the modern-day world starting with its application in computer vision such as image classification, face recognition, autonomous vehicle etc. it has been explored in various areas where human beings find it difficult to come up with solutions to the challenges at hand.By the word deep, it implies they are trained with millions, billions of parameters to achieve outstanding results.In this review paper, the fundamentals of deep learning have been discussed extensively starting with the classification, types of activation functions, different deep learning algorithms as well as their applications were also discussed.Recurrent neural network (RNNs) and its variant, convolution neural networks (CNNs) and various architectures, recursive neural networks (RvNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), generative adversarial networks (GANs) and other deep learning were discussed extensively.Some of the findings of researchers for some of these algorithms were highlighted.Based on various paper reviewed and thorough analysis carried out, it was observed that the exploration of deep learnings in this modern-day world has found applications in virtually all fields of life from medicine, academy, transportation, entertainments, particularly the exploration of CNNs, RNNs, and GANs.

Acquiring the growth information of tobacco plants quickly and accurately is an important prerequisite for their daily management. In view of the problem that the convolutional neural network is easily affected by background information, this study introduces the Convolutional Block Attention Module into the U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2-</sup> Net. This method is mainly composed of an encoder and a decoder. Before the feature map is decoded, it will go through the Convolutional Block Attention Module to filter more important features and inhibit the invalid features. In the decoder, in order to fuse the features of objects in different scales, the feature maps in the decoder are up-sampled and then concatenated. This information aggregation solves the problem of difficult object detection at different scales to some extent. The results of this method are compared with four current mainstream deep learning models(U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net, DeeplabV3+, PSP-Net, U-Net). The result shows: 1) The U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net network based on the Convolutional Block Attention Module has the highest extraction accuracy for the UAV visible light remote senseing of tobacco(accuracy was 98.53%, recall was 96.00%,F1-Score was 96.38%, kappa coefficient was 0.95, IOU was 93.02%); 2)In the extraction of the three test areas, the extraction value of this method is highly consistent with the label area value(the area ratios of a total of 13 plants in the three regions were kept between 90% and 100%),the effect is much higher than that of DeeplabV3+ and PSP-Net, and slightly higher than the extraction effect of U <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Net and U-Net. 3)In the segmentation of three typical canopy closure regions of A, B, and C, most of the accuracy factors in different regions can reach 94% and above. It can be seen that the method in this study has certain advantages in feature extraction and has certain value in practical application.

Anomaly detection in crowd surveillance videos is a critical task for ensuring public safety and security. In this research paper, we propose a comprehensive framework for anomaly detection using deep learning techniques, specifically Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Generative Adversarial Networks (GANs). Our objective is to narrow the gap between computational complexity and detection effectiveness while satisfying the demand for explainability in anomaly detection systems. The proposed framework leverages the UCSD Ped2 dataset and focuses on incorporating spatial constraints to enhance anomaly detection performance. We introduce novel approaches for feature extraction and anomaly detection using GANs, aiming to capture essential spatial and temporal information while reducing computational complexity. Specifically, we develop encoders within the GAN framework to map input data to a lower-dimensional latent space and perform anomaly detection based on the similarity between latent codes. Through extensive experimentation and evaluation, we compare the performance of RNNs, CNNs, and GANs in detecting anomalies in crowd surveillance videos. Various evaluation metrics, including precision, recall, F1 score, ROC-AUC, and PR-AUC, is considered to assess the effectiveness of each approach. Additionally, they studied hyperparameters such as learning rate, batch size, network architecture, and GAN-specific parameters to optimize anomaly detection performance. Our results demonstrate the usefulness of the proposed framework in achieving robust and explainable anomaly detection in crowd surveillance videos. By combining advanced deep learning techniques with spatial constraints and explainability considerations, our research contributes to the development of more reliable and interpretable anomaly detection systems for real-world applications.

Wind turbine rolling bearings are crucial components for ensuring the reliability and stability of wind power systems. Their failure can lead to significant economic losses and equipment downtime. Therefore, the accurate diagnosis of bearing faults is of great importance. Although existing deep learning fault diagnosis methods have achieved certain results, they still face limitations such as inadequate feature extraction capabilities, insufficient generalization to complex working conditions, and ineffective multi-scale feature capture. To address these issues, this paper proposes an advanced fault diagnosis method named the two-stream feature fusion convolutional neural network (TSFFResNet-Net). Firstly, the proposed method combines the advantages of one-dimensional convolutional neural networks (1D-ResNet) and two-dimensional convolutional neural networks (2D-ResNet). It transforms one-dimensional vibration signals into two-dimensional images through the empirical wavelet transform (EWT) method. Then, parallel convolutional kernels in 1D-ResNet and 2D-ResNet are used to extract multi-scale features, respectively. Next, the Convolutional Block Attention Module (CBAM) is introduced to enhance the network’s ability to capture key features by focusing on important features in specific channels or spatial areas. After feature fusion, CBAM is introduced again to further enhance the effect of feature fusion, ensuring that the features extracted by different network branches can be effectively integrated, ultimately providing more accurate input features for the classification task of the fully connected layer. The experimental results demonstrate that the proposed method outperforms other traditional methods and advanced convolutional neural network models on different datasets. Compared with convolutional neural network models such as LeNet-5, AlexNet, and ResNet, the proposed method achieves a significantly higher accuracy on the test set, with a stable accuracy of over 99%. Compared with other models, it shows better generalization and stability, effectively improving the overall performance of rolling bearing vibration signal fault diagnosis. The method provides an effective solution for the intelligent fault diagnosis of wind turbine rolling bearings.