Attentive Generative Adversarial Network with Dual Encoder-Decoder for Shadow Removal

Effective shadow detection and shadow removal can improve the performance of fruit recognition in natural environments and provide technical support for agricultural intelligence. In this study, a superpixel segmentation method was used to divide an image into multiple small regions. Based on the superpixel segmentation results, the shadow regions and the shadowless regions of the orchard images under natural light were compared and studied. Seven shadow saliency features (SSF) were explored and analyzed for shadow detection. The SSF were used to enhance the shadow characteristics. Then, the genetic algorithm (GA) was used to optimize the parameters, and support vector machine recursive feature elimination (SVM-RFE) was used to determine the best feature combination for shadow detection. According to the best feature combination, the support vector machine (SVM) algorithm was used to determine whether each segment of the superpixel segmentation results belonged to the shadow region. Shadow removal was carried out on each detected shadow region, and a natural light image after shadow removal was obtained. Finally, the accuracy of shadow detection was tested. The experimental results showed that the average accuracy of the shadow detection algorithm in this study was 91.91%. As a result, the precision and recall for fruits recognition after shadow removal generally improved.

In this paper we present a novel method for shadow removal in single images. For each shadow region we use a trained classifier to identify a neighboring lit region of the same material. Given a pair of lit-shadow regions we perform a region relighting transformation based on histogram matching of luminance values between the shadow region and the lit region. Then, we adjust the CIELAB \(a\) and \(b\) channels of the shadow region by adding constant offsets based on the difference of the median shadow and lit pixel values. We demonstrate that our approach produces results that outperform the state of the art by evaluating our method using a publicly available benchmark dataset.

We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects from images. We then employ an inpainting network, I-Net, to further refine the results. We train and test our framework on the most challenging shadow removal dataset (ISTD). Our method improves the state-of-the-art in terms of mean absolute error (MAE) for the shadow area by 20%. Furthermore, this decomposition allows us to formulate a patch-based weakly-supervised shadow removal method. This model can be trained without any shadow- free images (that are cumbersome to acquire) and achieves competitive shadow removal results compared to state-of-the-art methods that are trained with fully paired shadow and shadow-free images. Last, we introduce SBU-Timelapse, a video shadow removal dataset for evaluating shadow removal methods.

The objective of this work is to detect shadows in images. We pose this as the problem of labeling image regions, where each region corresponds to a group of superpixels. To predict the label of each region, we train a kernel Least-Squares Support Vector Machine (LSSVM) for separating shadow and non-shadow regions. The parameters of the kernel and the classifier are jointly learned to minimize the leave-one-out cross validation error. Optimizing the leave-one-out cross validation error is typically difficult, but it can be done efficiently in our framework. Experiments on two challenging shadow datasets, UCF and UIUC, show that our region classifier outperforms more complex methods. We further enhance the performance of the region classifier by embedding it in a Markov Random Field (MRF) framework and adding pairwise contextual cues. This leads to a method that outperforms the state-of-the-art for shadow detection. In addition we propose a new method for shadow removal based on region relighting. For each shadow region we use a trained classifier to identify a neighboring lit region of the same material. Given a pair of lit-shadow regions we perform a region relighting transformation based on histogram matching of luminance values between the shadow region and the lit region. Once a shadow is detected, we demonstrate that our shadow removal approach produces results that outperform the state of the art by evaluating our method using a publicly available benchmark dataset.

Residual image and illumination estimation have been proven to be helpful for image enhancement. In this article, we propose a general framework, called RI-GAN, that exploits residual and illumination using generative adversarial networks (GANs). The proposed framework detects and removes shadows in a coarse-to-fine fashion. At the coarse stage, we employ three generators to produce a coarse shadow-removal result, a residual image, and an inverse illumination map. We also incorporate two indirect shadow-removal images via the residual image and the inverse illumination map. With the residual image, the illumination map, and the two indirect shadow-removal images as auxiliary information, the refinement stage estimates a shadow mask to identify shadow regions in the image, and then refines the coarse shadow-removal result to the fine shadow-free image. We introduce a cross-encoding module to the refinement generator, in which the use of feature-crossing can provide additional details to promote the shadow mask and the high-quality shadow-removal result. In addition, we apply data augmentation to the discriminator to reduce the dependence between representations of the discriminator and the quality of the predicted image. Experiments for shadow detection and shadow removal demonstrate that our method outperforms state-of-the-art methods. Furthermore, RI-GAN exhibits good performance in terms of image dehazing, rain removal, and highlight removal, demonstrating the effectiveness and flexibility of the proposed framework.

Existing facial image shadow removal methods predominantly rely on pre‐extracted facial features. However, these methods often fail to capitalize on the full potential of these features, resorting to simplified utilization. Furthermore, they tend to overlook the importance of low‐frequency information during the extraction of prior features, which can be easily compromised by noises. In our work, we propose a frequency‐aware shadow removal network (FSRNet) for facial image shadow removal, which utilizes the skin color and texture information in the face to help recover illumination in shadow regions. Our FSRNet uses a frequency‐domain image decomposition network to extract the low‐frequency skin color map and high‐frequency texture map from the face images, and applies a color‐texture guided shadow removal network to produce final shadow removal result. Concretely, the designed fourier sparse attention block (FSABlock) can transform images from the spatial domain to the frequency domain and help the network focus on the key information. We also introduce a skin color fusion module (CFModule) and a texture fusion module (TFModule) to enhance the understanding and utilization of color and texture features, promoting high‐quality result without color distortion and detail blurring. Extensive experiments demonstrate the superiority of the proposed method. The code is available at https://github.com/laoxie521/FSRNet.

Shadow removal and detection is a critical computer vision problem because shadows usually degrade image quality and affect downstream applications such as object detection and scene understanding. Existing methods like the benchmarking framework in Vicente et al. (2023) provide useful datasets and baseline techniques but are still plagued with accurate shadow boundary refinement and realistic shadow removal reconstruction. In this paper, we present a confidence-weighted blending hybrid CNN–Retinex model for robust shadow detection and removal. The approach initially generates initial candidate shadow masks through HSV color space thresholding, which are further filtered using a CNN-based patch classifier for precise localization. For shadow removal, we have employed a Retinex-based illumination correction technique followed by an innovative confidence-weighted blending approach for unnoticeable blending of the shadowed and non-shadowed regions. We evaluate our method using the benchmark set suggested by Vicente et al. and other real images. Experimental results indicate significant improvement in quantitative metrics (PSNR, SSIM, Precision, Recall, F1, mAP) and qualitative visual quality compared to the baseline method. key words

Understanding shadows from a single image consists of two types of task in previous studies, containing shadow detection and shadow removal. In this paper, we present a multi-task perspective, which is not embraced by any existing work, to jointly learn both detection and removal in an end-to-end fashion that aims at enjoying the mutually improved benefits from each other. Our framework is based on a novel STacked Conditional Generative Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each with a generator and a discriminator. Specifically, a shadow image is fed into the first generator which produces a shadow detection mask. That shadow image, concatenated with its predicted mask, goes through the second generator in order to recover its shadow-free image consequently. In addition, the two corresponding discriminators are very likely to model higher level relationships and global scene characteristics for the detected shadow region and reconstruction via removing shadows, respectively. More importantly, for multi-task learning, our design of stacked paradigm provides a novel view which is notably different from the commonly used one as the multi-branch version. To fully evaluate the performance of our proposed framework, we construct the first large-scale benchmark with 1870 image triplets (shadow image, shadow mask image, and shadow-free image) under 135 scenes. Extensive experimental results consistently show the advantages of STC-GAN over several representative state-of-the-art methods on two large-scale publicly available datasets and our newly released one.

In the course of moving object detection, the shadow of objects may arise many problems, such as the distortion of the object shape, the adhesion between different objects and so on.To cope with these problems, a fast shadow removal method based on video image is proposed.By the computing model based on YUV color space, the distortion coefficient between moving object and background can be obtained, and then the shadow can be removed by this value.Finally, experimental results show that the method for shadow removal can detect the shadow and separate several joined objects correctly.

One of the major challenges for computer vision-based lane detection is the manifestation of shadows and other vehicles. It is challenging to diagnose the chaotic lanes when it has both the shadowed and unshadowed areas. Detection and removal of shadow is the procedure to intensify the computer vision application including image processing, video processing, etc. Shadows are partial darkness or obscurity within a part of space from which rays from a source of light are cut off by an interposed opaque body. The presence of shadows results in severe issues in road detection as the bounding line of shadows can be incorrectly recognized leading to a higher false rate detection. This paper attempts a simple and efficient algorithm to detect shadows in an image by finding out the shadow ratio and calculating the threshold and then creating a shadow boundary.

Shadow removal plays an important role in moving object detection and tracking. In this paper, a new shadow removal method is introduced for the accurate identification of moving objects. This method firstly uses background subtraction to detect moving objects that have casted shadows. Secondly, a background reconstruction scheme is developed to create a dynamically updated background reference image. Thirdly, shadows are removed by comparing and analysing the reference image with the original video frame. This method is capable of detecting and removing shadows in different environments under different lighting conditions. Five benchmark videos typically used to test the effectiveness of shadow removal techniques and two user recorded videos have been examined using the proposed method. Experimental results have shown that the method can produce satisfactory results with regard to robustness and consistency.

In this paper, we propose a new shadow removal method from an image by extracting an illumination-invariant feature representation for road scene understanding. We model the invariant representation based on a 3D log chromaticity feature space, where the radiance of the same type of material lies in a flat embedding. We exploit singular value decomposition (SVD) to extract a planar structure from this space. The illumination-invariant feature of each pixel can be calculated as the distance from it to the planar structure in the 3D log chromaticity feature space. To apply the obtained illumination-invariant feature representation, we design a vanishing point detection method and a road segmentation method based on our shadow removal scheme. Experiments show that our proposed approach is effective in terms of qualitative and quantitative comparisons with the other shadow removal works.

Розглянуто проблему ідентифікації та компенсації тіней на космічних знімках високого просторового розрізнення. У роботі використано знімки міської території, отримані супутником WorldView-3. Наявність тіней на знімках може привести до втрати корисної інформації і навіть помилок в роботі алгоритмів розпізнавання, виявлення, відстеження і класифікації об’єктів. Запропоновано новий алгоритм автоматичного виявлення та усунення тіней цифрових космічних знімків, що дозволяє відновити освітленість та підвищити якість цих знімків. Для цього використано перехід до кольорової метрики HSV, вейвлет-перетворення та контурну сегментацію. Порівняння кількісних показників, а також візуальні результати показали перевагу використання запропонованого алгоритму. Результати роботи можуть бути використані при подальшому розпізнаванні об’єктів та тематичній обробці космічних знімків.

In this paper, we propose an improved hybrid approach for text generation that combines the strengths of generative adversarial networks (GANs) and recurrent neural networks (RNNs). The proposed model, named TextGen-GAN, uses a GAN to generate high-level semantic representations of text, which are then passed through an RNN decoder to produce coherent and diverse text. The suggested method produces superior results in terms of both quality and diversity when tested on a number of benchmark datasets. The obtained results show that the combination of GAN and RNN outperforms GAN alone, RNN only, baseline, and state-of-the-art models, demonstrating the effectiveness of the proposed hybrid approach.

We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image that allows the shadow image to be expressed as a combination of the shadow-free image, the shadow parameters, and a matte layer. We use two deep networks, namely SP-Net and M-Net, to predict the shadow parameters and the shadow matte respectively. This system allows us to remove the shadow effects on the images. We train and test our framework on the most challenging shadow removal dataset (ISTD). Compared to the state-of-the-art method, our model achieves a 40% error reduction in terms of root mean square error (RMSE) for the shadow area, reducing RMSE from 13.3 to 7.9. Moreover, we create an augmented ISTD dataset based on an image decomposition system by modifying the shadow parameters to generate new synthetic shadow images. Training our model on this new augmented ISTD dataset further lowers the RMSE on the shadow area to 7.4.