Cloud Detection in Remote Sensing Images Based on Multiscale Features-Convolutional Neural Network

Introducing two Random Forest based methods for cloud detection in remote sensing images

Cloud cover is one of the major factors which affect the application of GaoFen-2 imagery. Cloud detection in GaoFen-2 imagery is fairly difficult due to the lack of enough infrared bands. This paper presents a cloud detection method for GaoFen-2 multi-spectral imagery based on the radiation transmission model. The scattering coefficient of remote sensing image is estimated by using radiation transmission, and then the cloud mask was obtained by combining the geometric and texture features in high-resolution remote sensing images. Experiments on GaoFen-2 multi-spectral images show that the accuracy of cloud detection is above 94.70%. The method proposed in this paper can effectively reduce the influence of highlighted buildings during cloud detection and achieve a high accuracy for GaoFen-2 imagery cloud detection with less bands. In addition, this paper provides an alternative distinction method for the quantitative researches of thick and thin clouds in optical satellite imagery.

The cloud covers most of the Earth's space and plays an important role in the Earth's energy balance. Moreover, the cloud is one of the most vital and active factors in the weather and climate. In addition, the cloud usually covers ground information, which causes many problems and difficulties in the processing of image registration and fusion. So cloud detection is very significant and necessary. DSCOVR (deep space climate observatory) satellite was launched in 2015. The new EPIC (earth polychromatic imaging camera) data from it has some special characteristics, such as the hemisphere scale and the wide range of band spectrum (from ultraviolet bands, visible bands to infrared bands). Hence, we propose a new cloud detection method for EPIC data in the way of normalized difference cloud index (NDCI). In our method, first, we analyze the different reflection characteristics of different bands, especially the new ultraviolet bands, and select appropriate bands to detect clouds. Combined with the applications of EPIC data bands, 340nm, 388nm, 680nm and 780nm are identified as the main research bands. Secondly, we analyze the reflection characteristics of clouds including thin clouds and residual clouds. Based on the above two aspects, we define the cloud index (CI) to detect clouds, which effectively reduces the influence of underlying surface on the cloud detection results. In order to verify the effectiveness of the proposed cloud detection method, other three cloud detection methods are compared, including the visible light cloud detection method, SVM cloud detection method and traditional NDCI cloud detection method. The experimental results show that our proposed method effectively detects thin clouds and residual clouds that are not detected by other methods, even in winter and in summer.

Cloud contamination is a critical source of errors in the data assimilation of hyperspectral infrared radiance (IR). Therefore, it is necessary to filter out cloudy observations. In this study, we review and summarize the principles and research progress of cloud detection methods for the hyperspectral IR in the past two decades. Based on the impact of IR data utilization on cloud detection results, cloud detection methods are categorized into five types, namely clear field-of-view (FOV) detection, clear channel detection, three-dimensional cloud detection, cloud-clearing and deep learning methods. Clear FOV methods and clear channel methods aim to identify the purely clear FOVs and spectral channels that are not affected by clouds, respectively. Cloud-clearing methods are used to reconstruct clear-column radiance for cloudy observations. Deep learning cloud detection methods can quickly learn the mapping relationship between infrared hyperspectral radiation characteristics and FOV cloud distribution from a large amount of infrared radiative information with known FOV cloud labels. In this paper, we discuss and provide an outlook on the key issues in current hyperspectral IR cloud detection. Specifically, we analyze and summarize the factors affecting cloud detection, such as surface background information, vertical cloud distribution, hyperspectral IR channel selection, improvements in cloud detection algorithms and model applicability. The results indicate the use of deep learning methods offer advantages in detection accuracy and algorithm efficiency of hyperspectral IR cloud detection.

Clouds significantly affect the quality of optical satellite images, which seriously limits their precise application. Recently, deep learning has been widely applied to cloud detection and has achieved satisfactory results. However, the lack of distinctive features in thin clouds and the low quality of training samples limit the cloud detection accuracy of deep learning methods, leaving space for further improvements. In this paper, we propose a weakly supervised cloud detection method that combines spectral features and a multi-scale scene-level deep network (SpecMCD) to obtain highly accurate pixel-level cloud masks. The method first utilizes a progressive training framework with a multi-scale scene-level dataset to train the multi-scale scene-level cloud detection network. Pixel-level cloud probability maps are then obtained by combining the multi-scale probability maps and cloud thickness map based on the characteristics of clouds in dense-cloud and large cloud-area images. Finally, adaptive thresholds are generated based on the differentiated regions of the scene-level cloud masks at different scales and combined with distance-weighted optimization to obtain binary cloud masks. Two datasets (i.e. WDCD and GF1MS-WHU) comprising a total of 60 Gaofen-1 multispectral (GF1-MS) images, were used to verify the effectiveness of the proposed method. Compared to the other weakly supervised cloud detection methods such as WDCD and WSFNet, the F1-score of the proposed SpecMCD method shows an improvement of over 7.82%, highlighting the superiority and potential of the SpecMCD method for cloud detection under different cloud coverage conditions.

Optical satellite images are the main source of geophysical information which can be used for various land surface studies. But one of the most challenging issue faced by these images are the severe cloud contamination. As far as optical satellite images are of concern, thick clouds will completely obstructs the observation of landscape and uneven illumination caused by thin clouds will further limits the processing of optical satellite images. So a method to eliminate the impact of clouds in optical satellite images is necessary to improve the effectiveness and availability of remote sensing data for various land surface studies. In this paper, a novel methodology for removing the uneven illumination caused by thick and thin clouds is intro-duced. For this approach, multitemporal cloud contaminated optical satellite images are required with the assumption that in each image cloud cover change significantly over a short duration of time. A cloud detection algorithm in frequency domain is used to detect the cloud contaminated portions in multitemporal optical satellite images. Suitable cloud free patch from reference images for reconstructing cloud contaminated portions in target image are selected based on lower amount of cloud rate and quality assessment technique. Some experimental analysis is conducted on collected Aqua MODIS sensor captured multitemporalsatellite images. Experimental result shows that the proposed algorithm can effectively remove majority of cloud contamination within the optical satellite images. Finally quantitative measures are performed to evaluate the feasibility of the methodology.

Urban geographical maps are important to urban planning, urban construction, land-use studies, disaster control and relief, touring and sightseeing, and so on. Satellite remote sensing images are the most important data source for urban geographical maps. However, for optical satellite remote sensing images with high spatial resolution, certain inevitable factors, including cloud, haze, and cloud shadow, severely degrade the image quality. Moreover, the geometrical and radiometric differences amongst multiple high-spatial-resolution images are difficult to eliminate. In this study, we propose a robust and efficient procedure for generating high-resolution and high-quality seamless satellite imagery for large-scale urban regions. This procedure consists of image registration, cloud detection, thin/thick cloud removal, pansharpening, and mosaicking processes. Methodologically, a spatially adaptive method considering the variation of atmospheric scattering, and a stepwise replacement method based on local moment matching are proposed for removing thin and thick clouds, respectively. The effectiveness is demonstrated by a successful case of generating a 0.91-m-resolution image of the main city zone in Nanning, Guangxi Zhuang Autonomous Region, China, using images obtained from the Chinese Beijing-2 and Gaofen-2 high-resolution satellites.

The recognition of snow versus clouds causes difficulties in cloud detection because of the similarity between cloud and snow spectral characteristics in the visible wavelength range. This paper presents a novel approach to distinguish clouds from snow to improve the accuracy of cloud detection and allow an efficient use of satellite images. Firstly, we selected thick and thin clouds from high resolution Sentinel-2 images and applied a matched filter. Secondly, the fractal digital number-frequency (DN-N) algorithm was applied to detect clouds associated with anomalies. Thirdly, spatial analyses, particularly spatial overlaying and hotspot analyses, were conducted to eliminate false anomalies. The results indicate that the method is effective for detecting clouds with various cloud covers over different areas. The resulting cloud detection effect possesses specific advantages compared to classic methods, especially for satellite images of snow and brightly colored ground objects with spectral characteristics similar to those of clouds.

Multispectral remote sensing images have shown unique advantages in many fields, including military and civilian use. Facing the difficulty in processing cloud contaminated remote sensing images, this paper proposes a multispectral remote sensing image enhancement algorithm. A model is constructed from the aspects of cloud detection and image enhancement. In the cloud detection stage, clouds are divided into thick clouds and thin clouds according to the cloud transmitability in multi-spectral images, and a multi-layer cloud detection model is established. From the perspective of traditional image processing, a bimodal pre-detection algorithm is constructed to achieve thick cloud extraction. From the perspective of deep learning, the MobileNet algorithm structure is improved to achieve thin cloud extraction. Faced with the problem of insufficient training samples, a self-supervised network is constructed to achieve training, so as to meet the requirements of high precision and high efficiency cloud detection under the condition of small samples. In the image enhancement stage, the area where the ground objects are located is determined first. Then, from the perspective of compressed sensing, the signal is analyzed from the perspective of time and frequency domains. Specifically, the inter-frame information of hyperspectral images is analyzed to construct a sparse representation model based on the principle of compressed sensing. Finally, image enhancement is achieved. The experimental comparison between our algorithm and other algorithms shows that the average Area Overlap Measure (AOM) of the proposed algorithm reaches 0.83 and the Average Gradient (AG) of the proposed algorithm reaches 12.7, which is better than the other seven algorithms by average AG 2.

Scattering in thick multifractal clouds, Part II: Multiple scattering

2MASS 1207 b, the first directly imaged planetary-mass companion, has been instrumental in advancing our understanding of exoplanets and brown dwarfs over the past 20 yr. We have performed extensive atmospheric retrieval analyses of 2MASS 1207 b’s JWST/NIRSpec spectrum using petitRADTRANS and a new atmospheric inhomogeneity framework, which characterizes homogeneous atmospheres, patchy clouds, cloud-free hot spots, or the combination of patchy clouds and spots. Among 24 retrieval runs with various assumptions, the most statistically preferred model corresponds to the patchy cloud scheme, with T eff = 117 4 − 3 + 4 K, log ( g ) = 3.6 2 − 0.02 + 0.03 dex, and R = 1.39 9 − 0.010 + 0.008 R Jup, along with near-solar atmospheric compositions of [M/H] = −0.05 ± 0.03 dex and C/O = 0.440 ± 0.012. This model suggests ∼9% of 2MASS 1207 b’s atmosphere is covered by thin iron and silicate clouds, producing L-dwarf-like spectra, while the remaining 91% consists of thick iron and silicate clouds, emitting blackbody-like spectra. These thin-cloud patches and thick-cloud regions resemble Jupiter’s belts and zones, respectively, and this scenario is consistently supported by other retrieval runs incorporating inhomogeneous atmospheres. We demonstrate that the weak CO absorption of 2MASS 1207 b can be explained by the veiling effects of patchy thick clouds; the absence of 3.3 μm CH4 absorption is attributed to its hot thermal structure, which naturally leads to a CO-dominant, CH4-deficient atmosphere. The retrieved atmospheric models also match the observed variability amplitudes of 2MASS 1207 b. Our analysis reveals that the inferred atmospheric properties show significant scatter in less statistically preferred retrieval runs but converge to consistent values among the preferred ones. This underscores the importance of exploring diverse assumptions in retrievals to avoid biased interpretations of atmospheric properties and formation pathways.

Satellite remote sensing technology is widely applied for real-time drawing maps, forecasting weather, and predicting natural disasters. However, many remote sensing satellite images contain cloud noise, resulting in difficulties in practical applications. Existing cloud removal methods are limited when an image contains lots of clouds or thick clouds. Besides, most methods need a cloudless image as a reference. Our study proposes an advanced algorithm to remove cloud noise (especially thick clouds) in remote sensing images, including a cloud segmentation model, prior knowledge refinement, and Dempster-Shafer (D-S) evidence theory. Firstly, we trained a Cloud-net model to segment cloud regions. RGB and Near Infrared (NIR) images are fed into the Cloud-net model. Then it outputs coarse segmentation images. Secondly, we introduced the prior knowledge to refine and recover segmentation results. Finally, we designed cloud removal rules based on D-S evidence theory to fuse muti-temporal remote sensing images. Our method achieves a surprising performance on GaoFen-4 (GF-4) satellite images, reducing the average percentage of cloud noise from 30\(\%\)–40\(\%\) to 2\(\%\)–8\(\%\) per image.KeywordsSatellite remote sensing imageD-S evidence theoryCloud segmentation modelMulti-temporal denoising

<strong class="journal-contentHeaderColor">Abstract.</strong> Ship tracks in subtropical marine low clouds are simulated and investigated using large eddy simulations. Five variants of a shallow subtropical stratocumulus-topped marine boundary layer (MBL) are chosen to span a range of background aerosol concentrations and variations in free-tropospheric (FT) moisture. Idealized time-invariant meteorological forcings and approximately steady-state aerosol concentrations constitute the background conditions. We investigate processes controlling cloud microphysical, macrophysical and radiative responses to aerosol injections. For the analysis, we use novel methods to decompose the liquid water path (LWP) adjustment into changes in cloud properties, and the cloud radiative effect (CRE) into contributions by cloud macro- and microphysics. The key results are that (a) the cloud top entrainment rate increases in all cases, with stronger increases for thicker than thinner clouds; (b) the drying and warming induced by increased entrainment is offset to differing degrees by corresponding responses in surface fluxes, precipitation and radiation; (c) MBL turbulence responds to changes caused by the aerosol perturbation, and this significantly affects cloud macrophysics; (d) across two days' simulation, clouds were brightened in all cases. In a pristine MBL, significant drizzle suppression by aerosol injections results not only in greater water retention, but also in turbulence intensification, leading to a significant increase in cloud amount. In this case, Twomey brightening is strongly augmented by an increase in cloud thickness and cover. In addition, a reduction in the loss of aerosol through coalescence scavenging more than offsets the entrainment dilution. This interplay precludes estimation of the lifetime of the aerosol perturbation. The combined responses of cloud macro- and microphysics lead to 10&ndash;100 times more effective cloud brightening in these cases relative to those in the non-precipitating MBL cases. In moderate and polluted MBLs entrainment enhancement makes the boundary layer drier, warmer and more stratified, leading to a decrease in cloud thickness. Counterintuitively, this LWP response offsets the greatest fraction of the Twomey brightening in a moderately moist free troposphere. This finding differs from previous studies which found larger offsets in a drier free troposphere, and results from a greater entrainment enhancement in initially thicker clouds, so the offsetting effects are weaker. The injected aerosol lifetime in cases with polluted MBLs is estimated as 2&ndash;3 days, which is longer than the estimates from satellite images.

Abstract. Ship tracks in subtropical marine low clouds are simulated and investigated using large-eddy simulations. Five variants of a shallow subtropical stratocumulus-topped marine boundary layer (MBL) are chosen to span a range of background aerosol concentrations and variations in free-tropospheric moisture. Idealized time-invariant meteorological forcings and approximately steady-state aerosol concentrations constitute the background conditions. We investigate processes controlling cloud microphysical, macrophysical, and radiative responses to aerosol injections. For the analysis, we use novel methods to decompose the liquid water path (LWP) adjustment into changes in cloud and boundary-layer properties and to decompose the cloud radiative effect (CRE) into contributions from cloud macro- and microphysics. The key results are that (a) the cloud-top entrainment rate increases in all cases, with stronger increases for thicker than thinner clouds; (b) the drying and warming induced by increased entrainment is offset to differing degrees by corresponding responses in surface fluxes, precipitation, and radiation; (c) MBL turbulence responds to changes caused by the aerosol perturbation, and this significantly affects cloud macrophysics; (d) across 2 d of simulation, clouds were brightened in all cases. In a pristine MBL, significant drizzle suppression by aerosol injections results not only in greater water retention but also in turbulence intensification, leading to a significant increase in cloud amount. In this case, Twomey brightening is strongly augmented by an increase in cloud thickness and cover. In addition, a reduction in the loss of aerosol through coalescence scavenging more than offsets the entrainment dilution. This interplay precludes estimation of the lifetime of the aerosol perturbation. The combined responses of cloud macro- and microphysics lead to 10–100 times more effective cloud brightening in these cases relative to those in the non-precipitating MBL cases. In moderate and polluted MBLs, entrainment enhancement makes the boundary layer drier, warmer, and more stratified, leading to a decrease in cloud thickness. This LWP response offsets the greatest fraction of the Twomey brightening in a moderately moist free troposphere. This finding differs from previous studies that found larger offsets in a drier free troposphere, and it results from a greater entrainment enhancement of initially thicker clouds, so the offsetting effects are weaker. The injected aerosol lifetime in cases with polluted MBLs is estimated to be 2–3 d, which is much longer than estimates of typical ship track lifetimes from satellite images.

Moderate resolution imaging spectrometer (MODIS) images are widely used in land, ocean, and atmospheric monitoring, due to their wide spectral coverage, high temporal resolution, and convenient data acquisition. Accurate cloud detection is critical to the fine processing and application of MODIS images. Owing to spatial resolution limitations and the influence of mixed pixels, most MODIS cloud detection algorithms struggle to effectively recognize of clouds and ground objects. Here, we propose a novel cloud detection method based on land surface reflectance and a multiscale feature convolutional neural network to achieve high-precision cloud detection, particularly for thin clouds and clouds over bright surface. A monthly surface reflectance dataset was constructed by MODIS products (MOD09A1) and employed to provide background information for cloud detection. Difference-based samples were obtained using surface reflectance as well MODIS images of different phases based on difference operations. The multiscale feature network (MFCD-Net) using an atrous spatial pyramid pooling and a channel and spatial attention module integrated low-level spatial features and high-level semantic information to capture multiscale features and generate a high-precision cloud mask. For cloud detection experiments and quantitative analysis, 61 MODIS images acquired at different times on various underlying surface types were used. Cloud detection results were compared to those of UNet, Deeplabv3&#x002B;, UNet&#x002B;&#x002B;, PSPNet, and top of atmosphere-based (MFCD-TOA) methods. The proposed method performed well, with the highest overall accuracy (96.55&#x0025;), precision (92.13&#x0025;), and recall (88.90&#x0025;). It improved cloud detection accuracy in various scenarios, reducing thin cloud omission and bright surface misidentification.