Analysis and performance of the adaptive multi-image matching algorithm for airborne digital sensor ADS40

The EarthCARE mission is dedicated to the study of clouds by observations from a satellite in low Earth orbit. The payload will include major radar and LIDAR instruments, supported by a multi-spectral imager (MSI) and a broadband radiometer. The paper describes development of detailed design concepts for the MSI, and analysis of critical performance parameters. The MSI will form Earth images at 500m ground sample distance (GSD) over a swath width of 150km, from a nominal platform altitude of around 400km. The task of the MSI is to provide spatial context for the single-point measurements made by the radar and LIDAR systems; it will image Earth in 7 spectral bands: one visible, one near-IR, two short-wave IR and three thermal IR. The MSI instrument will be formed in two parts: a visible-NIR-SWIR (VNS) system, radiometrically calibrated using a sunilluminated diffuser, and a thermal IR (TIR) system calibrated using cold space and an internal black-body. The VNS system will perform push-broom imaging, using linear array detectors (silicon and InGaAs) and 4 separate lenses. The TIR system will use a microbolometer array detector in a time delay and integration (TDI) mode. Critical issues discussed for the VNS system include detector selection and detailed optical design trade-offs. The latter are related to the desirability of dichroics to achieve a common aperture, which influences the calibration hardware and lens design. The TIR system’s most significant problems relate to control of random noise and bias errors, requiring optimisation of detector operation and calibration procedures.

Hyperspectral (HS) images obtained from space are useful for monitoring different natural phenomena on regional to global scales. The Environmental Mapping and Analysis Program (EnMAP) is a satellite recently launched by Germany to monitor the environment and explore the capabilities of hyperspectral sensors in the 420 and 2450 nm range of the spectrum. However, the data captured by the EnMAP mission have a ground sampling distance (GSD) of 30 m. This limits the use of the data for some applications that require higher spatial resolution (<10 m). This study examines the potential for improving the resolution of hyperspectral data using high resolution multispectral (MS) data obtained by Cubesats. Specifically, this work uses the data captured by the PlanetScope constellation, which has more than 150 CubeSats in low Earth orbit, with a high spatial and temporal resolution. The approach adopted leverages (1) the spectral capability of the hyperspectral EnMAP sensor, with a bandwidth of 6.5 nm in the visible and near infrared (VNIR) range (420–1000 nm) and 10 nm in the SWIR range (900–2450 nm), and (2) the spatial capability of the multispectral PlanetScope data, with a GSD of 3 meters, to enable significant spatial improvements due to its high spatial resolution. The main components of this work include: (i) area of interest clipping (ii) data co-registration, (iii) HS-MS data fusion, and (iv) quality assessments using the Jointly Spectral and Spatial Quality Index (QNR). In this study, a 2 km x 2 km area of interest was selected in the Malaucene region of France, where six state-of-the-art HS-MS fusion methods were evaluated: (1) fast multi-band image fusion algorithm (FUSE), (2) coupled nonnegative matrix factorization (CNMF), (3) smoothing filtered-based intensity modulation (SFIMHS), (4) maximum a posteriori stochastic mixing model (MAPSMM), (5) Hyperspectral Superresolution (HySure), and (6) generalized laplacian pyramid hypersharpening (GLPHS). Quality assessments of the enhanced data showed that high spectral and spatial fidelity are maintained, with the best performing fusion method being FUSE with a QNR of 0.625 followed by the MAPSMM method with a QNR of 0.604. Overall, this study advocates the benefits associated with the fusion of hyperspectral and multispectral data to obtain enhanced EnMAP data at 3 m GSD. 

The ability to automatically monitor agricultural fields is an important capability in precision farming, enabling steps towards more sustainable agriculture. Precise, high-resolution monitoring is a key prerequisite for targeted intervention and the selective application of agro-chemicals. The main goal of this paper is developing a novel crop/weed segmentation and mapping framework that processes multispectral images obtained from an unmanned aerial vehicle (UAV) using a deep neural network (DNN). Most studies on crop/weed semantic segmentation only consider single images for processing and classification. Images taken by UAVs often cover only a few hundred square meters with either color only or color and near-infrared (NIR) channels. Although a map can be generated by processing single segmented images incrementally, this requires additional complex information fusion techniques which struggle to handle high fidelity maps due to their computational costs and problems in ensuring global consistency. Moreover, computing a single large and accurate vegetation map (e.g., crop/weed) using a DNN is non-trivial due to difficulties arising from: (1) limited ground sample distances (GSDs) in high-altitude datasets, (2) sacrificed resolution resulting from downsampling high-fidelity images, and (3) multispectral image alignment. To address these issues, we adopt a stand sliding window approach that operates on only small portions of multispectral orthomosaic maps (tiles), which are channel-wise aligned and calibrated radiometrically across the entire map. We define the tile size to be the same as that of the DNN input to avoid resolution loss. Compared to our baseline model (i.e., SegNet with 3 channel RGB (red, green, and blue) inputs) yielding an area under the curve (AUC) of [background=0.607, crop=0.681, weed=0.576], our proposed model with 9 input channels achieves [0.839, 0.863, 0.782]. Additionally, we provide an extensive analysis of 20 trained models, both qualitatively and quantitatively, in order to evaluate the effects of varying input channels and tunable network hyperparameters. Furthermore, we release a large sugar beet/weed aerial dataset with expertly guided annotations for further research in the fields of remote sensing, precision agriculture, and agricultural robotics.

Lava flows have severe impacts on the Earth’s surface, communities, and natural habitats. Accurate modeling of lava flow paths is crucial for volcanic hazard assessment and risk mitigation. This research integrated remote sensing, geographic information systems, geomorphological techniques, and hydrologic analysis to numerically determine the optimal ground sample distance (GSD) of digital elevation models (DEMs) for delineating lava flow paths. Two volcanic regions were analyzed: the Afar Region of Ethiopia (Alu, Dalaffilla, and Erta Ale volcanoes) and the Reykjanes Peninsula of Iceland (Fagradalsfjall volcano). Forward and backward modeling methods were used to simulate potential lava flow paths. Remote sensing data, including multispectral and thermal images as well as DEMs, were utilized for analysis. A large-scale mean-shift image segmentation method was applied to identify lava objects and extract the main centerline for analysis. The forward model was applied to the fissures between the Alu and Dalaffilla volcanoes. The main lava centerline was derived from a 2008 ASTER image, while flow paths were simulated using the 2000 SRTM DEM. An optimal GSD of 35 m resulted in the lowest root mean square error (RMSE) of 146.89 m. This was validated using the 2017 Erta Ale eruption, confirming 35 m as a suitable GSD for delineating a major lava polygon. In the backward model, hydrologic analysis on resampled DEMs using TauDEM identified an optimal GSD of 38 m with an RMSE of 123.44 m. In the study of the Fagradalsfjall volcano, an optimal GSD of 71 m with an RMSE of 98.84 m was obtained, while the backward model yielded an optimal GSD of 33 m with an RMSE of 173.56 m. Statistical analyses were conducted to validate the results. This research contributed to volcanic hazard assessment by providing a framework that integrated geospatial technologies into lava flow modeling. Initiating simulation from the observed lava outlet, the forward model addressed a key limitation of the backward model. Results indicate that lava flow path accuracy was sensitive to the GSD of a DEM. The findings further demonstrate that the forward model may aid emergency response planning and risk mitigation strategies in volcanic regions.

This paper presents a near real-time automatic sub-pixel registration method of high-resolution panchromatic (PAN) and multispectral (MS) images using a graphics processing unit (GPU). In the first step, the method uses differential geo-registration to enable accurate geographic registration of PAN and MS images. Differential geo-registration normalizes PAN and MS images to the same direction and scale. There are also some residual misalignments due to the geometrical configuration of the acquisition instruments. These residual misalignments mean the PAN and MS images still have deviations after differential geo-registration. The second step is to use differential rectification with tiny facet primitive to eliminate possible residual misalignments. Differential rectification corrects the relative internal geometric distortion between PAN and MS images. The computational burden of these two steps is large, and traditional central processing unit (CPU) processing takes a long time. Due to the natural parallelism of the differential methods, these two steps are very suitable for mapping to a GPU for processing, to achieve near real-time processing while ensuring processing accuracy. This paper used GaoFen-6, GaoFen-7, ZiYuan3-02 and SuperView-1 satellite data to conduct an experiment. The experiment showed that our method’s processing accuracy is within 0.5 pixels. The automatic processing time of this method is about 2.5 s for 1 GB output data in the NVIDIA GeForce RTX 2080Ti, which can meet the near real-time processing requirements for most satellites. The method in this paper can quickly achieve high-precision registration of PAN and MS images. It is suitable for different scenes and different sensors. It is extremely robust to registration errors between PAN and MS.

To segment multi-spectral remote sensor images, feature extraction and object classification is an essential step that performs region-based segmentation instead of a pixel-based segmentation. Spectral based segmentation methods like K-Means, Mean-shift segmentation fail to extract optimal regions from multi-spectral images. In high-resolution multi-spectral images, segmentation main aim is to divide the image into set of non overlapping regions based on spatial features. In this proposed scheme, three phases are used to segment the remote sensing images. In the first phase, remote sensing image is divided into spatial blocks by applying the filter method. After the preprocessing step, watershed segmentation method is applied to get the initial segmented marked regions. In the second phase, noisy segmented regions are identified and then eliminated using statistical threshold method. In the third phase, area based reduced segmentation method is proposed to reduce the number of segmented regions. Experimental result shows the proposed approach has better performance compared to the traditional segmentation techniques in terms of time, noise and over segmentation.

For an earth observing satellite, a line rate is the number of lines which the CCD of push broom type camera scans in a second. It can be easily calculated by ground velocity divided by ground sample distance. Accurate calculation of line rate is necessary to obtain high quality image using TDI CCD. The earth observing satellite has four types of imaging missions which are strip imaging, stereo imaging, multi-point imaging, and arbitrary directional imaging. For the first three types of imaging, ground scanning direction is aligned with satellite velocity direction. Therefore, if the orbit propagation and spacecraft attitude information are available, the ground velocity and ground sample distance could be easily calculated. However, the calculation method might not be applicable to the arbitrary directional imaging. In the arbitrary directional imaging mode, the ground velocity is not fixed value which could be directly derived by orbit information. Furthermore, the ground sample distance might not be easily calculated by simple trigonometry which is possible for the other types of imaging. In this paper, we proposed a line rate calculation method for the arbitrary directional imaging. We applied spherical geometry to derive the equation of ground point which is the intersection between the line of sight vector of the camera and earth surface. The derivative of this equation for time is the ground velocity except the factor of earth rotation. By adding this equation and earth rotation factor, the true ground velocity vector could be derived. For the ground sample distance, we applied the equation of circle and ellipse for yaw angle difference. The equation of circle is used for the yaw angle representation on the plane which is orthogonal to the line of sight vector. The equation of ellipse is used for the yaw angle representation on the ground surface. We applied the proposed method to the KOMPSAT-3A (Korea Multi-Purpose Satellite 3A) mission which is the first Korean satellite with optical and infrared sensor. The satellite was launched by a Dnepr on 26 March 2015 and started normal operation on September 2015. The payload of the satellite is AEISS-A(Advanced Earth Imaging Sensor System-A) which has 0.55m GSD for panchromatic image, 2.2m GSD for multi-spectral image, and day-and-night infrared image. The main mission objective of the satellite is providing high resolution electro-optical images and infrared images for GIS application. By applying the proposed method, the line rate error was reduced to about 0.2% from 0.5% of previous method. The arbitrary directional imaging mode became a major operation mode and various application modes including due north directional imaging, pitch steering imaging, pitch step imaging are now developing. These application modes are based on technical achievement of the proposed method. In this paper, the details of line rate calculation method are described. The experimental results show the accuracy of the proposed method is less than 0.2% in average. For the application results, the mission operation of KOMPSAT-3A and arbitrary directional imaging results are described.

Rapid determination of fat content: Advanced spectroscopic methods across diverse fish species

The phenotypic parameters of crop plants can be evaluated accurately and quickly using an unmanned aerial vehicle (UAV) equipped with imaging equipment. In this study, hundreds of images of Chinese cabbage (Brassica rapa L. ssp. pekinensis) germplasm resources were collected with a low-cost UAV system and used to estimate cabbage width, length, and relative chlorophyll content (soil plant analysis development [SPAD] value). The super-resolution generative adversarial network (SRGAN) was used to improve the resolution of the original image, and the semantic segmentation network Unity Networking (UNet) was used to process images for the segmentation of each individual Chinese cabbage. Finally, the actual length and width were calculated on the basis of the pixel value of the individual cabbage and the ground sampling distance. The SPAD value of Chinese cabbage was also analyzed on the basis of an RGB image of a single cabbage after background removal. After comparison of various models, the model in which visible images were enhanced with SRGAN showed the best performance. With the validation set and the UNet model, the segmentation accuracy was 94.43%. For Chinese cabbage dimensions, the model was better at estimating length than width. The R2 of the visible-band model with images enhanced using SRGAN was greater than 0.84. For SPAD prediction, the R2 of the model with images enhanced with SRGAN was greater than 0.78. The root mean square errors of the 3 semantic segmentation network models were all less than 2.18. The results showed that the width, length, and SPAD value of Chinese cabbage predicted using UAV imaging were comparable to those obtained from manual measurements in the field. Overall, this research demonstrates not only that UAVs are useful for acquiring quantitative phenotypic data on Chinese cabbage but also that a regression model can provide reliable SPAD predictions. This approach offers a reliable and convenient phenotyping tool for the investigation of Chinese cabbage breeding traits.

Second generation disaster-monitoring microsatellite platform

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

The mineral-mapping capability of three spaceborne sensors with different spatial and spectral resolutions, the Environmental Mapping and Analysis Program (EnMap), Sentinel-2, and World View-3 (WV3), is assessed regarding bauxite mining wastes in Amphissa, Greece, with validation based on ground samples. We applied the well-established Linear Spectral Unmixing (LSU) and Spectral Angle Mapping (SAM) classification techniques utilizing endmembers of two established spectral libraries and incorporated ground data through geochemical and mineralogical analyses, X-ray fluorescence (XRF), Laser Ablation Inductively Coupled Plasma Mass Spectrometry (LA-ICP-MS), and X-ray Diffraction (XRD), to assess classification performance. The main lithologies in this area are bauxites and limestones; therefore, aluminum oxyhydroxides, calcite, and iron oxide minerals were the dominant phases as indicated by the XRF/XRD results. Almost all target minerals were mapped with the three sensors and both methods. The performance of EnMap is affected by its coarser spatial resolution despite its higher spectral resolution using these methods. Sentinel-2 is most effective for mapping iron-bearing minerals, particularly hematite, due to its higher spatial resolution and the presence of diagnostic iron oxide absorption features in the VNIR. World View 3 Shortwave Infrared (WV3-SWIR) performs better when mapping calcite, benefiting from its eight SWIR spectral bands and very high spatial resolution (3.7 m). Hematite and calcite yield the highest accuracy, especially with SAM, indicating 0.80 for Sentinel-2 (10 m) for hematite and 0.87 for WV3-SWIR (3.7 m) for calcite. AlOOH shows higher accuracy with SAM, ranging from 0.57 to 0.80 across the sensors, while LSU shows lower accuracy, ranging from 0.20 to 0.73 across the sensors. This study showcases each sensor’s ability to map minerals while also demonstrating that spectral coverage and the spatial and spectral resolution, as well as the characteristics of the selected endmembers, exert a critical influence on the accuracy of mineral mapping in mine waste.

The number of incorrect matches of SIFT keypoints will sharply increase when the SIFT-based registration algorithm is adopted for pairs of multi-spectral remote images, owing to the significant difference in the image intensity between multi-spectral images. Scale restriction criteria for keypoint matching are proposed, and experimental results demonstrate that it will greatly improve match performance.

Tensor completion via nonconvex tensor ring rank minimization with guaranteed convergence