Deep learning-driven super-resolution in Raman hyperspectral imaging: Efficient high-resolution reconstruction from low-resolution data

Coastal zones are recognised sinks for plastic debris, although studying the transport and dispersion of plastic debris in nearshore waters remains challenging. The understanding of coastal processes and how these can be applied to numerical modelling studies is not yet fully mature. Most Lagrangian numerical modelling studies of plastic debris transport to date have been conducted at the basin or sub-basin level, typically using low-resolution hydrodynamic data (>2.5 km). These resolutions are generally sufficient at larger scales, but at smaller scales they create uncertainties regarding mainly the beaching of particles on the coastline and poor resolution of coastal structures or complex geometries.The LOCATE numerical model addresses these constraints by incorporating high-resolution hydrodynamic data in nested grid configurations, focusing on areas of high interest. Specifically, this has been applied to the Barcelona coastline, where the LOCATE model used a coupled current-wave dispersion module with a three-way nested grid for current data, of 2.5 km resolution from CMEMS within the study domain, a 350 m high-resolution grid from the Spanish Port Authority (Puertos del Estado) covering the Barcelona metropolitan area, and a 70 m resolution grid covering the Port and the urban coastline. The use of high-resolution hydrodynamic data with the LOCATE model was validated using drifter data. Complex geometric structures were resolved, demonstrating that particles had residence times over 18 times longer within the Port when using high-resolution data compared to only using low-resolution data. Furthermore, a beaching module that calculated the real-time distance of a particle to the shoreline was developed using high-resolution shoreline data. This provided much more realistic beaching patterns, compared to determining particle beaching using its advected velocity, as well as shoreline continuity between the different resolutions in the hydrodynamic data.These adaptations alone, however, do not address coastline or beach processes not included even in the highest-resolution hydrodynamic data. For this, a probabilistic framework was assumed, where a beaching timescale of 26.35 h was determined using a sensitivity analysis for backtracking simulations initiated in nearshore environments where particles could only cross the land-water boundary at known discharge sources. A further condition based on a minimum particle trajectory distance was also introduced to avoid artefacts. This final probabilistic configuration allowed for particle advection near the shoreline with more realistic trajectories within complex shorelines, considering stochastic elements of particle transport at localised scales. These parameterisations highlight the adaptations required to coastal and nearshore Lagrangian numerical modelling and serve as an area of future research which may be transposed to other locations.AcknowledgementsThis study was carried out within the TRAP project (Participatory Strategies for the Management of Plastic Pollution on the Transboundary Coast), which was 65% co-financed by the European Union through the Interreg VI-A Spain-France-Andorra Programme (POCTEFA 2021-2027). The objective of POCTEFA is to strengthen the economic and social integration of the Spain-France-Andorra border region

Super-resolution reconstruction in ultrahigh-field MRI

BepiColombo is an interdisciplinary mission to Mercury scheduled for launch in 2015, arriving at Mercury in 2021. It is a joint mission between ESA and JAXA consisting of 2 complementary spacecraft, the Mercury Planetary Orbiter (MPO) and the Mercury Magnetospheric Orbiter (MMO). Some of the instruments on the MPO either rely on certain solar events or are interested in variations of the magnetic or plasma environment around Mercury, which cannot be predicted or planned for. Traditionally, these instruments would measure continuously, downlink all their data back to Earth and the Scientists would identify the interesting periods. The Selective Data Downlink approach would allow experiments to have the possibility to detect these variations in a way that only interesting data will be selected for downlink. This approach would make far better use of the limited available data downlink volume, returning only high-resolution science data for the interesting periods. To achieve this, the instruments would have to generate simultaneously low-resolution and high-resolution data of the same observation that will be stored in the on-board SSMM (Solid State Mass Memory). This high-resolution science data will be stored in a dedicated instrument packet store that will not be automatically returned to the Earth. Low-resolution science data, which will include selective downlink support data from the observation, used for quick-look data analysis on ground, will be downlinked, typically via the X-band, arriving at Earth nominally 1 or 2 days after its generation. If the low-resolution science data, data from other MPO instruments, or even MMO data indicate an interesting feature, a selection of the high-resolution data stored on-board can be copied to a separate packet store for subsequent downlink via Ka-band. This concept would guarantee that only scientifically useful data will be downlinked.

Recent growth in unmanned aerial vehicle (UAV) technology have promoted the detailed mapping of individual tree species. However, the in-depth mining and comprehending of the significance of features derived from high-resolution UAV data for tree species discrimination remains a difficult task. In this study, a state-of-the-art approach combining UAV-borne light detection and ranging (LiDAR) and hyperspectral was used to classify 11 common tree species in a typical natural secondary forest in Northeast China. First, comprehensive relevant structural and spectral features were extracted. Then, the most valuable feature sets were selected by using a hybrid approach combining correlation-based feature selection with the optimized recursive feature elimination algorithm. The random forest algorithm was used to assess feature importance and perform the classification. Finally, the robustness of features derived from point clouds with different structures and hyperspectral images with different spatial resolutions was tested. Our results showed that the best classification accuracy was obtained by combining LiDAR and hyperspectral data (75.7%) compared to that based on LiDAR (60.0%) and hyperspectral (64.8%) data alone. The mean intensity of single returns and the visible atmospherically resistant index for red-edge band were the most influential LiDAR and hyperspectral derived features, respectively. The selected features were robust in point clouds with a density not lower than 5% (~5 pts/m2) and a resolution not lower than 0.3 m in hyperspectral data. Although canopy surface features were slightly different from original LiDAR features, canopy surface information was also important for tree species classification. This study proved the capabilities of UAV-borne LiDAR and hyperspectral data in natural secondary forest tree species discrimination and the potential for this approach to be transferable to other study areas.

Chapter 4 - Hyperspectral imaging combined with chemometrics

Hyperspectral imaging is concerned with the measurement, analysis, and interpretation of spectra acquired from a given scene (or specific object) at a short, medium or long distance by an airbone or satellite sensor. Over the last few years, hyperspectral image data sets have been collected for a great amount of locations over the world, using a variety of instruments for Earth observation. Despite the increasing importance of hyperspectral images in remote sensing applications, there is no common repository of hyperspectral data intended to distribute and share hyperspectral data sets in the community. Quite opposite, the hyperspectral data sets which are available for public use are spread among different storage locations and present significant heterogeneity regarding the storage format, associated meta-data (if any), or ground-truth availability. As a result, the development of a standardized hyperspectral data repository is a highly desired goal in the remote sensing community. In this paper, we take a necessary first step towards the development of a digital repository for remotely sensed hyperspectral data. The proposed system allows uploading new hyperspectral data sets along with meta-data, ground-truth and analysis results, with the ultimate goal of sharing publicly available hyperspectral images within the remote sensing community. The database has been designed in order to allow storing relevant information for the hyperspectral data available through the system, including basic image characteristics (width, height, number of bands, format) and more advanced meta-data (ground-truth information, publications in which the data has been used). The current implementation consists of a front-end to ease the management of images through a web interface, thus containing both synthetic and real hyperspectral images from two highly representative instruments, such as NASAs Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite Mining District in Nevada. Most importantly, the developed system includes a spectral unmixing-based content based image retrieval (CBIR) functionality which allows searching for images on the spectral unmixing information (spectrally pure components or endmembers and their associated abundances in the scene). This information is stored as meta-data associated to each hyperspectral image instance, and then used to search and retrieve images based on information content. This paper presents the design of the system and a preliminary validation of the unmixing-based retrieval functionality using both synthetic and real hyperspectral images stored in the database.

The benefits and limitations of crop stress detection by hyperspectral data analysis have been examined in detail. It could thereby be demonstrated that even a differentiation between healthy and fungal infected wheat stands is possible and profits by analyzing entire spectra or specifically selected spectral bands/ranges. For reasons of practicability in agriculture, spatial information about the health status of crop plants beyond a binary classification would be a major benefit. Thus, the potential of hyperspectral data for the derivation of several disease severity classes or moreover the derivation of continual disease severity has to be further examined. In the present study, a state-of-the-art regression approach using support vector machines (SVM) has been applied to hyperspectral AISA-Dual data to derive the disease severity caused by leaf rust (Puccinina recondita) in wheat. Ground truth disease ratings were realized within an experimental field. A mean correlation coefficient of r=0.69 between severities and support vector regression predicted severities could be achieved using indepent training and test data. The results show that the SVR is generally suitable for the derivation of continual disease severity values, but the crucial point is the uncertainty in the reference severity data, which is used to train the regression.

MotivationIncreasing evidence suggests that post-transcriptional ribonucleic acid (RNA) modifications regulate essential biomolecular functions and are related to the pathogenesis of various diseases. Precise identification of RNA modification sites is essential for understanding the regulatory mechanisms of RNAs. To date, many computational approaches for predicting RNA modifications have been developed, most of which were based on strong supervision enabled by base-resolution epitranscriptome data. However, high-resolution data may not be available.ResultsWe propose WeakRM, the first weakly supervised learning framework for predicting RNA modifications from low-resolution epitranscriptome datasets, such as those generated from acRIP-seq and hMeRIP-seq. Evaluations on three independent datasets (corresponding to three different RNA modification types and their respective sequencing technologies) demonstrated the effectiveness of our approach in predicting RNA modifications from low-resolution data. WeakRM outperformed state-of-the-art multi-instance learning methods for genomic sequences, such as WSCNN, which was originally designed for transcription factor binding site prediction. Additionally, our approach captured motifs that are consistent with existing knowledge, and visualization of the predicted modification-containing regions unveiled the potentials of detecting RNA modifications with improved resolution.Availability implementationThe source code for the WeakRM algorithm, along with the datasets used, are freely accessible at: https://github.com/daiyun02211/WeakRMSupplementary informationSupplementary data are available at Bioinformatics online.

Development in the field of computer-aided learning and testing have stimulated the progress of novel and efficient knowledge-based expert systems that have shown hopeful outcomes in a broad variety of practical applications. In particular, deep learning techniques have been extensively carried out to identify remote sensed data obtained by the instruments of Earth observation. Hyperspectral imaging (HSI) is an evolving area in the study of remotely sensed data due to the huge volume of information found in these images, which enables better classification and processing of the Earth’s surface by integrating ample of spatial and spectral features. Nevertheless, because of the high-dimensional data and restricted training samples available, HSI presents some crucial challenges for classification of supervised methods. In particular, it addresses the problems of spectral and spatial resolution, volume of data, and model conversion from multimedia images to hyperspectral data. Various deep learning-based architectures are currently being established to solve these limitations, showing significant results in the analysis of hyperspectral data. In this paper, we deal primarily with the hyperspectral datasets, the dimensionality curse problem, and methods for classifying those datasets using some deep neural networks (DNN), especially convolutional neural networks (CNN). We provide a comparative analysis of various dimensionality reduction (DR) and classification techniques used for finding accuracies based on the datasets used. We also explore certain hyperspectral imaging applications along with some of the research axes.

In this paper, a novel method based on wavelet transformation of functional data for accurate classification of hyperspectral images is proposed. The motivation of the proposed method is that in the hyperspectral images, the spectral curve of each pixel can be viewed as a function, mathematically. And it can make the best of the abundant spectral information of the original hyperspectral images. Based on this, we have proposed an effective hyperspectral image classification method which mainly used functional principal component analysis (FPCA) before. However, FPCA method focuses on the global features analysis of hyperspectral image data and may not well extract local structural features. In this paper, in order to better reflect the local structural features of hyperspectral images, we make use of the time-frequency localization characteristics of wavelet transformation to decompose hyperspectral images. Then, the sparse wavelet decomposition coefficients are classified by using extreme learning machine (ELM). Experimental results on two popular hyperspectral images show that our method outperforms some state-of-the-art hyperspectral image classification methods.

Multivariate curve resolution-alternating least squares (MCR-ALS) applied to hyperspectral Raman imaging is extensively used to spatially and spectrally resolve the individual, pure chemical species within complex, heterogeneous samples. A critical aspect of performing MCR-ALS with hyperspectral Raman imaging is the selection of the number of chemical components within the experimental data. Several methods have previously been proposed to determine the number of chemical components, but it remains a challenging task that if done incorrectly, can lead to the loss of chemical information. In this work, we show that the choice of 'optimal' number of factors in the MCR-ALS model may vary depending on the relative contribution of the targeted species to the overall spectral intensity. In a data set consisting of 27 hyperspectral Raman images of TiO2 polymorphs, it was observed that the more dominant species were best resolved with a parsimonious model. However, species with intensities near the noise level often needed more factors to be resolved than was predicted by standard methods. Based on the observations in this data set, we propose a new method that employs approximate reference spectra for determining optimal model complexity for identifying minor constituents with MCR-ALS.

This paper covers predicting high-resolution electricity peak demand features given lower-resolution data. This is a relevant setup as it answers whether limited higher-resolution monitoring helps to estimate future high-resolution peak loads when the high-resolution data is no longer available. That question is particularly interesting for network operators considering replacing high-resolution monitoring by predictive models due to economic considerations. We propose models to predict half-hourly minima and maxima of high-resolution (every minute) electricity load data while model inputs are of a lower resolution (30 min). We combine predictions of generalized additive models (GAM) and deep artificial neural networks (DNN), which are popular in load forecasting. We extensively analyze the prediction models, including the input parameters’ importance, focusing on load, weather, and seasonal effects. The proposed method won a data competition organized by Western Power Distribution, a British distribution network operator. In addition, we provide a rigorous evaluation study that goes beyond the competition frame to analyze the models’ robustness. The results show that the proposed methods are superior to the competition benchmark concerning the out-of-sample root mean squared error (RMSE). This holds regarding the competition month and the supplementary evaluation study, which covers an additional eleven months. Overall, our proposed model combination reduces the out-of-sample RMSE by 57.4% compared to the benchmark.

The resolution of seismic data determines the ability to characterize individual geological structures in a seismic image. Sparse spike inversion (SSI) is an effective approach for improving the resolution of seismic data. However, the basic assumption of SSI is that the strong reflectivity of the formation is sparse, which may not be a reasonable fit for weak thin-layer reflections. In this study, we propose a deep learning-based method to reconstruct high-resolution seismic data by combining information from the longitudinal reflectivity distribution and lateral geological structure features in the field data. A U-shaped network that fuses residual block and attention mechanism is used to learn the relationship between low resolution and high resolution. In addition, we use a hybrid loss function that combines <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${\ell _{1}}$ </tex-math></inline-formula> loss and structural similarity (SSIM) loss to optimize the network parameters for better distinguishing the geometrical features characterized by structural amplitude changes. To train the network, we adopt a workflow to automatically generate numerous 2-D low-resolution data and their corresponding high-resolution data. In this workflow, the prior information, such as statistical reflectivity distribution of well logs and structural features of the data are considered. The synthetic data and field data tests show that our method can work well compared to the traditional method even though only a few well logs are available. Especially in the field data example, our method recovers thin layers better and yields laterally more consistent high-resolution results.

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyperspectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively.

Discrimination of tropical forest types, dominant species, and mapping of functional guilds by hyperspectral and simulated multispectral Sentinel-2 data