A Transformer-based multimodal-learning framework using sky images for ultra-short-term solar irradiance forecasting

As the deployment of solar power utilizing devices increases, accurate forecasts of solar energy become increasingly important for managing and stabilizing the electricity grid. This study investigates the validity of 7-day ahead global horizontal irradiance (GHI) forecasts for South Africa provided by the Global Forecast System (GFS). The clear-sky index was used to interpolate GFS mixed 3-hourly and 6-hourly averages to generate hourly forecasts. The model’s performance was validated against radiometric GHI data from 15 locations spread across South Africa’s six climatic zones. The results showed that the GFS model can satisfactorily forecast GHI in South Africa. Overall, the model performed best in interior climatic zones and relatively poorly in coastal climatic zones. The accuracy of the GFS model performance decreases as the forecast lead time increases. The model overestimates GHI in overcast conditions while it underestimates GHI in clear and partly cloudy conditions. On average, the model underestimates GHI in all seasons. Forecasting errors are highest in summer and lowest in winter and moderate in autumn and spring. The inadequacy of the GFS model to precisely determine the cloud quantities is the major source of error; errors increase as the cloud fraction increases. Despite this, the study’s findings suggest that GFS forecasts may be used for forecasting solar photovoltaic output for energy planning in South Africa. Significance Statement This paper provides the first comprehensive assessment of Global Forecast System (GFS) forecasts of global horizontal irradiance (GHI) in six climatic regions of South Africa, exploring how cloud cover, site location, climatic zone, season, forecast horizon, and hour of the day impact the model’s performance. This research was carried out in a country where GFS GHI models have never been validated before and that presents a wide range of climatic conditions. The results allow the accuracy of other global and mesoscale GHI forecasting models to be compared against the current results in the region. The outcomes of this study indicate the possibility of using GFS GHI forecasts for forecasting solar photovoltaic output for energy planning. South Africa has an electricity supply deficit, and being able to accurately forecast the output of the increasing capacity of solar PV is one of the things needed to stabilize the grid.

For many critical energy-related applications, such as reliable PV power production, accurate Global Horizontal Irradiance (GHI) short-term forecasts are crucial. Forecasts of GHI, as analyzed in this study, are produced by the state-of-the-art, high-resolution forecasts developed within the Destination Earth initiative, the ECMWF Digital Twin Engine using the ExtremeDT (Weather-Induced Extremes Digital Twin) dataset, that produces daily global simulations at resolutions of 4.4 km kilometres up to four days ahead. However the GHI ExtremesDT forecasts do not take into account the aerosol effects, which may introduce systematic biases, especially during periods of high aerosol load events. We investigated the impact of aerosols on DT GHI forecasts and the associated PV power predictions within the context of the DestinE Destination Renewable Energy (DRE) use case.We analyze almost one year of data comprising two-day-ahead DT GHI forecasts and the corresponding aerosol optical depth (AOD) forecasts from the Copernicus Atmosphere Monitoring Service (CAMS). The DT GHI forecasts are corrected for aerosol effects using fast radiative transfer model techniques utilising the lidRadtran [1, 2] package.Within the DRE project, and based on co-design activities with the project's end user, site-specific PV power production forecasts tailored to the user’s needs and infrastructure were developed, using DT GHI forecasts and a machine learning (ML) model that was trained on the user’s historical data. The impact of aerosol correction was also evaluated by comparing PV power production forecasts derived from GHI forecasts with and without aerosol correction against the actual PV power production of the plant.A machine learning (ML) model, trained on historical, site-specific production data from the QUEST PV park, was created within the DRE project to convert predicted solar irradiance into PV park power output in order to evaluate the impact on power production. The ML model propagates the original and aerosol-corrected GHI forecasts, which are then compared to actual production.The results show how CAMS-based aerosol correction of GHI forecasts can reduce bias and consistently improve PV power prediction. Results show the value of incorporating atmospheric composition data with ML-based power conversion models for operational energy applications, as well as the significance of aerosol representation in solar forecasting.AcknowledgmentsDRE project has received funding from the European Space Agency under the DESTINATION EARTH USE CASES – DESP USE CASES - ROUND 1. The duration of the project is 12 months (November 2023 - November 2024).We would like also to acknowledge the COST Action HARMONIA (International network for harmonization of atmospheric aerosol retrievals from ground based photometers), CA21119.Bibliography(1) Mayer, B.; Kylling, A. Atmospheric Chemistry and Physics 2005, 5, 1855–1877.(2) Emde, C.; Buras-Schnell, R.; Kylling, A.; Mayer, B.; Gasteiger, J.; Hamann, U.; Kylling, J.; Richter, B.; Pause, C.; Dowling, T.; Bugliaro, L. Geoscientific Model Development 2016, 9, 1647–1672.

Long-term average solar radiation prediction via seasonal Global Horizontal Irradiance (GHI) forecasting is increasingly leveraging Machine Learning (ML) to uncover complex relationships in historical GHI and climatic parameters surpassing traditional methods. This study investigates seasonal GHI forecasting across four climatically distinct Indian locations namely Chennai, Jaisalmer, Leh and Mawsynram using data from the National Solar Radiation Database. Following rigorous data quality assessment and feature selection based on Spearman's correlation and Mutual Information, eight key features are identified for training three ML models: Efficient Linear Regression (ELR), Regression Trees (RT) and Gaussian Process Regression (GPR). The GPR model demonstrates superior accuracy in seasonal GHI prediction across diverse Indian climatic conditions making it ideal for reliable and cost-efficient Stand-Alone Photovoltaic (SAPV) systems. It is observed that the GPR model achieves the lowest Root Mean Square Error (RMSE) of 0.0030, Mean Absolute Error (MAE) of 0.0022 and Coefficient of Determination (R²) of 0.9999 achieving a reduction of 189.1% in RMSE, 190.09% in MAE and 20.56% improvement in R² compared to ELR. Furthermore, the GPR model surpasses the RT model with a reduction of 124.05% in RMSE and 111.1% in MAE, together with an improvement of 0.2604% in R². While RT provides reasonably accurate forecasts, ELR shows the least reliability. These results confirm GPR's exceptional precision and ability to capture nearly all dataset variability.

Accurate short-term GHI forecasting using a novel temporal convolutional network model

We estimate parameters from the Katz and Thompson permeability model using laboratory complex electrical conductivity (CC) and nuclear magnetic resonance (NMR) data to build permeability models parameterized with geophysical measurements. We use the Katz and Thompson model based on the characteristic hydraulic length scale, determined from mercury injection capillary pressure estimates of pore throat size, and the intrinsic formation factor, determined from multisalinity conductivity measurements, for this purpose. Two new permeability models are tested, one based on CC data and another that incorporates CC and NMR data. From measurements made on forty‐five sandstone cores collected from fifteen different formations, we evaluate how well the CC relaxation time and the NMR transverse relaxation times compare to the characteristic hydraulic length scale and how well the formation factor estimated from CC parameters compares to the intrinsic formation factor. We find: (1) the NMR transverse relaxation time models the characteristic hydraulic length scale more accurately than the CC relaxation time ( of 0.69 and 0.33 and normalized root mean square errors (NRMSE) of 0.16 and 0.21, respectively); (2) the CC estimated formation factor is well correlated with the intrinsic formation factor (NRMSE=0.23). We demonstrate that that permeability estimates from the joint‐NMR‐CC model (NRMSE=0.13) compare favorably to estimates from the Katz and Thompson model (NRMSE=0.074). This model advances the capability of the Katz and Thompson model by employing parameters measureable in the field giving it the potential to more accurately estimate permeability using geophysical measurements than are currently possible.

A robust model for diagnosing water stress of winter wheat by combining UAV multispectral and thermal remote sensing

An effective method for the rapid detection of microplastics in soil

Solar photovoltaics (PV) has advanced at an unprecedented rate and the global cumulative installed PV capacity is growing exponentially. However, the ability to forecast PV power remains a key technical challenge due to the variability and uncertainty of solar irradiance resulting from the changes of clouds. Ground-based remote sensing with high temporal and spatial resolution may have potential for solar irradiation forecasting, especially under cloudy conditions. To this end, we established two ultra-short-term forecasting models of global horizonal irradiance (GHI) using Ternary Linear Regression (TLR) and Back Propagation Neural Network (BPN), respectively, based on the observation of a ground-based sky imager (TSI-880, Total Sky Imager) and a radiometer at a PV plant in Dunhuang, China. Sky images taken every 1 min (minute) were processed to determine the distribution of clouds with different optical depths (thick, thin) for generating a two-dimensional cloud map. To obtain the forecasted cloud map, the Particle Image Velocity (PIV) method was applied to the two consecutive images and the cloud map was advected to the future. Further, different types of cloud fraction combined with clear sky index derived from the GHI of clear sky conditions were used as the inputs of the two forecasting models. Limited validation on 4 partly cloudy days showed that the average relative root mean square error (rRMSE) of the 4 days ranged from 5% to 36% based on the TLR model and ranged from 12% to 32% based on the BPN model. The forecasting performance of the BPN model was better than the TLR model and the forecasting errors increased with the increase in lead time.

Stochastic weather generators and crop growth models are used to explore the impact of climate change on crop development and yield. Synthetic weather data from the stochastic weather generator AAFC‐WG have been evaluated with annual crop models but not with a perennial grass model. Our objective was to evaluate synthetic weather data from AAFC‐WG using a perennial crop model, the Canadian Timothy Model (CATIMO), at five sites representing different agro‐ecological regions of Canada. Synthetic 300‐yr weather data from AAFC‐WG and observed 30‐yr weather data, both for the period 1961 to 1990, were used to simulate dates of growth onset and harvests, and yield and nutritive value (neutral detergent fiber [NDF] concentration and in vitro digestibility of NDF [dNDF]) of timothy (Phleum pratense L.) grown in cycles of five consecutive years before reseeding and with two harvests per year. Dates of growth onset and harvests simulated from synthetic weather data were close to those simulated from observed weather data, with normalized root mean square errors (NRMSEs) of 4.2% for dates of growth onset and <0.6% for harvest dates. Simulated dry matter yield (NRMSE < 4.5%), NDF concentration (NRMSE < 1.1%), and dNDF (NRMSE < 0.4%) with synthetic and observed weather data were also very close. Synthetic weather data generated by AAFC‐WG accurately reproduced weather conditions of observed data for timothy development, yield, and nutritive value, confirming that they can be used with the CATIMO model to predict the impact of climate change at several sites in Canada.

Due to rapidly changing sky conditions, the available solar irradiance for energy generation is subject to wide swings in amplitude. As the market penetration of solar energy continues to increase rapidly, these variations in solar power generation are beginning to have an impact on grid stability and increased wear on power switches. Solar power forecasting plays a critical role in operations for Independent System Operators (ISOs) and utilities. Accurate forecasts help maintain grid reliability, optimize generation from renewables, and reduce operating costs. Of particular interest to the ISOs and utilities are sudden changes in solar irradiance, termed “ramp events,” due to the movement of clouds. One significant impact of ramp events on the grid is additional ancillary service requirements necessary to manage such variability. Ramp events can also cause voltage fluctuations in the distribution grids and trigger actions of automated line equipment (e.g., tap changers), leading to additional maintenance costs. In high penetration solar regions, forecasts must be made for both transmission-connected and distribution-connected resources – either behind the meter or in front of it. Particularly for distributed solar energy resources, forecasting can be a challenge due to the lack of visibility of the energy resource. Brookhaven National Laboratory (BNL) has been working towards nowcasting Global Horizontal Irradiance (GHI) using low-cost technologies for several years now. The Solar NowCasting technology being developed by BNL is a 0 – 30 min solar “nowcasting” technology applicable to a scale covering both large generating facilities and residential, distributed solar installations that relies on a network of ground-based high-definition (HD) cameras and surface pyranometers. GHI forecasts are being produced for both regions using 8 ground-based HD cameras and at least 2 surface pyranometers. When stitched together, the camera images can be used to forecast the impact of clouds on available solar irradiance over a domain of ~50 km². Phase I of the project was the engineering scale up conceptual design stage. Phase II was the initial field test and demonstration, in which the technology was scaled up by a factor of 20 times and successfully demonstrated in eastern Long Island. In Phase III, an additional forecasting network was added in upstate NY and both networks are currently being operated until at least a full year of data is gathered by the Albany network to allow for the collection of sufficient data to evaluate performance against the persistence and smart persistence models using the Solar Arbiter.

Measurements of spectral-band solar irradiance in Bangi, Malaysia

Functional electrical stimulation (FES) is widely used in neurorehabilitation to improve patients’ motion ability. It has been verified to promote neural remodeling and relearning, during which FES has to produce an accurate movement to obtain a good efficacy. Therefore, many studies have focused on the relationship between FES parameters and the generated movements. However, most of the relationships have been established in static contractions, which leads to an unsatisfactory result when applied to dynamic conditions. Therefore, this study proposed a FES control strategy based on the surface electromyography (sEMG) and kinematic information during dynamic contractions. The pulse width (PW) of FES was determined by a direct transfer function (DTF) with sEMG features and joint angles as the input. The DTF was established by combing the polynomial transfer functions of sEMG and joint torque and the polynomial transfer functions of joint torque and FES. Moreover, the PW of two FES channels was set based on the muscle synergy ratio obtained through sEMG. A total of six healthy right-handed subjects were recruited in this experiment to verify the validity of the strategy. The PW of FES applied to the left arm was evaluated based on the sEMG of the right extensor carpi radialis (ECR) and the right wrist angle. The coefficient of determination (R2) and the normalized root mean square error (NRMSE) of FES-included and voluntary wrist angles and torques were used to verify the performance of the strategy. The result showed that this study achieved a high accuracy (R2 = 0.965 and NRMSE = 0.047) of joint angle and a good accuracy (R2 = 0.701 and NRMSE = 0.241) of joint torque reproduction during dynamic movements. Moreover, the DTF in real-time FES system also had a nice performance of joint angle fitting (R2 = 0.940 and NRMSE = 0.071) and joint torque fitting (R2 = 0.607 and NRMSE = 0.303). It is concluded that the proposed strategy is able to generate proper FES parameters based on sEMG and kinematic information for dynamic movement reproduction and can be used in a real-time FES system combined with bilateral movements for better rehabilitation.

Remote sensing datasets offer a unique opportunity to observe spatial and temporal trends in water quality indicators (WQIs), such as chlorophyll-a, salinity, and turbidity, across various aquatic ecosystems. In this study, we used available in situ WQI measurements (chlorophyll-a: 17, salinity: 478, and turbidity: 173) along with Landsat-8 surface reflectance data to examine the capability of empirical and machine learning (ML) models in retrieving these indicators over Matagorda Bay, Texas, between 2014 and 2023. We employed 36 empirical models to retrieve chlorophyll-a (12 models), salinity (2 models), and turbidity (22 models) and 4 ML families—deep neural network (DNN), distributed random forest, gradient boosting machine, and generalized linear model—to retrieve salinity and turbidity. We used the Nash–Sutcliffe efficiency coefficient (NSE), correlation coefficient (r), and normalized root mean square error (NRMSE) to assess the performance of empirical and ML models. The results indicate that (1) the empirical models displayed minimal effectiveness when applied over Matagorda Bay without calibration; (2) once calibrated over Matagorda Bay, the performance of the empirical models experienced significant improvements (chlorophyll-a—NRMSE: 0.91 ± 0.03, r: 0.94 ± 0.04, NSE: 0.89 ± 0.06; salinity—NRMSE: 0.24 ± 0, r: 0.24 ± 0, NSE: 0.06 ± 0; turbidity—NRMSE: 0.15 ± 0.10, r: 0.13 ± 0.09, NSE: 0.03 ± 0.03); (3) ML models outperformed calibrated empirical models when used to retrieve turbidity and salinity, and (4) the DNN family outperformed all other ML families when used to retrieve salinity (NRMSE: 0.87 ± 0.09, r: 0.49 ± 0.09, NSE: 0.23 ± 0.12) and turbidity (NRMSE: 0.63± 0.11, r: 0.79 ± 0.11, NSE: 0.60 ± 0.20). The developed approach provides a reference context, a structured framework, and valuable insights for using empirical and ML models and Landsat-8 data to retrieve WQIs over aquatic ecosystems. The modeled WQI data could be used to expand the footprint of in situ observations and improve current efforts to conserve, enhance, and restore important habitats in aquatic ecosystems.

Multivariable Recursive Subspace Identification with Application to Artificial Pancreas Systems

Evaluation and improvement of TAPM in estimating solar irradiance in Eastern Australia