Localised Magnetic Substorm Forecasting using Machine Learning

Shadows from moving clouds in the troposphere impact the energy generated by photovoltaic systems. intra-hour solar forecast can be used to regulate solar energy dispatch. This investigation develops a data processing method for cloud dynamic feature extraction from raw sky images and Global Solar Irradiance (GSI) measurements that can be integrated into solar forecasting algorithms to reduce the operational supervision of hardware. Sky images and GSI measurements are acquired from a low-cost long-wave infrared radiometric camera and a pyranometer. This sky imager is mounted on a solar tracker that maintains the Sun in the center of sky images throughout the day. Multiple processing methods are proposed here that take advantage of a hybrid approach to approximate the optimal parameters of physical models using computationally inexpensive machine learning models. A signal processing method removes cyclostationary biases in high-resolution clear sky index values found when detrending GSI measurements using the clear sky GSI. Image processing methods are then used to remove the effects of atmospheric radiation and the Sun’s direct radiation from infrared sky images, plus the radiation effect emitted by debris on the sky imager’s germanium outdoor lens. The result is an adaptive solar forecasting algorithm that can reduce the operational cost of power grids with the high participation of solar energy in the generation mix.

In our solar system, the main source of plasma is the Sun, which produces the so-called solar wind by continuously pushing its outermost layer -the corona- into space. The turbulent solar wind impinges on our planet and interacts with its magnetic field, creating a region of space called Earth’s magnetosphere. From its birth to its impact on our planet, the solar wind still harbors numerous unanswered questions. Answering these questions requires the numerical modelling of the plasma itself.The most physically accurate numerical methods are based on kinetic modeling, which tracks the particles' velocity distribution function. However, these methods are numerically demanding since they involve modeling the complex six-dimensional particle distribution function as it evolves in time. To simplify the problem, such distribution is integrated over the velocity coordinates leading to the (more efficient) three-dimensional fluid plasma framework. Still, the passage to the fluid equations comes with an important caveat. The fluid system of equations needs to be closed by choosing a proper “closure”. The objective of this work is to tackle the closure problem by employing a combination of kinetic simulation and machine learning techniques.We perform multiple decaying turbulence plasma simulations using a Hybrid-PIC [1] (i.e. kinetic ions, fluid electrons) model. By varying different physical parameters, notably the ion beta, we explore the variability of the solar wind. These kinetic simulations serve as the ground truth to train a machine learning model. The machine's task is to "learn" the best approximation for the closure equation. We focus in particular on the reconstruction of the pressure tensor. We explore various machine learning techniques [2, 3] (CNN, GAN, FNO) that have shown promise in atmospheric science but are new to this specific problem. We show how this reconstructed closure performs better than other analytical approximations [4] (polytropic, CGL, CGL+FLR effects). The final goal is to learn a closure equation that can effectively incorporate complex kinetic physics into a simplified, yet more accurate, fluid simulation. This will significantly increase the fidelity of solar wind models without making them prohibitively expensive to compute.[1] Behar, Etienne, Shahab Fatemi, Pierre Henri, e Mats Holmström. «Menura: A Code for Simulating the Interaction between a Turbulent Solar Wind and Solar System Bodies». Annales Geophysicae 40, fasc. 3 (2022): 281–97. https://doi.org/10.5194/angeo-40-281-2022.[2] Kovachki, Nikola, Zongyi Li, Burigede Liu, et al. «Neural Operator: Learning Maps Between Function Spaces». Preprint, 2 maggio 2024. https://doi.org/10.5555/3648699.3648788.[3] Jeong, Hyun-Jin, Mingyu Jeon, Daeil Kim, et al. «Prediction of the Next Solar Rotation Synoptic Maps Using an Artificial Intelligence–Based Surface Flux Transport Model». The Astrophysical Journal Supplement Series 278, fasc. 1 (2025): 5. https://doi.org/10.3847/1538-4365/adc447.[4] Hunana, P., A. Tenerani, G. P. Zank, et al. «An Introductory Guide to Fluid Models with Anisotropic Temperatures. Part 1. CGL Description and Collisionless Fluid Hierarchy». Journal of Plasma Physics 85, fasc. 6 (2019): 205850602. https://doi.org/10.1017/S0022377819000801.

Ahead-of-time forecasting of the output power of power plants is essential for the stability of the electricity grid and ensuring uninterrupted service. However, forecasting renewable energy sources is difficult due to the chaotic behavior of natural energy sources. This paper presents a new approach to estimate short-term solar irradiance from sky images. The proposed algorithm extracts features from sky images and use learning-based techniques to estimate the solar irradiance. The performance of proposed machine learning (ML) algorithm is evaluated using two publicly available datasets of sky images. The datasets contain over 350,000 images for an interval of 16 years, from 2004 to 2020, with the corresponding global horizontal irradiance (GHI) of each image as the ground truth. Compared to the state-of-the-art computationally heavy algorithms proposed in the literature, our approach achieves competitive results with much less computational complexity for both nowcasting and forecasting up to 4 h ahead of time.

The lunar surface is constantly bombarded by the solar wind, photons, and meteoroids, which can liberate Na atoms from the regolith. These atoms are subsequently accelerated by solar photon pressure to form a long comet‐like tail opposite the sun. Near new moon, these atoms encounter the Earth's gravity and are “focused” into a beam of enhanced density. This beam appears as the ∼3° diameter Sodium Moon Spot (SMS). Data from the all sky imager at the El Leoncito Observatory have been analyzed for changes in the SMS shape and brightness. New geometry‐based relationships have been found that affect the SMS brightness. The SMS is brighter when the Moon is north of the ecliptic at new moon; the SMS is brighter when new moon occurs near perigee; and the SMS peaks in brightness ∼5 h after new moon. After removing these effects, the data were analyzed for long term and seasonal patterns that could be attributed to changes in source mechanisms. No correlation was found between the SMS brightness and the 11‐year solar‐cycle, the proton or the He++ flow pressure, the density, the speed or the plasma temperature of the solar wind, but an annual pattern was found. A ∼0.83 correlation (Pearson's “r”) was found between the SMS brightness and a 4‐year average of sporadic meteor rates at Earth, suggesting a cause‐and‐effect. The new insights gained from this long‐term study put new constraints on the variability of the potential sources of the Na atoms escaping from the Moon.

In recent decades, voltage instability has become an important contributor to large-scale power outages globally. Periodic voltage security assessments are key for the secure operation of power systems. There are various indicators which have been implemented in different types and complexity of systems to provide a measure of closeness to voltage collapse, but some of these algorithms may require advanced computational resources or may not perform well, depending on the prevailing conditions. Expanding interconnected power systems is needed like never before due to the increasing world demand for electricity, and voltage stability is therefore a worth consideration. The growing demand penetration of solar photovoltaic (PV) systems has some potential stability challenges due to their intermittent characteristics. Existing research evaluates the test systems of voltage stability and does not optimize using Intelligent models, which have variable power generation from Solar PV and Wind, as well as uncertainties in load demand and the initial step considering the evaluation of the model have to cover either inputs. A feed–forward back propagation artificial neural network (ANN) for voltage stability assessment to classify the operating scenario of a power system into one of two states, i.e. safe or unsafe. Neural network input data is generated from load flow analysis using Newton-Raphson (NR), and stability indices are computed over different scenarios. Line Stability Index (Lmn), counts the stability of a transmission line in power systems and provides an indication of its potential voltage collapse. The Fast Voltage Stability Index (FVSI), which also estimates line stability through reactive power and impedance characteristics. Voltage Collapse Proximity Index (VCPI) assesses system proximity to voltage collapse and identifies weak transmission lines. Values approaching 1 suggest voltage collapse conditions. The FVSI, VCPI, and Lmn are compared to a baseline NR stability index. Moreover, self-organizing map (SOM) neural network is used for stability results classification. Multi-objective grasshopper optimization algorithm (MOGOA) has been employed as an optimization network on the dual axis solar PV and wind turbine-based hybrid power generation system to analyze technical, economic, as well as environmental relations of energy conversion. The findings reveal that the adapted grid-tied solar-wind composite power system substantially mitigates the power deficiencies, reduces the real-power outages, diminishes the motion voltage small amendments, maximizes voltage prosperity, and improves power system resilience. The main findings show a significant reduction in real power loss, which went from 289,391.30 kW to only 5.43 kW, and a remarkable decrease in voltage deviation, which reduced from 0.56 p.u to 0.0493 p.u. Ensures Improved Service Quality and Enhanced Equipment Protection While gas generation costs increased by a small amount, it is important to note that this is an acceptable trade-off for a more robust and sustainable system, with costs per kWh of $0.142 vs $0.2881 per kWh, respectively. In addition, stability index was substantially decreased from 0.7666 to 0.0768, indicating better grid and resilience to disturbances. Findings emphasize the contribution of embedding MOGOA in conjunction with machine learning models including ANN and Self Organizing Maps (SOM) to fulfill the primal requisites of stability, efficiency and sustainability in modernized power systems, illustrating a coupled advancement in the realm of power systems engineering and optimization.

We introduce a new framework called Machine Learning (ML) based Auroral Ionospheric electrodynamics Model (ML‐AIM). ML‐AIM solves a current continuity equation by utilizing the ML model of Field Aligned Currents of Kunduri et al. (2020, https://doi.org/10.1029/2020JA027908), the FAC‐derived auroral conductance model of Robinson et al. (2020, https://doi.org/10.1029/2020JA028008), and the solar irradiance conductance model of Moen and Brekke (1993, https://doi.org/10.1029/92gl02109). The ML‐AIM inputs are 60‐min time histories of solar wind plasma, interplanetary magnetic fields (IMF), and geomagnetic indices, and its outputs are ionospheric electric potential, electric fields, Pedersen/Hall currents, and Joule Heating. We conduct two ML‐AIM simulations for a weak geomagnetic activity interval on 14 May 2013 and a geomagnetic storm on 7–8 September 2017. ML‐AIM produces physically accurate ionospheric potential patterns such as the two‐cell convection pattern and the enhancement of electric potentials during active times. The cross polar cap potentials (ΦPC) from ML‐AIM, the Weimer (2005, https://doi.org/10.1029/2004ja010884) model, and the Super Dual Auroral Radar Network (SuperDARN) data‐assimilated potentials, are compared to the ones from 3204 polar crossings of the Defense Meteorological Satellite Program F17 satellite, showing better performance of ML‐AIM than others. ML‐AIM is unique and innovative because it predicts ionospheric responses to the time‐varying solar wind and geomagnetic conditions, while the other traditional empirical models like Weimer (2005, https://doi.org/10.1029/2004ja010884) designed to provide a quasi‐static ionospheric condition under quasi‐steady solar wind/IMF conditions. Plans are underway to improve ML‐AIM performance by including a fully ML network of models of aurora precipitation and ionospheric conductance, targeting its characterization of geomagnetically active times.

The evolving ambient solar wind is one of the key links between the Sun and planetary bodies in our solar system. Here we present a comprehensive catalogue of solar wind properties, stream interaction regions, and coronal mass ejections at different locations in the inner heliosphere. Our database incorporates observational data products and also solar wind modelling results. The solar wind modelling is based on two different approaches for modelling the conditions in the ambient solar wind. While the WSA/THUX model combination solves the viscous form of the underlying Burgers equation to compute the two-dimensional solar wind conditions in our solar system, the second approach is a computationally fast machine learning method for predicting the ambient solar wind flows at Earth. Statistics of the ambient solar wind model results for more than 15 years in combination with a catalogue of coronal mass ejections observed at the Earth, Mars and STEREO satellites along with stream interaction regions provide a comprehensive overview of the past and present solar wind behaviour for shaping planetary space weather.

During the past years, the convergence of machine learning (ML) technologies with renewable energy sectors has become a significant key area of innovation as a key area of innovation, enhancing the efficiency and predictability of sustainable energy sources. ML algorithms, adept at handling complex data, have become essential in forecasting energy outputs from variable sources like solar and wind. This integration has led to the development of smarter, more adaptive grid systems, capable of efficiently managing the variability of renewable energy sources. This review paper focuses on several key areas: firstly, it provides a summary of related work, specifically focusing on ML in the renewable energy field. Secondly, it delves into ML models and evaluation metrics used for solar and wind energy forecasting. Thirdly, it analyzes 21 studies published from 2019 to 2023, primarily centered on solar energy (60%) and wind energy (40%), with an emphasis on various forecasting horizons, highlighting the results of the ML algorithms used and the performance metrics to evaluate their effectiveness. Finally, it identifies gaps and opportunities in this field. The state-of-the-art review and its findings can offer a solid foundation for future research initiatives.

Substorms are a magnetospheric phenomenon which causes high geomagnetic and auroral disturbances. It is widely accepted that substorm activity is controlled by solar wind conditions. It is, however, difficult to predict substorms deterministically because of the complex physical processes underlying substorm occurrences. We propose a framework for modelling time series of event occurrences controlled by external forcing. In this framework, occurrences of external-driven events are represented with a non-stationary Poisson process, and its intensity, which corresponds to the occurrence rates per unit time, is described with a simple machine learning model, the echo state network, which is fed with forcing variables. The echo state network is trained by maxmising the likelihood given the event time series data. We apply this approach for analysing time series of substorm onsets identified from Pi2 pulsations, which are irregular geomagnetic oscillations associated with substorm onsets. We train the echo state network to well describe the response of substorm activity to solar-wind conditions. We then examine the characteristics of the substorm activity by feeding synthetic solar-wind data into the echo state network. The results show what solar wind variables effectively contribute to the substorm occurrence. Our echo state network model is also useful for examining the statistical properties of the substorm occurrence rate. For example, we can evaluate what mainly controls the seasonal and UT variations of substorm activity. There are two explanations for the seasonal and UT variations. One explanation is that the seasonal and UT variations is controlled by the inner product between the solar-wind magnetic field and the Earth's dipole axis. The other is that the variations are due to the angle between the solar-wind flow and the Earth's dipole axis. Since these two effects are related with different input variables in our echo state network model, we can examine the contribution of each effect to the substorm occurrence frequency. The result shows that the seasonal and UT variations are mostly dependent on the angle between the solar-wind flow and the Earth's dipole axis. 

Auroral electrojet (AE) activity is a widely used system-level indicator of how the magnetosphere-ionosphere system responds to solar wind driving across a wide range of spatial and temporal scales. While large-scale solar wind parameters are known to control overall levels of geomagnetic activity, auroral responses often show substantial variability under similar upstream conditions. This suggests that additional aspects of solar wind variability, beyond mean magnetic field and plasma properties, may influence how energy and momentum are transferred into the coupled system. In particular, the role of multiscale solar wind turbulence and structured variability in modulating auroral activity remains incompletely understood.In this work, we examine how multiscale solar wind turbulence contributes to auroral variability using an interpretable machine learning approach. OMNI observations are combined with AE index measurements to construct models that integrate conventional bulk drivers with measures describing solar wind variability across multiple timescales. Interpretable diagnostics are then used to assess how turbulence-related information influences auroral responses under different upstream conditions. While the overall improvement in forecasting skill obtained by including turbulence measures is modest, the results reveal consistent and scale-dependent contributions associated with structured solar wind variability. These findings suggest that solar wind turbulence plays a secondary but informative role in shaping auroral activity, providing insight into how mesoscale variability can modulate system-level coupling and highlighting the value of interpretable machine learning for advancing both physical understanding and space weather prediction.

Magnetic reconnection is a fundamental physical process in planetary magnetospheres, in which plasma can be exchanged between the solar wind and a planetary magnetosphere, and material can be disconnected and ultimately lost from a magnetosphere. Magnetic reconnection in a planetary magnetotail can result in the release of plasmoids downtail and dipolarizations planetward of an x‐line. The signatures of these products include characteristic deflections in the north‐south component of the magnetic field which can be detected by in‐situ spacecraft. These signatures have been identified by eye, semi‐automated algorithms, and recently machine learning (ML) methods. Here, we apply statistical analysis to the most thorough catalog of Kronian magnetospheric reconnection signatures created through ML methods to improve understanding of magnetospheric evolution. This research concludes that no quasi‐steady position of the magnetotail x‐line exists within 70 . This research introduces prediction equations to estimate the distribution of duration of plasmoid passage over the spacecraft (, bin width = 1 min) and north‐south field deflection (, bin width = 0.25 nT) expected to be identified by an orbiting spacecraft across a year of observations. Furthermore, this research finds a local time asymmetry for reconnection identifications, with a preference for dusk‐side over dawn‐side. This may indicate a preference for Vasyliunas style reconnection over Dungey style for Saturn. Finally, through these distributions, the reconnection rate of Saturn’s magnetotail can be estimated as 3.22 reconnection events per day, with a resulting maximum mass loss from plasmoids of 44.87 kg on average, which is comparable with the magnetospheric mass loading from Enceladus (8–250 kg ).

Prediction of solar irradiation and wind speed are essential for enhancing the renewable energy integration into the existing power system grids. However, the deficiencies caused to the network operations provided by their intermittent effects need to be investigated. Regarding reserves management, regulation, scheduling, and dispatching, the intermittency in power output become a challenge for the system operator. This had given the interest of researchers for developing techniques to predict wind speeds and solar irradiation over a large or short-range of temporal and spatial perspectives to accurately deal with the variable power output. Before, several statistical, and even physics, approaches have been applied for prediction. Nowadays, machine learning is widely applied to do it and especially regression models to assess them. Tuning these models is usually done following manual approaches by changing the minimum leaf size of a decision tree, or the box constraint of a support vector machine, for example, that can affect its performance. Instead of performing it manually, this paper proposes to combine optimization methods including the bayesian optimization, grid search, and random search with regression models to extract the best hyper parameters of the model. Finally, the results are compared with the manually tuned models. The Bayesian gives the best results in terms of extracting hyper-parameters by giving more accurate models.KeywordsRenewable energyForecastingMachine learningOptimizationWind speedSolar irradiation

This paper presents a novel Machine Learning (ML) approach to ionospheric forecasting, including forecasting the space weather impact on the ionosphere. It exploits a data-driven approach in which the models learn underlying processes and relationships from data describing solar activity, solar wind, interplanetary and Earth’s magnetic fields, and the ionosphere. We applied a multi-model and multi-data ensemble forecasting approach using diverse models of different learning algorithms with different training datasets to generate 1-day VTEC forecasts. This approach improved forecasting accuracy compared to a single-model-based approach. In addition, the forecast uncertainty of the super-ensemble model was assessed by estimating an ensemble spread. The results show potential for forecasting VTEC in different ionospheric regions during quiet and storm periods while quantifying their uncertainties.

Dynamic interactions between the solar wind and the magnetosphere give rise to dramatic auroral forms that have been instrumental in the ground-based study of magnetospheric dynamics. The general mechanism of aurora types and their large-scale patterns are well-known, but the morphology of small- to meso-scale auroral forms observed in all-sky imagers and their relation to magnetospheric dynamics  and the coupling of the magnetosphere to the upper atmosphere remain in question. Machine learning has the potential to provide answers to these questions, but most existing auroral image data lack the ground-truth labels required for supervised learning and conventional statistical analyses. To mitigate this issue, we propose a novel self-supervised semi-supervised algorithm to automatically label the THEMIS all-sky image database. Specifically, we adapt the self-supervised Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn latent representations of THEMIS all-sky images. These representations are finetuned using a small set of manually labeled data from the Oslo Aurora THEMIS (OATH) dataset, after which semi-supervised classification is used to train a classifier, beginning by training on the manually labeled OATH dataset and gradually incorporating the classifier’s most confident predictions on unlabeled data into the training dataset as ground-truth. We demonstrate that (a) classifiers fit to the learned representations of the manually labeled images achieve state–of–the–art performance, improving the classification accuracy by almost 10% over the current benchmark on labeled data; and (b) our model’s learned representations naturally cluster into more clusters than manually assigned categories, suggesting that existing categorizations are coarse and may obscure important connections between auroral types and their drivers. Finally, we introduce AuroraClick, a citizen science project with the goal of manually annotating a large representative sample of THEMIS all-sky images for the validation of our current models and the training of future models.  

The Kp index is a measure of the midlatitude global geomagnetic activity and represents short‐term magnetic variations driven by solar wind plasma and interplanetary magnetic field. The Kp index is one of the most widely used indicators for space weather alerts and serves as input to various models, such as for the thermosphere and the radiation belts. It is therefore crucial to predict the Kp index accurately. Previous work in this area has mostly employed artificial neural networks to nowcast Kp, based their inferences on the recent history of Kp and on solar wind measurements at L1. In this study, we systematically test how different machine learning techniques perform on the task of nowcasting and forecasting Kp for prediction horizons of up to 12 hr. Additionally, we investigate different methods of machine learning and information theory for selecting the optimal inputs to a predictive model. We illustrate how these methods can be applied to select the most important inputs to a predictive model of Kp and to significantly reduce input dimensionality. We compare our best performing models based on a reduced set of optimal inputs with the existing models of Kp, using different test intervals, and show how this selection can affect model performance.