Ensemble Regression

This study presents findings from a real-time forecast experiment that compares legacy deterministic hydrologic stage forecasts to ensemble mean and median stage forecasts from the NOAA/NWS Meteorological Model-Based Ensemble Forecast System (MMEFS). The NOAA/NWS Ohio River Forecast Center (OHRFC) area of responsibility defines the experimental region. Real-time forecasts from subbasins at 54 forecast point locations, ranging in drainage area, geographic location within the Ohio River valley, and watershed response time serve as the basis for analyses. In the experiment, operational hydrologic forecasts, with a 24-h quantitative precipitation forecast (QPF) and forecast temperatures, are compared to MMEFS hydrologic ensemble mean and median forecasts, with model forcings from the NOAA/NWS National Centers for Environmental Prediction (NCEP) North American Ensemble Forecast System (NAEFS), over the period from 30 November 2010 through 24 May 2012. Experiments indicate that MMEFS ensemble mean and median forecasts exhibit lower errors beginning at about lead time 90 h when forecasts at all locations are aggregated. With fast response basins that peak at ≤24 h, ensemble mean and median forecasts exhibit lower errors much earlier, beginning at about lead time 36 h, which suggests the viability of using MMEFS ensemble forecasts as an alternative to OHRFC legacy forecasts. Analyses show that ensemble median forecasts generally exhibit smaller errors than ensemble mean forecasts for all stage ranges. Verification results suggest that OHRFC MMEFS NAEFS ensemble forecasts are reasonable, but needed improvements are identified.

The July–August–September (JAS) seasonal predictability of the Atlantic Warm Pool (AWP) is examined in the extensive set of seasonal hindcasts of the National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS). We conduct both deterministic and probabilistic skill analyses of the NCEP CFS Seasonal Hindcasts (SHs). This study reveals that the SHs have a reasonable climatology of the AWP. The presence of robust decadal variability in sea surface temperature (SST) observations, while absent in the SHs, does seem to indicate its influence on the seasonal errors of the AWP in the NCEP CFS. However, after filtering out the observed SST for variability greater than 8 years, there is no systematic relation of AWP seasonal errors in the NCEP CFS with lead time. The signal to noise ratio of the area of the AWP in the SHs decreases with lead time, largely due to increases in the ensemble spread (noise). It is sobering to note that there is barely any probabilistic skill in the NCEP CFS SHs for the prediction of the anomalous AWP events (occurring either in the upper or lower terciles) at nearly all lead times. However if we examine the SHs from 1981–1996, when the observed SST in the AWP region is in one uniform phase of the prevalent decadal variation, then the SHs display useful skill in the prediction of anomalously large AWP events at all lead times. Furthermore, these truncated SHs also show significant probabilistic skill in the prediction of small (lower tercile) and normal (middle tercile) AWP events up to a lead time of 1 month.

Improving ensemble forecasts is a complex process which involves proper scores such as the continuous ranked probability score (CRPS). A homogeneous Gaussian (hoG) model is introduced in order to better understand the characteristics of the CRPS. An analytical formula is derived for the expected CRPS of an ensemble in the hoG model. The score is a function of the variance of the error of the ensemble mean, the mean error of the ensemble mean and the ensemble variance. The hoG model also provides a score decomposition into reliability and resolution components. We examine whether the hoG model provides a useful approximation of the CRPS when applied to operational ECMWF medium‐range ensemble forecasts. The hoG approximation describes the spatial variations of the CRPS well while moderately overestimating the mean score. Seasonal averages over large domains are within 10% of the actual CRPS. Furthermore, the ability to approximate score changes is evaluated by (a) comparing raw ensemble forecasts with postprocessed ensemble forecasts, and (b) by examining score changes associated with a recent upgrade of the IFS. Overall, the hoG approximation predicts the actual CRPS changes well. One of the main anticipated applications of the hoG approximation are new diagnostics in verification software used by NWP developers routinely. The purpose of the diagnostics is to help developers explain impacts of forecast system changes on the CRPS in terms of the changes in mean error, changes in error variance and changes in ensemble variance. The diagnostics require little additional computational resources compared to the alternative of verifying postprocessed versions of the ensemble forecasts. Therefore, it will be feasible to apply the diagnostics easily to all variables that are examined as part of the model development process.

The National Center for Environmental Predictions (NCEP) has produced an ensemble meteorological reforecast product by using a fixed version of Global Forecast System (GFS) ensemble prediction system since 1 January 1979. The 15‐member ensemble product, with a global coverage at a 2.5° × 2.5° spatial resolution and a 14‐day lead time, has been used successfully by the River Forecast Centers of the National Weather Service (NWS) to produce basin scale precipitation and temperature ensemble forecasts in the US for several years now. This study evaluates the predictive skill of post‐processed ensemble forecasts based on GFS precipitation reforecast in China's Huai river basin. The evaluation is carried out in 15 sub‐areas of the Huai river basin and covers the 1/1/1981–31/12/2003 period. The Ensemble Pre‐Processing system version 3 (EPP3), developed at NWS, is used to develop joint probability distributions between forecasted ensemble mean precipitation and corresponding observations and to generate individual ensemble members that preserve space–time correlation of the observed precipitation data. Several statistical verification measures are used to quantify the goodness of fit between post‐processed (i.e. EPP3 processed) ensemble mean and observation and to assess the ensemble spread. Results indicate that the post‐processed forecasts have meaningful predictive skill for the first few days for ensemble daily precipitation forecasts. Predictive skill of ensemble forecasts of cumulative precipitation for lead times up to 14 days are significant. The forecast skill is highly dependent on seasonality, with relatively lower skills seen for wet summer season, when convective storm patterns dominate, as compared with other seasons. The predictive skill of the post‐processed ensemble precipitation is much better than the raw forecasts and the climatological ensemble forecasts. The results from this study suggest that the NCEP's GFS reforecasts can be a valuable resource for places other than the US. Copyright © 2012 John Wiley & Sons, Ltd.

Forecast performance of data-driven models depends on the local weather and climate regime, which makes model selection a tedious task for forecast practitioners. Ensemble forecasting, or forecast combination, is beneficial in such cases, in that, forecasts from multiple models are combined to form a final forecast. In ensemble forecasting, additional to the final deterministic-style forecasts, predictive distributions are also available, which can be used by grid operators for better decision-making. Such empirical predictive distributions are useful to represent the uncertainty associated with the forecasts. However, raw ensemble forecasts are often not calibrated, e.g., due to the lack of diversity in the ensemble members. The lack of ensemble spread is known as underdispersion, and it can be ameliorated through post-processing.This study aims to calibrate hourly ensemble clear-sky index forecasts, generated by 20 data-driven models, using both parametric and nonparametric post-processing techniques. Four years of data collected at 7 research-grade sites are used in the empirical part of the paper. Quantitative and qualitative methods are used to evaluate the performance of post-processing techniques in terms of calibration and sharpness. Post-processed ensemble forecasts outperform raw ensemble forecasts under all verification metrics. The proposed parametric post-processing technique, namely, generalized additive models for location, scale and shape, substantially reduces the continuous ranked probability score (CRPS) of the raw ensemble forecasts from 32–59 W/m2 to 25–45 W/m2 and quantile score from 16–30 W/m2 to 13–23 W/m2. In terms of CRPS skill score, the proposed method achieved 38–58% improvements over a climatology reference.

The National Centers for Environmental Prediction (NCEP) Climate Forecast System (CFS) is initialized four times per day (0000, 0600, 1200, and 1800 UTC). NCEP upgraded their operational CFS to version 2 on March 30, 2011. This is the same model that was used to create the NCEP Climate Forecast System Reanalysis (CFSR), and the purpose of this dataset is to extend CFSR. The 6-hourly atmospheric, oceanic and land surface analyzed products and forecasts, available at 0.2, 0.5, 1.0, and 2.5 degree horizontal resolutions, are archived here beginning with January 1, 2011 as an extension of CFSR. The RDA is not archiving any of the CFS seasonal forecasts. For more information about CFS, please see http://cfs.ncep.noaa.gov/ [http://cfs.ncep.noaa.gov/].

Bias correction of ensemble precipitation forecasts in the improvement of summer streamflow prediction skill

Comparison of different incremental analysis update schemes in a realistic assimilation system with Ensemble Kalman Filter

Abstract. Stream water temperatures influence water quality, with effects on aquatic biodiversity, drinking-water provision, electricity production, agriculture, and recreation. Therefore, stakeholders would benefit from an operational forecasting service that would support timely action. Deep-learning models are well-suited to providing probabilistic forecasts at individual stations of a monitoring network. Here, we train and evaluate several state-of-the-art models using 10 years of data from 54 stations across Switzerland. Static catchment features, time of the year, meteorological observations from the past 64 d, and their ensemble forecasts for the following 32 d are included as predictors in the models to estimate daily maximum water temperature over the next 32 d. Results show that the temporal fusion transformer (TFT) model performs best, with a continuous rank probability score (CRPS) of 0.70 °C averaged over all lead times, stations, and 90 forecasts distributed over 1 year. The TFT is followed by the recurrent neural network encoder–decoder, with a CRPS of 0.74 °C, and the neural hierarchical interpolation for time series, with a CRPS of 0.75 °C. These deep-learning models outperform other simpler models trained at each station: random forest (CRPS = 0.80 °C), multi-layer perceptron neural network (CRPS = 0.81 °C), and autoregressive linear model (CRPS = 0.96 °C). The average CRPS of the TFT degrades from 0.38 °C at lead a time of 1 d to 0.90 °C at a lead time of 32 d, largely driven by the uncertainty of the meteorological ensemble forecasts. In addition, TFT water temperature predictions at new and ungauged stations outperform those from the other models. When analyzing the importance of model inputs, we find a dominant role of observed water temperature and future air temperature, while including precipitation and time of the year further improves predictive skill. Operational probabilistic forecasts of daily maximum water temperature are generated twice per week with our TFT model and are publicly available at https://www.drought.ch/de/impakt-vorhersagen-malefix/wassertemperatur-prognosen/ (last access: 20 March 2025). Overall, this study provides insights into the extended-range predictability of stream water temperature and into the applicability of deep-learning models in hydrology.

Abstract. This paper investigates the uncertainties in hourly streamflow ensemble forecasts for an extreme hydrological event using a hydrological model forced with short-range ensemble weather prediction models. A state-of-the art, automated, short-term hydrologic prediction framework was implemented using GIS and a regional scale hydrological model (HEC-HMS). The hydrologic framework was applied to the Hudson River basin ( ∼ 36 000 km2) in the United States using gridded precipitation data from the National Centers for Environmental Prediction (NCEP) North American Regional Reanalysis (NARR) and was validated against streamflow observations from the United States Geologic Survey (USGS). Finally, 21 precipitation ensemble members of the latest Global Ensemble Forecast System (GEFS/R) were forced into HEC-HMS to generate a retrospective streamflow ensemble forecast for an extreme hydrological event, Hurricane Irene. The work shows that ensemble stream discharge forecasts provide improved predictions and useful information about associated uncertainties, thus improving the assessment of risks when compared with deterministic forecasts. The uncertainties in weather inputs may result in false warnings and missed river flooding events, reducing the potential to effectively mitigate flood damage. The findings demonstrate how errors in the ensemble median streamflow forecast and time of peak, as well as the ensemble spread (uncertainty) are reduced 48 h pre-event by utilizing the ensemble framework. The methodology and implications of this work benefit efforts of short-term streamflow forecasts at regional scales, notably regarding the peak timing of an extreme hydrologic event when combined with a flood threshold exceedance diagram. Although the modeling framework was implemented on the Hudson River basin, it is flexible and applicable in other parts of the world where atmospheric reanalysis products and streamflow data are available.

In countries where drought is a serious threat to agriculture, water resources and increases risk of wildfires, there has been an increasing interest in using weather modification techniques to improve local precipitation.  Despite the fact that cloud seeding is used in many places around the world, there is still not a clear consensus on its effectiveness. Various evaluation methods, including aircraft and ground-based measurements, remote sensing, statistical analysis, and numerical simulation have been widely used to evaluate the effect of cloud seeding. While each of these methods has its own benefits, they also come with limitations. For example, numerical models can simulate cases with and without cloud seeding and these scenarios can be then compared to determine how much rainfall increases after the seeding. Predictions provided by such scenarios can help with decision-making before conducting a cloud seeding experiment. The downside of numerical simulations is the presence of both systematic and random errors that originate from uncertainties initial conditions and numerical approximations. Ensemble forecasts can capture some of these uncertainties and provide a range of possible outcomes. The main goal of this study is to explore the potential of an ensemble forecasting system in evaluating the efficacy of cloud seeding. We used a limited-area ensemble forecasting system which is based on Met Office Unified Model coupled with Weather Research and Forecasting Model (WRF). The initial conditions of 13 ensemble members were created by downscaling of global ensemble model that was perturbed using the Ensemble transform Kalman filter. The WRF model was used to downscale the ensemble members to a finer resolution and simulate the seeding effect using an algorithm that was added to the Morrison microphysics scheme. As study cases, we utilized several cloud seeding experiments that were conducted during the weather modification campaign of 2022 and 2023 by the National Institute of Meteorological Sciences and the Korea Meteorological Administration. Ground-based hygroscopic cloud seeding was conducted in the mountainous region of South Korea using calcium chloride flares. Simulations were conducted for scenarios with and without seeding, and the difference in rainfall we assessed using ensemble mean, ensemble spread, and probabilities of various rainfall increment thresholds. The overall ensemble performance in rain predictability of the ensemble forecasts was evaluated using commonly used techniques, including the Brier score, reliability diagrams and ROC curves. Acknowledgments: This work was funded by the Korea Meteorological Administration Research and Development Program “Research on Weather Modification and Cloud Physics” under Grant (KMA2018-00224).

To support disaster prevention, it is essential to know in advance when scenarios start to distinguish one from the others, thus requiring the development of early detection methods of such separations. Ensemble prediction systems has been developed to provide scenarios of evolutions via their ensemble members, because the future state of the atmosphere predicted by a single ensemble member is less meaningful than the estimate of the future probability density from all the ensemble members. By construction, the primary function of an ensemble prediction system is to provide forecasters with a degree of uncertainty and level of confidence. For the last couple of years, we have developed different approaches which take advantage of the information provided by different ensemble prediction systems.Tropical cyclone tracks forecasted by a prediction system sometimes group together into trajectories parting away from each other. An objective method, based on a robust clustering approach, has been created to detect such separation scenarios in the Mesoscale Ensemble Prediction System developed by the Japan Meteorological Agency. At each initialization time, when the number of clusters is greater than 1, local separation scenarios exist. Separations are related to different steering environments predicted by the different ensemble members.On the same data used for the clustering, we also applied biological concepts such as the Waddington’s epigenetic landscape, and bioinformatics techniques like the graph-Hodge decomposition, to produce “MeteoScape”, an innovative way to characterize the evolution of a tropical cyclone. Particularly, “MeteoScape” can detect the possible paths and their associated probabilities of realization, as well as the regional separatrix, i.e., the spatial boundary between paths/scenarios, regardless of the initialization time (as in the clustering approach).Using the cases of intense precipitations that occurred in western Japan in July 2018 and August 2021, we introduce a way to detect separation scenarios in a n-dimensional space. A dimensionality reduction technique is applied to the geopotential height at 500 hPa extracted from two different ensemble prediction systems (the Japanese Mesoscale Ensemble Prediction System and the North American National Centers for Environmental Prediction). In the resulting 3-dimensional latent space, trajectories of ensemble members at different initialization times can also part away. We show that these separations are linked to attracting atmospheric trajectories.Further, the different techniques developed provide information on controllability, particularly where and when a manipulation could be performed.

We present an ensemble prediction system using a Deep Learning Weather Prediction (DLWP) model that recursively predicts six key atmospheric variables with six‐hour time resolution. This computationally efficient model uses convolutional neural networks (CNNs) on a cubed sphere grid to produce global forecasts. The trained model requires just three minutes on a single GPU to produce a 320‐member set of six‐week forecasts at 1.4° resolution. Ensemble spread is primarily produced by randomizing the CNN training process to create a set of 32 DLWP models with slightly different learned weights. Although our DLWP model does not forecast precipitation, it does forecast total column water vapor and gives a reasonable 4.5‐day deterministic forecast of Hurricane Irma. In addition to simulating mid‐latitude weather systems, it spontaneously generates tropical cyclones in a one‐year free‐running simulation. Averaged globally and over a two‐year test set, the ensemble mean RMSE retains skill relative to climatology beyond two‐weeks, with anomaly correlation coefficients remaining above 0.6 through six days. Our primary application is to subseasonal‐to‐seasonal (S2S) forecasting at lead times from two to six weeks. Current forecast systems have low skill in predicting one‐ or 2‐week‐average weather patterns at S2S time scales. The continuous ranked probability score (CRPS) and the ranked probability skill score (RPSS) show that the DLWP ensemble is only modestly inferior in performance to the European Center for Medium Range Weather Forecasts (ECMWF) S2S ensemble over land at lead times of 4 and 5–6 weeks. At shorter lead times, the ECMWF ensemble performs better than DLWP.

[Departement_IRSTEA]Eaux [TR1_IRSTEA]ARCEAU

Ensemble forecasts are often calibrated using regression methods known as ensemble model output statistics (EMOS). Most of the EMOS methods described in the literature make the assumption that statistical parameter uncertainty is negligible, although making this assumption has been shown to be detrimental to forecast quality. We use simulations to show that this assumption would be expected to give unreliable forecasts and underprediction of extremes. Previous research has described how parameter uncertainty can be included in EMOS models using bootstrapping. We show that bootstrapping can be replaced with an analytical approach based on right Haar prior (RHP) theory, a branch of objective Bayesian statistics. For two commonly used EMOS models, simple linear regression (SLR) and heteroscedastic linear regression (HLR, also known as non‐homogeneous Gaussian regression), we show that RHP theory leads to perfectly reliable predictions, if the distributional assumptions are met. We apply SLR and HLR methods, with and without parameter uncertainty, to 5 years of past medium‐range forecasts for eight Northern Hemisphere locations and evaluate forecast accuracy and reliability. We find that the RHP version of HLR is the most accurate in terms of both log‐score and continuous ranked probability score. It is also the most reliable, and it reduces the underestimation of extremes seen in the other methods. Our results with regard to the benefits of including parameter uncertainty agree with the previous results based on bootstrapping, although the RHP calculations are faster and more accurate. A free software library is available for applying the various methods, and execution is computationally inexpensive. We conclude that the RHP version of HLR should be used in preference to the other methods tested. Further research would be needed to understand how to use objective Bayesian methods to include parameter uncertainty in other EMOS models.