Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting

[Departement_IRSTEA]Eaux [TR1_IRSTEA]ARCEAU

Problems and prospects in long and medium range weather forecasting

The forecast of typhoon tracks remains uncertain and is positively related to the accuracy of the storm surge forecast. The storm surge prediction error increases dramatically when the forecast track error is larger than 100 km. This study aims to develop an ensemble storm surge prediction system using parametric weather models to account for the uncertainty in typhoon track prediction. The storm surge model adopted in this study is COMCOT-SS storm surge forecast system. Two methods are introduced and analyzed to generate the ensemble members in this study. One is from the weather ensemble prediction system (WEPS), and the other is from the error distribution of the deterministic forecasts (EDF). The ensemble prediction results show that the ensemble mean of WEPS performs similarly to the deterministic forecast. However, the maximum surge height of WEPS is often lower than one from EDF. The verification results suggest that, for disaster prevention, EDF provides stronger warnings to the coastal region than WEPS. However, it may provide overestimated forecasts in some cases.

Ensemble forecasting has become the prevailing method in current operational weather forecasting. Although ensemble mean forecast skill has been studied for many ensemble prediction systems (EPSs) and different cases, theoretical analysis regarding ensemble mean forecast skill has rarely been investigated, especially quantitative analysis without any assumptions of ensemble members. This paper investigates fundamental questions about the ensemble mean, such as the advantage of the ensemble mean over individual members, the potential skill of the ensemble mean, and the skill gain of the ensemble mean with increasing ensemble size. The average error coefficient between each pair of ensemble members is the most important factor in ensemble mean forecast skill, which determines the mean-square error of ensemble mean forecasts and the skill gain with increasing ensemble size. More members are useful if the errors of the members have lower correlations with each other, and vice versa. The theoretical investigation in this study is verified by application with the T213 EPS. A typical EPS has an average error coefficient of between 0.5 and 0.8; the 15-member T213 EPS used here reaches a saturation degree of 95% (i.e., maximum 5% skill gain by adding new members with similar skill to the existing members) for 1–10-day lead time predictions, as far as the mean-square error is concerned.

TIGGE (THORPEX International Grand Global Ensemble) was a major part of the THORPEX (Observing System Research and Predictability Experiment). It integrates ensemble precipitation products from all the major forecast centers in the world and provides systematic evaluation on the multimodel ensemble prediction system. Development of meteorologic-hydrologic coupled flood forecasting model and early warning model based on the TIGGE precipitation ensemble forecast can provide flood probability forecast, extend the lead time of the flood forecast, and gain more time for decision-makers to make the right decision. In this study, precipitation ensemble forecast products from ECMWF, NCEP, and CMA are used to drive distributed hydrologic model TOPX. We focus on Yi River catchment and aim to build a flood forecast and early warning system. The results show that the meteorologic-hydrologic coupled model can satisfactorily predict the flow-process of four flood events. The predicted occurrence time of peak discharges is close to the observations. However, the magnitude of the peak discharges is significantly different due to various performances of the ensemble prediction systems. The coupled forecasting model can accurately predict occurrence of the peak time and the corresponding risk probability of peak discharge based on the probability distribution of peak time and flood warning, which can provide users a strong theoretical foundation and valuable information as a promising new approach.

The National Center for Environmental Predictions (NCEP) has produced an ensemble meteorological reforecast product by using a fixed version of Global Forecast System (GFS) ensemble prediction system since 1 January 1979. The 15‐member ensemble product, with a global coverage at a 2.5° × 2.5° spatial resolution and a 14‐day lead time, has been used successfully by the River Forecast Centers of the National Weather Service (NWS) to produce basin scale precipitation and temperature ensemble forecasts in the US for several years now. This study evaluates the predictive skill of post‐processed ensemble forecasts based on GFS precipitation reforecast in China's Huai river basin. The evaluation is carried out in 15 sub‐areas of the Huai river basin and covers the 1/1/1981–31/12/2003 period. The Ensemble Pre‐Processing system version 3 (EPP3), developed at NWS, is used to develop joint probability distributions between forecasted ensemble mean precipitation and corresponding observations and to generate individual ensemble members that preserve space–time correlation of the observed precipitation data. Several statistical verification measures are used to quantify the goodness of fit between post‐processed (i.e. EPP3 processed) ensemble mean and observation and to assess the ensemble spread. Results indicate that the post‐processed forecasts have meaningful predictive skill for the first few days for ensemble daily precipitation forecasts. Predictive skill of ensemble forecasts of cumulative precipitation for lead times up to 14 days are significant. The forecast skill is highly dependent on seasonality, with relatively lower skills seen for wet summer season, when convective storm patterns dominate, as compared with other seasons. The predictive skill of the post‐processed ensemble precipitation is much better than the raw forecasts and the climatological ensemble forecasts. The results from this study suggest that the NCEP's GFS reforecasts can be a valuable resource for places other than the US. Copyright © 2012 John Wiley & Sons, Ltd.

In the companion paper (Järvinen et al., 2011. Q. J. R. Meteorol. Soc. DOI: 10.1002/qj.923) we suggested a concept to estimate numerical weather prediction model closure parameters on‐line, coupled with an operational ensemble prediction system. In this paper, such a method is developed and demonstrated in low‐order numerical tests. The method utilizes the massive amount of operational model predictions to make statistical inference about the underlying probability distributions of the closure parameters. This is achieved with practically no additional computations. The only required change to the ensemble prediction system is to allow perturbation of the model closure parameters for different ensemble members. Otherwise, the method is straightforward. The parametric uncertainty is presented using a hierarchical statistical model. Proposed parameter values are resampled based on their respective likelihood function values, as evaluated against verifying observations. Update formulas are used to feed back the parametric information to the evolving proposal distribution. The method for ensemble prediction and parameter estimation system (EPPES) is demonstrated using a stochastic version of the Lorenz‐95 model. The numerical tests show that the EPPES method is capable of detecting unknown and wrongly specified parameter values, and lead to optimal forecast skill in an independent test run. Potential show‐stoppers are discussed. Our current research is directed towards a realistic ensemble prediction system. Copyright © 2011 Royal Meteorological Society

Uncertainty propagation within a water level ensemble prediction system

Forecast verification is important across scientific disciplines, as it provides a framework for evaluating the performance of a forecasting system. In the atmospheric sciences, probabilistic skill scores are often used for verification, as they provide a way of ranking the performance of different probabilistic forecasts unambiguously. In order to be useful, a skill score must be proper: it must encourage honesty in the forecaster and reward forecasts that are reliable and have good resolution. A new score, the error‐spread score (ES), is proposed, which is particularly suitable for evaluation of ensemble forecasts. It is formulated with respect to the moments of the forecast. The ES is confirmed to be a proper score and is therefore sensitive to both resolution and reliability. The ES is tested on forecasts made using the Lorenz ′96 system and found to be useful for summarizing the skill of the forecasts. The European Centre for Medium‐Range Weather Forecasts (ECMWF) ensemble prediction system (EPS) is evaluated using the ES. Its performance is compared with a perfect statistical probabilistic forecast: the ECMWF high‐resolution deterministic forecast dressed with the observed error distribution. This generates a forecast that is perfectly reliable if considered over all time, but does not vary from day to day with the predictability of the atmospheric flow. The ES distinguishes between the dynamically reliable EPS forecasts and the statically reliable dressed deterministic forecasts. Other skill scores are tested and found to be comparatively insensitive to this desirable forecast quality. The ES is used to evaluate seasonal range ensemble forecasts made with the ECMWF System 4. The ensemble forecasts are found to be skilful when compared with climatological or persistence forecasts, though this skill is dependent on the region and time of year.

<scp>RMetS M</scp>eteorological <scp>M</scp>asterclass <scp>S</scp>eries: Anticipating floods, droughts and heatwaves

Abstract: With the development of technology, major operational centers and scientific research institutions have carried out research and development of convection permitting ensemble prediction technology and systems. In order to meet the needs of accurate forecasting in the capital, the Convection permitting ensemble prediction system of North China CMA-BJ-EN v1.0 developed by the Institute of Urban Meteorology was officially put into operation in January 2023. This paper introduces the research and development background and key technologies of the system in detail. The results show that the CMA-BJ-EN v1.0 system has a resolution of 3km convective resolution scale, covering 21 members, and can provide hourly probability prediction results of 48h forecast effectiveness in North China; The system is coupled with advanced technologies such as initial conditon perturbation of ensemble data assimilation, stochastic physical process tendency(SPPT) model perturbation, and has complete pre-processing, initial condition perturbation, model perturbation, and post-processing configuration, which can provide a series of probability prediction products; The qualitative and quantitative evaluation of CMA-BJ-EN v1.0 system and its comparison with NCEP global ensemble forecast system show that the system can effectively grasp several major weather processes in North China, and can obtain local refined probability forecast results relative to global ensemble; The statistical results also show that compared with the NCEP GEFS global ensemble forecast, the CMA-BJ-EN v1.0 forecast system can effectively reduce the RMSE of 2m temperature and 10m wind speed, and can effectively improve the probability forecast score of precipitation forecast. The operation of the system can provide users with effective refined probability forecast reference.Keywords: convection permitting, ensemble prediction system, initial condition perturbation, model perturbation

Developing an ensemble hydrologic prediction system is essential for reservoir operations and flood early warning. However, efforts to build hydrologic ensemble prediction systems considering the influence of reservoirs have been lacking in India. We examine the potential of the Extended Range Forecast System (ERFS, 16 ensemble members) and Global Ensemble Forecast System (GEFS, 21 ensemble members) forecast for streamflow prediction in India using the Narmada River basin as a testbed. We use the Variable Infiltration Capacity (VIC) with reservoir operations (VIC-Res) scheme to simulate the daily river flow at four locations in the Narmada basin. We examined the streamflow forecast skills of the ERFS forecast for the period 2003–2018 at 1–32 day lead. We compared the streamflow forecast skills of raw meteorological forecasts from ERFS and GEFS at a 1–10 day lead for the summer monsoon (June–September) 2019–2020. The ERFS forecast underestimated extreme precipitation against the observations compared to the GEFS during the summer monsoon of 2019–2020. However, both the forecast products showed better skills for minimum and maximum temperatures than precipitation. Ensemble streamflow forecast from the GEFS performed better than the ERFS during 2019–2020. The performance of the GEFS based ensemble streamflow forecast declines after five days lead. Overall, the GEFS ensemble streamflow forecast can provide reliable skills at a 1–5 day lead. Our findings provide directions for developing a flood early warning system based on ensemble streamflow prediction considering the influence of reservoirs in India.

Developing an ensemble hydrologic prediction system is essential for reservoir operations and flood early warning. However, efforts to build hydrologic ensemble prediction systems considering the influence of reservoirs have been lacking in India. We examine the potential of the Extended Range Forecast System (ERFS, 16 ensemble members) and Global Ensemble Forecast System (GEFS, 21 ensemble members) forecast for streamflow prediction in India using the Narmada River basin as a testbed. We use the Variable Infiltration Capacity (VIC) with reservoir operations (VIC-Res) scheme to simulate the daily river flow at four locations in the Narmada basin. We examined the streamflow forecast skills of the ERFS forecast for the period 2003–2018 at 1–32 day lead. We compared the streamflow forecast skills of raw meteorological forecasts from ERFS and GEFS at a 1–10 day lead for the summer monsoon (June–September) 2019–2020. The ERFS forecast underestimated extreme precipitation against the observations compared to the GEFS during the summer monsoon of 2019–2020. However, both the forecast products showed better skills for minimum and maximum temperatures than precipitation. Ensemble streamflow forecast from the GEFS performed better than the ERFS during 2019–2020. The performance of the GEFS based ensemble streamflow forecast declines after five days lead. Overall, the GEFS ensemble streamflow forecast can provide reliable skills at a 1–5 day lead. Our findings provide directions for developing a flood early warning system based on ensemble streamflow prediction considering the influence of reservoirs in India.

U.S. National Weather Service (NWS) River Forecast Centers (RFCs) have produced monthly to seasonal hydrologic forecast ensemble outlooks for many years using historical precipitation and temperature as the forcing data for the ensembles within the Advanced Hydrologic Prediction System (AHPS). This has worked well for many water resources applications relying on long lead-time ensembles of greater than thirty days. In the eastern United States, there is considerable interest in shorter lead-time ensembles, particularly in the period of one to seven days that convey hydrologic forecast uncertainty resulting directly from the uncertainties in precipitation and temperature forecasts, with a focus on flood forecasting. This paper demonstrates the use of numerical weather prediction (NWP) model ensembles as the forcing for hydrologic models within the NWS hydrologic forecasting system, the Deltares Flood Early Warning System (FEWS) based Community Hydrologic Prediction System (CHPS), utilizing Ensemble Streamflow Prediction (ESP) system techniques. This approach makes it possible to produce probabilistic hydrologic forecasts that incorporate the skill and uncertainty of atmospheric models. Significant issues exist with directly using the meteorological ensembles, including biases and spatial resolution of the available data, but the approach allows for the generation of valuable information for end-user decision makers, covering a time frame for which there is little current probabilistic hydrologic guidance available. This paper presents a description of a joint project among four NWS RFCs to incorporate ensemble forecasts from a variety of meteorological model ensembles. These include the National Oceanic and Atmospheric Administration (NOAA) NWS National Centers for Environmental Prediction (NCEP) Global Ensemble Forecast System (GEFS), North American Ensemble Forecast System (NAEFS), and Short Range Ensemble Forecast System (SREF). Verification results are presented for operational NAEFS and SREF based MMEFS real-time probabilistic hydrologic forecasts covering multi-year period. Improvements to the MMEFS are also recommended.

To support disaster prevention, it is essential to know in advance when scenarios start to distinguish one from the others, thus requiring the development of early detection methods of such separations. Ensemble prediction systems has been developed to provide scenarios of evolutions via their ensemble members, because the future state of the atmosphere predicted by a single ensemble member is less meaningful than the estimate of the future probability density from all the ensemble members. By construction, the primary function of an ensemble prediction system is to provide forecasters with a degree of uncertainty and level of confidence. For the last couple of years, we have developed different approaches which take advantage of the information provided by different ensemble prediction systems.Tropical cyclone tracks forecasted by a prediction system sometimes group together into trajectories parting away from each other. An objective method, based on a robust clustering approach, has been created to detect such separation scenarios in the Mesoscale Ensemble Prediction System developed by the Japan Meteorological Agency. At each initialization time, when the number of clusters is greater than 1, local separation scenarios exist. Separations are related to different steering environments predicted by the different ensemble members.On the same data used for the clustering, we also applied biological concepts such as the Waddington&amp;#8217;s epigenetic landscape, and bioinformatics techniques like the graph-Hodge decomposition, to produce &amp;#8220;MeteoScape&amp;#8221;, an innovative way to characterize the evolution of a tropical cyclone. Particularly, &amp;#8220;MeteoScape&amp;#8221; can detect the possible paths and their associated probabilities of realization, as well as the regional separatrix, i.e., the spatial boundary between paths/scenarios, regardless of the initialization time (as in the clustering approach).Using the cases of intense precipitations that occurred in western Japan in July 2018 and August 2021, we introduce a way to detect separation scenarios in a n-dimensional space. A dimensionality reduction technique is applied to the geopotential height at 500 hPa extracted from two different ensemble prediction systems (the Japanese Mesoscale Ensemble Prediction System and the North American National Centers for Environmental Prediction). In the resulting 3-dimensional latent space, trajectories of ensemble members at different initialization times can also part away. We show that these separations are linked to attracting atmospheric trajectories.Further, the different techniques developed provide information on controllability, particularly where and when a manipulation could be performed.