Do convection-permitting ensembles lead to more skilful short-range probabilistic rainfall forecasts over tropical East Africa?

By Dr Carlo Cafaro for GCRF African SWIFT


Most of the rainfall in the tropical Africa region comes from convective systems. Forecasting convection is both challenging and crucial – slight changes in surface temperatures, cloud amounts, or winds aloft can drastically affect the timing, location and intensity of the convection. Thus, convective rainfall forecasts can be uncertain even a few hours/days ahead of the event. Also, convective systems can produce heavy rainfall amounts which could lead to floods and landslides across the region (Figure 1), posing a serious threat to the local population.

To this end, for the first time, the UK Met Office ran a set of novel convection-permitting (CP) ensemble simulations at short-range timescales (up to three days ahead) to assist a forecasting testbed held at the Kenya Meteorological Department in April-May 2019.

CP models are run at kilometre scale and can therefore represent convective storms of a few kilometres across. The ensemble consists of multiple model runs (members), with slightly different initial and boundary conditions to provide a range of the possible outcomes. This helps the forecasters to assess the uncertainty of the forecasts and the end-users to take more informed decisions.

Figure 1 – Flooding wipes out a bridge over Tana river, Kenya. (Public domain photo by SGT R.A. Ward, U.S. Marine Corps.)

Past studies have shown that CP ensembles are more skilful than other cheaper forecasting systems such as CP deterministic or coarser resolution ensembles, and they have been used operationally by many national meteorological services (NMS) in the world.

However, such studies have mainly focussed on the mid-latitude regions, with only a few of them dealing with tropical areas, and Africa in particular.

Therefore, a first quantification of the skill of CP ensembles over tropical East Africa was performed by researchers at the University of Reading, University of Leeds, and the UK Met Office. This task was part of the Evaluation Work package of the GCRF African SWIFT project with the aim to assess whether such models can also be implemented in the operational practices of the local NMSs partners to increase forecasting capability.

Probabilistic forecasts of heavy rainfall were generated from the CP ensembles, CP deterministic and coarser resolution ensembles, and verified using
a variety of verification metrics to assess the ability to predict accurately the location of the rainfall event and discriminate between events and non-events over the whole domain and different sub-regions respectively (Figure 2).

Figure 2 – A map showing the elevation for the domain spanned by the convection-permitting ensemble model for tropical East Africa. Black dashed boxes enclose the different subregions considered for regional differences in rainfall characteristics, including the Lake Victoria basin (LV). The red dashed box encloses the region used for calculating the fractions skill score (FSS). Ocean points are not considered.

Verification results show that the CP ensemble does provide more skill than either a CP deterministic or the global ensemble in predicting the location of the most intense 3h rainfall accumulations (Figure 3). This implies that including CP ensembles into the operational practices of African NMSs could lead to better and more informative forecasts.

Figure 3 – Mean Fractions Skill Score (FSS) corresponding to the (a) 90th, (b) 95th, (c) 97th, (d) 99th, (e) 99.5, and (f) 99.9 percentile as a function of the forecast hour. FSS is computed for a neighbourhood of n=23 (~ 250 km) for rainfall exceeding the 97th percentile. The dashed line indicates the FSS value of 0.5: when FSS is above this value, it is considered “useful”. Values on the x axis represent starting forecast hours of the 3-h accumulation periods (e.g., an x-axis value of 24 is for 3 h accumulated between 24 and 27 h).

However, there is still room for improvement since the skill remains quite low with respect to mid-latitude regions. CP ensembles were shown to be biased in terms of rainfall amount (Figure 4) and rainfall location was under-spread (Figure 5), meaning that uncertainty in the location of the rainfall was not well captured by the ensemble.

Figure 4 – Average physical thresholds [mm (3 h)] over all the forecasts corresponding to (a) 90th, (b) 95th, (c), 97th, (d) 99th, (e) 99.5th, and (f) 99.9th percentile threshold as a function of the forecast hour. The physical thresholds were computed for the large domain (red dashed box in Fig. 1) for each day and for each 3-h period separately. The green and blue shadings encompass the CP and global ensembles distributions, respectively. Values on the x axis represent starting forecast hours of the 3-h accumulation periods (e.g., an x-axis value of 24 is for 3 h accumulated between 24 and 27 h).
Figure 5: Measures of the forecast skill (eFSS) and of the ensemble agreement (dFSS) as a function of forecast lead time for 3-h accumulation periods (a) 0000–0300 UTC and (b) 1200–1500 UTC. FSS is computed for a neighbourhood of n=23 (~ 250 km) for rainfall exceeding the 97th percentile. Different markers correspond to different model initialization times.

Overall, this research underlines the importance of implementing kilometre-scale models to provide forecasts with more detail about the timing and location of the rainfall events, which will allow national meteorological services to issue more accurate severe weather warnings. Ensembles can add even more skill to forecasts and give an uncertainty estimate for an event, provided that the spread of the members is not too small.

Kilometre-scale ensembles are still novel, especially in the Tropics, so the best way to display information from these models is an interesting area of research. For African SWIFT Testbed 3, scheduled for September 2021, efforts should be made to construct even more useful and clear probabilistic products from the ensembles to ensure the information can be easily digested by local forecasters.

Paper Information

Cafaro, C., Woodhams, B. J., Stein, T. H. M., Birch, C. E., Webster, S., Bain, C. L., Hartley, A., Clarke, S., Ferrett, S. & Hill, P. (2021). Do Convection-Permitting Ensembles Lead to More Skilful Short-Range Probabilistic Rainfall Forecasts over Tropical East Africa?, Weather and Forecasting, 36(2), 697-716. https://doi.org/10.1175/WAF-D-20-0172.1

About the Author

Dr Carlo Cafaro has been a research scientist at the Department of Meteorology, University of Reading since January 2019 where he has been working on the evaluation of convection-permitting ensembles and interpretation of probabilistic forecasts of heavy rainfall.

Before joining the African SWIFT project, Dr Cafaro completed a PhD in Mathematics at University of Reading as part of the Mathematics of Planet Earth Centre for Doctoral Training with a thesis on quantifying the information gain from convection-permitting ensembles for sea breezes and wind gust forecasts.