A comparative verification of precipitation forecasts for selected numerical weather prediction models over Kenya

Abstract

This study examined the level of skill of daily precipitation forecasts of Numerical Weather Prediction (NWP) models over Kenya. These models are the United Kingdom Meteorological office (UKMO) model, United States' National Centers for Environmental Prediction (NCEP) global forecasting system and India's National Center for Medium Range Weather forecast (NCMRWF) global spectral model. The models' grid box rainfall was averaged and assigned to an observing station within the box. Verifications were then carried out against 24-hr observations. The rainfall data were from 22 synoptic stations for a period of one year, 2004/2005. Verification scores were derived from two way contingency tables of events and non events. Thresholds (i.e. 0.1, 0.2, 0.5, 1, 2, 5, 10, 15, 20, 25 mm/day) were considered to define the transition between a rain versus a non-rain event. At each verification point, each event/non rain .. event was scored as falling under one of the four categories of false alarms, misses, hits or correct non-rain forecasts. The main objective of the study was to perform a comparative verification of skills of three NWP models currently in use at Kenya Meteorological Department at predicting precipitation. In order to achieve the objective, categorical statistics based on contingency tables were derived. These comprised of bias score, probability of detection, false alarm ratio, and Hansen-Kuipers score. Root mean square errors for checking the accuracy of the different models were also calculated. Statistics are presented of the bias score, probability of detection, false alarm ratios and the Hansen-Kuipers score also known as the 'true skill statistic'. The results showed significant monthly variations of root mean square values of rainfall. For all the models the root mean square error was found to be largest during the rainy season of March-May. However the error was found to be relatively low for the drier months. Bias results indicated that the bias to be well above one at low thresholds and below one at higher thresholds. This means that the models overestimate the frequency of rainfall for light rainfall while underestimating the same for heavy rainfall. The probabilities of detection values were found to be very high especially for the UKMO model for rainfall up to about athreshold of Smm. The false alarm ratios scores also varied geographically with higher values in the dry southeastern, Northeastern and Northwestern parts of the country. Model skill as measured by Hansen- Kuipers score indicated that UKMO and NCMR WF models had greater skill in forecasting rainfall than the NCEP model over most parts of the country. The Hansen-Kuipers(HK) score peaks between the thresholds of2mm -Smm. Thereafter the HK score decreases rapidly. The results of this study may provide a basic source of information to operational weather forecasters on the skill of precipitation forecasts from the various models used as input to their daily operations. The scores may also form a benchmark against which to measure skill improvement ofNWP models in future. On the overall, no single model was better th~n the others in all aspects of accuracy and skill. The UKMO tended to have higher values of bias than the other models. It also overestimated amounts of rainfall. However it was able to detect presence of rainfall better than the other models and its Hansen-Kuipers score was higher. On the other hand NCMRWF model tended to have lower values of bias. Its false alarm ratio scores were also lower. Its Hansen - Kuipers values were also higher than the other models especially over western Kenya. On the whole NCEP model seemed to perform poorly in most of the skill scores than the other two models. However it was giving fewer alarms than the UKMO model.