Evaluation of CMPAS precipitation products over Sichuan, China

Shiying Li , Xiaolong Huang , Wei Wu , Bing Du , Yuhe Jiang

Sichuan Meteorological Observation and Data Center, Heavy Rain and Drought-Flood Disasters in Plateau and Basin Key Laboratory of Sichuan Province, Chengdu,China

Keywords:Multisource precipitation products High-resolution precipitation CMPAS Applicability assessment

ABSTRACT High-quality and high-resolution precipitation data are the basis for mesoscale numerical weather forecasting,model verification, and hydrological monitoring, which play an important role in meteorological and hydrological disaster prevention and mitigation. In this study, high-density rain gauge data are used to evaluate the fusion accuracy of the China Meteorological Administration Multisource Precipitation Analysis System (CMPAS), and four CMPAS products with different spatial and temporal resolution and different data sources are compared, to derive the applicability of CMPAS. Results show that all the CMPAS products show high accuracy in the Sichuan Basin, followed by Panxi Area and the western Sichuan Plateau. The errors of the four products all rise with the increase in precipitation. CMPAS overestimates precipitation in summer and autumn and underestimates it in spring and winter. Overall, the applicability of these fused data in the Sichuan Basin is quite good. Due to the lack of observations and the influence of the terrain and meteorological conditions, the evaluation of CMPAS in the plateau area needs further analysis.

1. Introduction

Precipitation is a significant part of the water cycle and energy exchange in the climate system, as well as an important indicator of climate change ( Syed et al., 2004 ). Extreme weather and climate events related to precipitation, such as floods and droughts, have a considerable impact on human life ( Hirabayashi et al., 2008 ). Besides, high-quality precipitation observation products are also needed to support numerical weather prediction ( Lin et al., 2005 ). Therefore, the development of precipitation analysis products with high spatiotemporal resolution and high precision is needed to reasonably and accurately estimate the spatiotemporal distribution of precipitation, which is of great significance for research in the fields of weather, climate, ecology, agriculture, and environment.

Precipitation data can be obtained in three ways: from surface rain gauges, ground-based radar, and satellite remote sensing. Observations from surface rain gauges are the most reliable, but there are obvious discontinuities in the spatial and temporal distribution of station observations, making it difficult to reflect the basic climatological characteristics in terms of overall spatial change ( Morrissey et al., 1995 ;Villarini and Krajewski, 2008 ). Satellite remote sensing can carry out continuous detection over a wide range of space and has a high temporal resolution for certain target areas ( Michaelides et al., 2009 ). However, due to the limitations of the physical principles and algorithms of satellite precipitation retrieval, the retrieval accuracy is relatively low, especially for solid precipitation ( Prigent, 2010 ). Also, because of the problems with estimation methods and the calibration of several radars, the accuracy of precipitation estimations from radar is not very high ( Montopoli et al., 2017 ). Therefore, how to effectively combine the advantages of precipitation data from different sources and develop technology that integrates surface observation, satellite and radar precipitation products into much higher quality products has emerged as a hot topic in international research in recent years.

To meet the needs of meteorological operations and scientific research, several high-quality and high-resolution precipitation products based on rain gauge observations and satellites have already been launched internationally. For instance, the National Oceanic and Atmospheric Administration in the United States has developed a number of products, starting with the Climate Prediction Center(CPC) Merged Analysis of Precipitation ( Xie et al., 2007 ) and the Global Precipitation Climatology Project ( Huffman et al., 1997 ), and then later the Tropical Rainfall Measuring Mission (TRMM) satellite equipped with the Precipitation Radar instrument to monitor precipitation ( Kummerow et al., 1998 ). At this time, high spatiotemporal resolution satellite merged products were further developed, including the TRMM_3B43, TRMM_3B42 ( Huffman et al., 2007 ), CMORPH (CPC Morphing Technique) Bias-corrected, and CMORPH Blended products( Joyce et al., 2004 ; Joyce and Xie, 2011 ; Xie et al., 2017 ). The Japan Meteorological Agency also adopted technology similar to CMORPH and produced its Global Satellite Mapping of Precipitation product( Okamoto et al., 2005 ).

In recent years, with the development of meteorological forecasts and services, the resolution and accuracy requirements of precipitation products have been improved. The National Meteorological Information Center (NMIC) of the China Meteorological Administration (CMA)introduced the “PDF (probability density function) + OI (optimal interpolation) ” merging method ( Xie and Xiong, 2011 ), which was developed by CPC in the United States, and then continuously optimized and improved this fusion method. Additionally, the NMIC has also adopted the “PDF + BMA (Bayesian model averaging) + OI + DS (downscaling) ”method ( Pan et al., 2015 , 2018a , 2018b ) and an integrated radar precipitation estimation approach; plus, it has developed a three-source (gauge,satellite, radar) merged precipitation product with spatial/temporal resolutions of 5 km/1 h and 1 km/1 h. This fused product is called the CMA Multisource Precipitation Analysis System (CMPAS), which takes full advantage of single-source precipitation products to form a comprehensive and high-quality merged precipitation product in China.

CMPAS at the 5 km resolution was transitioned into operation in June 2018, while the 1 km resolution version was operationalized in July 2020. So far, however, there have been few studies on the applicability of CMPAS in China. The present paper aims to address this knowledge gap, as the need to evaluate and improve the ability of CMPAS to support forecasting services and other applications is an urgent one.

2. Data and methods

2.1. Study area

Sichuan Province is located in southwestern China within the area 26°03 ′ —34°19 ′ N and 97°21 ′ —108°12 ′ E. Meteorologically, Sichuan is generally divided into three regions for analysis: the western Sichuan Plateau, the Sichuan Basin (central and eastern Sichuan), and Panxi Area(southwestern Sichuan). The western Sichuan Plateau is located on the east side of the Qinghai—Tibet Plateau, with an altitude of 4000—4500 m. A total of 17 cities are located in the Sichuan Basin, which is the core area of this province. Panxi Area is a part of the Yunnan—Guizhou Plateau.

2.2. Data

The CMPAS datasets are provided by the NMIC, CMA. Four CMPAS precipitation products are evaluated in this study: CMPAS_5km_FAST,CMPAS_5km_FRT, CMPAS_1km_RT, and CMPAS_1km_NRT. CMPAS_5km_FAST is a satellite—gauge merged precipitation product and CMPAS_5km_FRT is a radar—satellite—gauge merged precipitation product. The horizontal resolution of these two products is 0.05° × 0.05°(native resolution: 5 km), and the temporal resolution is hourly.CMPAS_1km_RT and CMPAS_1km_NRT are both radar—satellite—gauge merged precipitation products with a resolution of 0.01° × 0.01°(native resolution: 1 km), and again the temporal resolution is hourly.However, CMPAS_1km_RT is a real-time product (i.e., updated in real time) and CMPAS_1km_NRT is a near-real-time product (updated with a delay of about 24 h).

The fusion accuracy of CMPAS is evaluated using high-density rain gauge data. The hourly surface rain gauge data are collected from 156 national automatic weather stations (NAWS) and 5128 regional automatic weather stations (RAWS) in Sichuan Province, provided by the CMA and Sichuan Meteorological Service. Among them, the rain gauge data of 2923 RAWS are not fused into CMPAS. All the observational data have been quality controlled. The evaluation period in this study is from 0000 UTC 1 August 2019 to 2300 UTC 31 July 2020.

2.3. Analysis methods

Taking the hourly surface rain gauge data of 156 NAWS and 5128 RAWS in Sichuan as the real values, the CMPAS products are interpolated to 156 NAWS and 5128 RAWS by the bilinear interpolation method. The error and correlation between them in a period are statistically compared. The statistical indicators include the mean error (ME),relative error (RE), mean absolute error (MAE), correlation coefficient(COR), graded root-mean-square error ( RMS Ek), and threat score ( TSk) .The ME, RE, MAE, and COR —calculated as

and

whereOiis the station observation value,Giis the value obtained by interpolating the CMPAS products to stations, andNis the total number of samples (number of stations) —are used to analyze the spatial and temporal distributions; while RMS Ekand TSk—calculated as

and

wherekrepresents the precipitation classification level,Ukis the upper bound of thekth precipitation level interval,Lkis the lower bound of thekth precipitation level interval, and [] represents an operator that converts logic into a numerical value —are used for graded evaluation,in which the hourly precipitation is divided into five levels: 0.1—1.9 mm,2—4.9 mm, 5—9.9 mm, 10—19.9 mm, and 20 mm and above. When the logical value is positive, it is taken as 1; otherwise, it is taken as 0. The termsOi,GiandNhave the same meaning as in Eqs. (1) —(4) .

Lastly, the CMPAS products needed to be interpolated to station sites,for which the nearest-neighbor method was used. Taking the value of the nearest grid cell among four adjacent grid cells as the value of the interpolated grid cell, the distance from the interpolation grid cell,g(i,i) ,to the four adjacent grid cells,fk(i,i)(k= 1,2,3,4 ) , isdk(k= 1,2,3,4 ) :

3. Results and discussion

3.1. Comparative analysis of spatial distribution characteristics

The general spatial distribution characteristics of the two products at 5 km are similar, and the two products at 1 km also show a similar situation. The CMPAS_5KM_FRT product is used to represent the accuracy of the 5 km products and the CMPAS_1KM_NRT product to represent the 1 km products. The distribution of ME between the CMPAS products and observations is shown in Fig. 1 . According to the probability density function (PDF) distribution, the Sichuan Basin shows the smallest ME, especially for the 1 km product; 66% of the ME is in [ ? 0.01,0.01] mm h?1. The ME of the two resolutions in the western Sichuan Plateau is relatively large; only 34% (FRT) and 30% (NRT) is in [ ? 0.01,0.01] mm h?1. There are no obvious characteristics of overestimation and underestimation. The regional difference of ME for 1 km products is relatively notable, but the trend of the PDF distribution in the three regions is generally similar.

The distribution of MAE is shown in Fig. 2 . In the Sichuan Basin,the MAE of 42% of stations in the 1 km products is less than 0.02 mm h?1, but the 5 km product is significantly more, in [0.04, 0.05] mm h?1(41%). The MAE of Panxi Area and the western Sichuan Plateau is relatively large, and the difference between Panxi Area and the western Sichuan Plateau is not significant in the 5 km product, but the MAE of the 1 km product in Panxi Area is markedly smaller than that of the Plateau. For the 5 km product, 41.2% stations have an MAE in [0.07,0.1] mm h?1in Panxi Area, with 36% in [0.09, 0.12] mm h?1in the Plateau. For the 1 km product, the MAE of the western Sichuan Plateau is relatively larger, while 24% of stations have a smaller MAE ([0, 0.01]mm h?1) in Panxi Area.

The distribution of RE is shown in Fig. 3 . In the Sichuan Basin, 73%of stations have an RE in [0.045, 0.09] with the 5 km product, while 70%of stations have an RE lower than 0.06 with the 1 km product. For Panxi Area, most of the RE results are concentrated in [0.03, 0.135] with the 5 km products; whereas, the 1 km product has 30% of RE values lower than 0.03. For the western Sichuan Plateau, the RE results are striking;the majority of stations have an RE in [0.12, 0.18] with both products.

The distribution of COR is shown in Fig. 4 . In the Sichuan Basin and Panxi Area, the CORs in [0.8, 0.95] account for 84% and 60% of the 5 km products respectively, while 46% and 27% of the stations have CORs above 0.95 with the 1 km product; the 1 km resolution product is more correlated. In the western Sichuan Plateau, the correlation is lower than that in other regions, mostly in [0.6, 0.8], and the 5 km product shows a better correlation (61% in the 5 km product versus 21% in the 1 km product).

Fig. 1. Distribution of ME between CMPAS products and observations (units: mm h ? 1 ): (a) spatial distribution of CMPAS_5KM_FRT; (b) spatial distribution of CMPAS_1KM_NRT; (c) PDF distribution of CMPAS_5KM_FRT; (d) PDF distribution of CMPAS_1KM_NRT.

Fig. 2. Distribution of MAE between CMPAS products and observations (units: mm h ? 1 ): (a) spatial distribution of CMPAS_5KM_FRT; (b) spatial distribution of CMPAS_1KM_NRT; (c) PDF distribution of CMPAS_5KM_FRT; (d) PDF distribution of CMPAS_1KM_NRT.

Fig. 3. Distribution of RE between CMPAS products and observations: (a) spatial distribution of CMPAS_5KM_FRT; (b) spatial distribution of CMPAS_1KM_NRT; (c)PDF distribution of CMPAS_5KM_FRT; (d) PDF distribution of CMPAS_1KM_NRT.

Fig. 4. Distribution of COR between CMPAS products and observations: (a) spatial distribution of CMPAS_5KM_FRT; (b) spatial distribution of CMPAS_1KM_NRT;(c) PDF distribution of CMPAS_5KM_FRT; (d) PDF distribution of CMPAS_1KM_NRT.

Comparing the 1 km products to the 5 km products, the ME of about 75% stations is decreasing, the MAE of about 30% stations is reduced by more than 60%, 77% of stations show a decrease in RE, and the COR of over 60% of stations is rising. As a consequence, most errors in the 5 km products can be reduced in the 1 km products.

Overall, all products have higher accuracy in the Sichuan Basin, followed by Panxi Area and the western Sichuan Plateau. On the one hand,most of the stations in the plateau area are located in high mountains and valleys, and as a result the representativeness of the gauge data is poor. On the other hand, lacking automatic weather stations in the plateau area means that rain gauge data are insufficient, and for this reason the evaluation of CMPAS in the plateau area needs to be further analyzed in combination with more relevant data.

3.2. Comparative analysis of different precipitation levels

The samples with precipitation of more than 0.1 mm h?1account for about 10% of the total sample size. The RMSEs of different precipitation levels are shown in Table 1 . With the increase in precipitation level,the sample size gradually reduces, and the RMSE is on the rise. When the precipitation is 0.1—9.9 mm h?1, the RMSE of CMPAS_5KM_FRT is smaller. CMPAS_1KM_NRT performs better when the precipitation is larger than 10 mm h?1.

Table 1 RMSE of different precipitation levels.

In terms of TS ( Table 2 ), there is a downward trend in TS with increased precipitation. Apart from when the precipitation is 0.1—1.9 mm h?1, the performance of CMPAS_5KM_FRT is slightly better, and the TSs of CMPAS_1KM_NRT are higher in other levels.

Table 2 Threat scores of different precipitation levels.

3.3. Comparative analysis of temporal distribution characteristics

The seasonal variation characteristics are provided in Fig. 5 . The four products overestimate the precipitation in summer and autumn, while in spring and winter it is underestimated. The MAE is larger in summer and autumn, and the MAE of the 5 km resolution products is more noticeable. The RE is more distinct in summer and autumn, followed by spring and winter. There is more precipitation in summer, and the heavier the precipitation, the larger the errors. The CORs of the four seasons show a moderate change. Generally, the 1 km resolution products are more likely to have slight errors in the four seasons.

Fig. 5. Seasonal variation of evaluation characteristics: (a) ME; (b) MAE; (c); (RE); (d) COR.

4. Conclusions

High-density rain gauge data are used to evaluate the fusion accuracy of CMPAS over different terrain conditions, different precipitation levels, and different seasons. The accuracy of the four products is higher with NAWS. There is no obvious spatial distribution of overestimation and underestimation between the two different resolution products and gauge observations. The MAE of the four products is mostly concentrated below 0.12 mm h?1. More than 70% of stations have REs lower than 0.18 for all products. All the errors in the Sichuan Basin are relatively small. However, the 5 km resolution products are better correlated with observations in plateau areas, whereas in the Sichuan Basin the 1 km resolution products have a higher correlation.

The errors arise with the increase in precipitation. Seasonally, the four products overestimate precipitation in summer and autumn, while in spring and winter it is underestimated. The CORs of the four seasons are basically unchanged, and most are significantly correlated with observations.

In conclusion, all the CMPAS products show high fusion accuracy in the Sichuan Basin, followed by Panxi Area and the western Sichuan Plateau. As for the reason for the large errors in the plateau area, firstly,lacking surface observation data is a direct cause. Moreover, various topographic factors (altitude, slope, aspect, vegetation, etc.) and meteorological factors (temperature, humidity, wind speed, etc.) have a certain impact on surface precipitation observation, and consequently the rain gauge data may not be real values. Further work needs to be done to use precipitation observation data from non-meteorological industries to evaluate the accuracy of CMPAS, to promote the local application of products and improve product service capabilities.

Funding

This study was supported by the Sichuan Meteorological Bureau,the Sichuan Meteorological Observation and Data Center, the Heavy Rain and Drought-Flood Disasters in Plateau and Basin Key Laboratory of Sichuan Province [grant number SCQXKJQN202121], the Key Technology Development Project of Weather Forecasting [grant number YBGJXM(2020)1A-08], and the Innovative Development Project of the China Meteorological Administration [grant number CXFZ2021Z007].