Analysis of Problems and Policy Recommendations for the Third-party Evaluation in the Present Poverty Alleviation

Targeted Poverty Alleviation and Development Institute, Yunnan University of Finance and Economics, Kunming 650221, China

Abstract The introduction of third-party evaluation system and the implementation of third-party evaluation system in poverty alleviation is a major innovation in the targeted poverty alleviation work in China. In order to improve the third-party evaluation mechanism, further improve the efficiency of third-party evaluation, and enhance the effectiveness of targeted poverty alleviation, this paper analyzes and summarizes the main problems existing in the current third-party evaluation practice. In view of the related problems, this paper puts forward the corresponding policy recommendations, which are believed to play a positive role in further promoting the development and improvement of the third-party evaluation of getting rid of poverty.

Key words Poverty alleviation, Third-party evaluation, Existing problems, Policy recommendations

1 Introduction

By 2020, "it is necessary to ensure that the rural poor are lifted out of poverty under the current standards of our country and all the poor counties will shake off poverty, so as to solve the overall regional poverty"[1]. This is the general goal of our country to win the battle to get rid of poverty, and it is also the most arduous task of building a well-off society in an all-round way. China’s poverty alleviation and development has entered the stage of overcoming difficulties. To achieve the goal of lifting more than 70 million rural poor people out of poverty in an all-round way by 2020, time is tight and the task is onerous.

We must open up innovative ideas and methods for poverty alleviation and development on the basis of the existing ones, and firmly promote the final victory in getting rid of poverty[2]. Based on this situation, the third-party evaluation mechanism has been widely used in the process of getting rid of poverty in our country. In February 2016, the General Office of the CPC Central Committee and the General Office of the State Council issued theMeasuresforAssessingtheEffectivenessofPovertyAlleviationandDevelopmentWorkoftheProvincialPartyCommitteeandtheGovernment, which clearly proposed that the Leading Group of Poverty Alleviation and Development of the State Council entrust relevant scientific research institutions to carry out a third-party evaluation of the effectiveness of targeted poverty alleviation work, and carry out an annual evaluation from 2016 to 2020[3]. On the exit of poor counties, in order to ensure that the results of poverty alleviation are true and credible and the effectiveness of poverty alleviation is accurate, the state stipulates that third-party evaluations of poverty alleviation or poverty reduction in poor counties should be organized. The so-called "third-party evaluation" means that the Leading Group for Poverty Alleviation and Development entrusts relevant scientific research institutions and social organizations to evaluate the relevant performance indicators by means of special investigation, sampling survey and on-the-spot verification, to make poverty eradication data more reliable and impartial. Because of the neutral identity and professional perspective of "third-party", it can effectively avoid the official "self-speaking" evaluation method, so that the evaluation results tend to be more objective and reliable[4]. Therefore, it has become a feasible external checks and balances mechanism. Using the way of "promoting reform by evaluation", it has effectively promoted the path optimization of targeted poverty alleviation implementation of the government, and improved the effectiveness of poverty alleviation. Third-party evaluation has been given a special historical mission and has a great responsibility. Therefore, it is particularly important to sum up the experience and existing problems of third-party evaluation in time.

2 Brief situation of research at home and abroad

Foreign third-party evaluation developed earlier and is relatively mature. In the 1950s, the United States took the lead in implementing policy evaluation at home to improve the effectiveness of policy and the efficiency of government administration. By the 1980s, there had been a variety of third-party evaluations in Western countries. The main body of the evaluation was mainly composed of social organizations, expert groups, the general public and the leaders of other departments. At the same time, the evaluation methods were innovated and applied to the evaluation of government performance[5]. However, in China, it is still in the initial stage, there are still many contents to be improved in the application of third-party evaluation mechanism in the field of poverty alleviation. Liu Jianshengetal.[5]studied the theoretical origin of third-party evaluation, and put forward the SMART operation framework and evaluation framework of "third-party evaluation" from the macro and micro level. Qi Zhongshan[6]summarized the restrictive factors of third-party evaluation performance in the field of poverty alleviation through practice, and put forward some measures and suggestions. Meng Zhihuaetal.[7]analyzed and summarized the theoretical origin and mechanism of the third-party evaluation of targeted poverty alleviation, and further put forward the implementation path of optimizing the third-party evaluation of targeted poverty alleviation. In view of the challenges faced by the third-party evaluation of targeted poverty alleviation, Mo Guanghuietal.[8]summarized and put forward the relevant strategies of constructing the long-term mechanism of the third-party evaluation of targeted poverty alleviation. Wang Zaolan[9]summarized the problems of the third-party evaluation of targeted poverty alleviation in Guangxi, and put forward the corresponding measures. Based on the problems existing in the application of third-party evaluation of poverty exit and poverty alleviation in Hebei Province, Qin Weietal.[10]put forward a solution to improvement. Based on the provincial audit and evaluation case of county poverty alleviation performance, Jiang Tianyou[11]put forward the basic strategy to improve the objectivity of third-party evaluation from the perspective of stakeholder interest game. Through review of the literature, it is found that the theoretical research of accurate third-party evaluation of poverty alleviation is constantly deepening and refining. However, on the whole, the research on the third-party evaluation of targeted poverty alleviation in China is still relatively weak, and there are still many problems in the process of practical application. Therefore, it is of great theoretical and practical significance to explore the problems existing in the current third-party evaluation of poverty alleviation, analyze the reasons, and put forward reasonable measures and suggestions.

3 Existing problems and cause analysis

The third-party evaluation of poverty alleviation is that the third-party evaluation agencies (trustees), entrusted by the policy-maker (client), conduct field surveys on the family status and benefits from poverty alleviation policies and so on for the beneficiaries (poor households or evaluation media) of the policy, to assess the effectiveness of policy implementers (local governments or assessed parties) in poverty alleviation and development. According to the roles of the parties involved in the third-party evaluation, the third-party organizations play the same role as auditors, supervisors and referees. Therefore, independence, objectivity, fairness and scientificity are the four basic principles that third parties must adhere to. At the same time, the work attitude, professional level, insight of the third-party evaluators and the interference of the external environment will affect and restrict the third-party evaluation. Based on the existing evaluation practice, the current third-party evaluation of poverty alleviation mainly has the following problems and shortcomings.

3.1 Subjective aspects

3.1.1The evaluation capacity and work attitude of third-party evaluation agencies affect the results of the evaluation. At the current stage, various localities lead economic and social development by getting rid of poverty, and poverty alleviation-related policies involve all areas of social development. Therefore, the third-party evaluation of poverty alleviation is a complex and multi-disciplinary process. In the process of poverty alleviation evaluation, there are not only high requirements for the professionalism of third-party institutions, but also high requirements for the comprehensive quality of the evaluation team. At present, the evaluation team composed of scientific research institutions, universities and so on is limited by its own ability (including professional knowledge, social practice and policy understanding,etc.). Their ability to identify problems in the evaluation is poor, and their sensitivity is not high, which is likely to lead to the omission of key issues. From the perspective of the evaluation attitude of third-party organizations, the task of evaluation is important, not only to be responsible to farmers, but also to recognize the hard work of local governments, so there is a need for an objective and fair attitude to analyze and evaluate. This requires third-party evaluation agencies to adopt a realistic attitude and conscientiously and responsibly assess the actual situation of poverty alleviation work, responsibility, and policy implementation. In the process of practice, the evaluation work is often short of time and arduous. The areas surveyed on site belong to poor areas, and the conditions in all aspects are relatively difficult. With the fatigue and loss of the third-party evaluation team, there may be a positive attitude in the early stage, and a phenomenon of keeping things moving in the late stage, thereby ignoring the further verification of the suspected problems. There has even been a "perfunctory attitude" towards the evaluation. This negative attitude will have a negative impact on the evaluation process, and the "perfunctory attitude" is likely to result in a huge evaluation error[12], which will have a significant impact on the results of the evaluation.

3.1.2The degree of specialization of evaluation agencies and personnel restricts the accuracy of evaluation data. The degree of specialization of third-party evaluation institutions is the premise and guarantee of the authority of the evaluation results and the recognition of the public. The credibility of third-party evaluation agencies must be demonstrated through their professionalism. It is necessary to participate in the specific practice of various evaluations for a long time to build the authority of third-party evaluation institutions. In the process of evaluation, the standardized evaluation index system, evaluation methods and evaluation reports adopted by third-party organizations reflect their professionalism. At present, colleges and universities and scientific research institutions in China have a high degree of professionalism, authority and social recognition in related fields. However, as far as targeted poverty alleviation evaluation is concerned, the experience is still in the accumulation period, and it is difficult to achieve a professional evaluation level by carrying out short-term technical, methodological and policy training for the evaluation team. In addition, most of the household interviewers and data collection personnel are graduate students and college students, who still lack a strong sense of social responsibility and are highly mobile, so it is difficult for them to perfectly deal with the evaluation work. It is easy to have a direct impact on the accuracy of the evaluation results.

3.2 Objective aspects

3.2.1The measurement criteria are difficult to unify, limiting the accuracy of the evaluation data. There is complexity in the evaluation objects (involving poor areas, poor counties, townships and towns, villages and households), and there are great differences in natural conditions and living habits in different areas. For example, there are great regional differences in mountain and plain areas, ethnic minority areas and non-ethnic minority areas. On the other hand, there are many kinds of agricultural products for income measurement, and the market fluctuates greatly. At the same time, the planting and breeding industry is obviously restricted by natural conditions, and the standard of income measurement is difficult to be unified. For example, a poor household has a vegetable shed of 0.067 ha, and its yield is quite different due to the influence of planting technology, planting variety, climate and other factors. At the same time, market prices may change, and even if prices remain unchanged, cost inputs (fertilizers, pesticides) may be different. Therefore, the income measurement results are only estimates, and the measurement error is inevitable.

3.2.2It lacks legal safeguards and evaluation norms need to be strengthened. At present, the state has not yet legislated on third-party evaluation, and the third-party evaluation mechanism has not been given due status protection at the national legal level. Its evaluation rules, evaluation processes and evaluation methods have not yet formed a normal and replicable model, which makes the progress of the system slow[7]. Once separated from the legal protection, the "third-party evaluation" will completely rely on the authority and support given by the evaluation client in the specific operation and evaluation process. In other words, in the process of evaluation, the information asymmetry between the evaluator and the evaluated party still exists. Therefore, it will also be difficult to ensure the accuracy, authenticity and scientific nature of the evaluation results[8]. On the other hand, due to the lack of legal norms, the threshold of third-party institutions and access mechanisms are not sound, leading to the emergence of "chaos" in third-party evaluation. Many of the participants in the evaluation are not from specific areas of expertise in targeted poverty alleviation and lack expertise and results in the field of poverty alleviation and development. Some people who have never been involved in poverty alleviation and development-related work participate in the evaluation, or even act as the organizers or heads of third-party evaluation teams in the fight against poverty. There is no doubt that this has greatly weakened the professionalism and authority of the evaluation results, and it is difficult to ensure that the evaluation results are objective, true and effective.

3.2.3The evaluation process is vulnerable to interference from external forces. The "third-party evaluation" of poverty alleviation is very important to the work performance of local party committees and government departments, especially to measuring the performance of major leaders in poor areas. Therefore, in order to obtain a satisfactory evaluation result, they may use every means to exert varying degrees of interference with the evaluation process[8]. Taking the third-party evaluation of the effectiveness of poverty alleviation work in 2018 as an example, in the process of participating in the third-party evaluation, we have also encountered interference in the evaluation, such as leaders’ advance arrangement, training farmers in advance by local staff, and guiding farmers to answer questions. In the actual evaluation work, when the evaluators conduct surveys and visits to the masses interviewed, once the subjects of the sample survey are reported to the local cadres, some local cadres will often take targeted measures to tackle the inspection. For example, taking a long way to delay time, falsely claiming that farmers are not at home, and even transferring "problem" farmers, and so on, cause external interference to the evaluation work. Some may try to reach the evaluators in a variety of ways, get in touch with them, try to find out the information, and even expect the evaluators to pre-determine the results of the evaluation. There is no doubt that this external interference from all sides will have a serious impact on the entire evaluation process and the results of the evaluation.

3.2.4Stakeholder interest games affect the accuracy of evaluation results. Third-party evaluations have clients who pursue specific interests and the evaluation object directly related to them, as well as a group of potential stakeholders[13]. There are four main stakeholders in the third-party evaluation of poverty alleviation, namely, policy makers, policy implementers, third parties and farmers. In the game structure, the fundamental interests appeal of farmers is to obtain more materials, techniques and other resources, to increase their income as soon as possible and improve the living standards of the family. With the progress of poverty alleviation work, farmers realize that targeted poverty alleviation policy is a major government policy to benefit the people, which will bring them all kinds of benefits. And if they get out of poverty and withdraw from the ranks of poor households, they fear that they will lose the relevant policy benefits. As a result, there is often a reluctance to get out of poverty, or even a scramble to go about telling people how hard up they are. Policy implementers, on the other hand, must lift poor households out of poverty because of their achievements needs. Therefore, there are often cases of man-made interference with the third-party evaluation process, the provision of false data and so on. For third-party organizations, studies have suggested that third-party evaluators will implement "adverse selection" behavior taking into account the investment of time, economy and energy[14]. In addition, in order to maintain the authority of third-party evaluation, third-party organizations may also implement "adverse selection". The specific manifestation is that if the initial evaluation results (such as those commissioned by the provincial level,etc.) are "do not agree to withdraw", whether it is "agree to withdraw" or "do not agree to withdraw" in the subsequent special evaluation and review results of the State Council, it is beneficial and harmless to third-party evaluation agencies[11]. Because, if the subsequent special evaluation and review results of the State Council are also "do not agree to withdraw", it will further illustrate the accuracy of the original provincial third-party evaluation results. If the re-examination result of the special evaluation is "agree to withdraw", it can also be said that the rectification and reform suggestions put forward in the third-party evaluation at the provincial level have played a role in promoting the poor counties to reach the exit standard through the rectification and reform for the problems.

4 Policy recommendations

Third-party evaluation has been paid great attention to in the effectiveness evaluation of poverty alleviation and poverty county exit evaluation in our country. The independence of the third-party evaluation can help to effectively get rid of the shortcomings of the previous government, such as acting as both "athletes and referees", and then strengthen the objectivity and fairness of the evaluation results. However, in the actual evaluation process, there are also constraints from the third-party itself and external factors, which to a certain extent have an impact on the objectivity and fairness of the evaluation. In view of the problems existing in the above-mentioned third-party evaluation of poverty alleviation, it is suggested that improvements should be made from the following six aspects.

4.1 Strengthening the independence of third-party evaluation and establish a correct value orientationThe third party is an evaluation agency independent of government departments and farmers. It should not only be independent of the government departments, maintain the formal independence, but also be independent of the stakeholders, and maintain the independence of interests. This requires that the evaluation organization should first define its own position. As a professional monitoring body for the effectiveness of poverty alleviation, third-party evaluations should remain "neutral" in field investigations and evaluations. It can neither become the "spokesman" of the local government, nor can it become the "caregiver" of the local farmers. Secondly, the members of the evaluation should strengthen their professional ethics. The professional ethics and professional competence of the evaluation team are directly related to the social credibility of third-party evaluation agencies. Therefore, the evaluators must set up "six kinds of consciousness", that is, political consciousness, responsibility consciousness, team consciousness, professionalism consciousness, safety consciousness and confidentiality consciousness, in order to improve their own political accomplishment and professional ethics. At the same time, it is necessary to reduce the impact of the external environment and ensure that the third-party evaluation work is fair and objective.

4.2 Strengthening the building of third-party evaluation teams and improving professional standardsThird-party evaluation institutions must further refine and standardize their own appraisal and evaluation system, training programs, work discipline and other third-party evaluation rules and regulations concerning targeted poverty alleviation. In the assessment of evaluation personnel, it is necessary to further strengthen the examination system, make strict examination standards, and bring the members who meet the evaluation conditions into the construction of the backbone team; implement backbone and hierarchical management, carry out hierarchical division according to the evaluation level and ability of evaluators, and carry out classified, concentrated and strengthened training according to different levels of evaluators. In terms of training methods, it is necessary to standardize the training process and training contents, enhance the ability of evaluators to master operational procedures, core business, and strictly control the quality of investigation through professional training, business guidance, field operation and so on, and strengthen their evaluation skills and communication skills, so as to comprehensively improve the comprehensive quality and evaluation ability of the evaluation subject. In terms of work discipline, both the evaluation team and the evaluators are required to sign the work responsibility and confidentiality commitment, and bear the corresponding legal responsibility, so that they can have rules to follow in the aspects of professional standards, industry norms, service quality and work discipline. It is necessary to constantly improve their professional skills and establish a set of relatively perfect evaluation system, so as to ensure the quality of work and social reputation of targeted third-party assessment agencies for poverty alleviation.

4.3 Accelerating the legalization of third-party evaluationThe legalization of third-party evaluation is an effective measure to ensure its sustainable development. From the national legal level to establish and protect the legitimate rights and interests and subject status of third-party evaluation institutions, it is very necessary to standardize the rights and responsibilities of all parties involved in the evaluation work. We should speed up the establishment of an industry qualification evaluation system for third-party evaluation institutions, and define the business scope, evaluation principles, evaluation types, evaluation procedures, evaluation methods, and relevant requirements for the writing of evaluation reports, and methods of outcome review, and reduce the arbitrariness of the selection of evaluation institutions. It is necessary to improve the third-party evaluation system and industry norms, strengthen the supervision and management of third-party evaluation institutions, and promote the standardization, institutionalization and long-term effectiveness of third-party evaluation of targeted poverty alleviation.

4.4 Enhancing the transparency of information and preventing local interference in evaluationInterference in evaluation is not only reflected in the interference of local governments, but also reflected in farmers’ complaint about impoverishment. It mainly comes from the demand and profit consciousness of the stakeholders in the interest game (local government and farmers). The premise is the information asymmetry between the third-party and the two sides of the game. In a relatively short evaluation period, it is difficult for the third-party to truly know the implementation of local government’s poverty alleviation policies, and they can not clearly grasp the actual situation of rural households, so that the interference phenomenon can be taken advantage of. Therefore, it is necessary to continuously improve the transparency of poverty information and the awareness rate of local government’s poverty alleviation and development. At the same time, we should strengthen the relevant reward and punishment system to reward or punish those who conceal or falsely report information.

4.5 Strictly implementing evaluation standardsThe third-party evaluation of poverty alleviation is a comprehensive evaluation, and at the same time, it is very difficult to determine the content and criteria of the evaluation. In particular, the calculation of indicators such as net income contains a large estimation component. This requires third-party evaluation agencies to strictly implement uniform standards in the evaluation. In the measurement of indicators, we should follow the principles of systematicness, maneuverability, independence and objectivity through professional methods and technical means to refine and standardize them. For suspected problems, strict verification system, methods and procedures must be developed in order to improve the objectivity and accuracy of the evaluation results.

4.6 Enhancing public participation in third-party evaluationsThird-party evaluation is burdened with heavy social responsibility because of its special historical mission. To a certain extent, the third-party evaluation more reflects the social supervision over local government’s poverty alleviation work, and it is the concentrated representative of the public. The results of the evaluation should reflect the views and recognition of the public on the effectiveness of local government’s poverty alleviation work. Third-party evaluation is a form of evaluation of government performance in targeted poverty alleviation, the public should be encouraged and allowed to participate in the whole evaluation process, in order to strengthen the supervision of public opinion[11].