Study on the Establishment of Efficiency Evaluation System of Drug Supervision and Sampling Test

TAN Yi-ping, ZHANG Hao, YANG Yue

(School of Business Administration, Shenyang Pharmaceutical University, Shenyang 110016, China)

Study on the Establishment of Efficiency Evaluation System of Drug Supervision and Sampling Test

TAN Yi-ping, ZHANG Hao, YANG Yue

(School of Business Administration, Shenyang Pharmaceutical University, Shenyang 110016, China)

Objective To set up the evaluation system for evaluating the efficiency of drug supervision and sampling test, to improve drug sampling test level, to make them targeted, applicable, rational and scientific. Methods Literature review was conducted to study related issues that may influence the efficiency of drug supervision and sampling test. Results and Conclusion 10 indicators of the drug sampling process and drug sampling results are put forward, including testing mechanism, testing management, testing capabilities, testing targeted, testing coverage and the hit rate of drug sampling to set up a complete standard score system. The establishment of a scientific and rational evaluation system of drug supervision and sampling test will improve the efficiency of drug sampling test.

drug supervision and sampling test; efficiency; evaluation system

Drug supervision test refers to the quality supervision and inspection made by drug regulatory departments to test the suspicious drugs[1]. In order to protect people’s drug safety, it is drug regulator’s work to get rid of the inferior drugs and purify drug market. Therefore, the government invests hundreds of millions every year to establish the evaluation system for drug supervision test. Blind testing, however, will not only cause serious waste of the government’s testing funds, but also bring unnecessary loss to some pharmacy enterprises. Moreover, it may prevent timely and accurately from identifying counterfeit and inferior drugs and cause safety problems.

Therefore, to avoid blind and invalid sampling and improve drug sampling test to be more scientific, the establishment of evaluation system for drug supervision is necessary. The evaluation should maximize our national testing resources and improve drug supervision and management services[2]. This article studies the establishment of evaluation system for the effectiveness of drug supervision test for post-marketing drugs.

1 Basic meaning of effective evaluation system for drug sampling test

An effective evaluation system for drug sampling test should achieve the highest sampling efficiency with the lowest test cost. By improving the quality of the drug regulatory, we can ultimately ensure the safety of people’s drug use. The drug regulatory department can have reasonable layout of supervising and sampling drugs and pharmacy enterprises to enlarge the coverage of supervision and control the frequency of the supervision. At last, it can find out counterfeit drugs with the lowest cost[3].

Evaluation system should tell the effectiveness of sampling test. Firstly, it must find out the problems in drug sampling, secondly, the problems should be analyzed to get the cause. After that, the related policies and measures are formulated based on the cause. Finally, it implements monitoring and assessment to the policy and measures and constantly increase the efficiency of the drug sampling system[4]. (Figure 1)

2 Principles of establishment of an effective evaluation system for drug sampling test

To prevent counterfeit drugs from flowing into consumers’ hands, an effective evaluation system for drug sampling test must be established on the basis of science and rationality. It should be formulated in compliance with certain principles when indicators are settled and screened.The followings are the specific principles.

Figure 1 The cycle systems for evaluating the effectiveness of drug sampling test

2.1 Comprehensive system

Indicators should be taken into consideration in the aspects of drug production, management and use. Meanwhile, this system should not only assess the efficiency of the drug regulatory agency, but also check its law enforcement. Therefore, an effective evaluation system of drug sampling test is a system composed of a series of single index.

2.2 Scientific principle

Drug sampling test is a high-tech and policyintensive job. The sampling drug must be a representative of other drugs[5]. The scientific drug sampling is the key to improving drug supervision effectiveness. Therefore, the selection of indicators and the establishment of the structure of the index system should be based on objectivity. Only adhering to the principle of objectivity, the information we get will be reliable and the evaluation result will be credible.

2.3 Principle of operation

Drug supervision test is a practical job which has a wide coverage and great influence, easy operation and high timeliness. For making scientific and accurate evaluation, the effective evaluation index should have a clear concept and definition. It can also collect data and information for the qualitative and quantitative analysis and the evaluation method should be simple and operable.

2.4 Principle of timeliness

With the advancement of technology, the supervision methods are constantly updated. The dynamic evaluation index system should be combined with sampling work to adjust based on the actual situation.

3 Establishment of an effective evaluation system for drug sampling test

An effective evaluation system for drug sampling test must cover all aspects of drug supervision, including the evaluation process and the results. Drug sampling process can influence the sampling results and the sampling results will reflect the sampling process. (Figure 2)

Figure 2 The interaction of sampling test process and sampling test results

3.1 Sampling process evaluation

Drug sampling process evaluation involves many aspects, such as the inspection ability of drug testing institutions which include the effective use of personnel, finance and materials, the operational efficiency, inspection cost and management etc. These factors affect the operation of drug sampling and determine whether drug sampling is efficient or not. Sampling process evaluation includes the following four aspects as testing mechanism, testing management, testing cycle and testing capabilities.

3.1.1 Testing mechanism

Drug supervision test is a technical and professional job in that it needs to be planned and organized to carry out drug sampling. Therefore, in accordance with the requirements of sampling, we should formulate operable plan, strengthen the organization and leadership of the agency. Furthermore, we need to pay attention to the personnel training. Meanwhile, we also need to make full use of network information platform to establish drug sampling information database. Strengtheningthe communication and collaboration between drug administrative department and technology department can help to establish joint sampling mechanism.

3.1.2 Testing management

Drug supervision test should be in accordance with Drug Sampling Implementation Plan in different provinces and cities. The delivery, inspection, sampling and reporting material should meet the requirements. Drug sampling should be in compliance with the rules and procedures, for example, the sampling list and sealing need to be filled based on the specification, at the same time and the preservation of sampling drugs also should be kept in accordance with the relevant provisions.

3.1.3 Testing cycle

The drug inspection institutions should complete testing within the stipulated period and the institution that accepts the reinspection shall come to a conclusion in the prescribed time. Therefore, to improve the efficiency of drug sampling, we must evaluate the testing cycle and the testing effectiveness of drug institutes.

3.1.4 Testing capabilities

With the advancement of technology, the means of drug counterfeiting show a high tech. Supervising ability of the inspection agency is being tested; drug sampling test often involves a lot of instruments, equipments, reagents, special chemical substances and standard substances. For example, drug testing vehicles are a system which integrates administrative supervision and the management of information and technology test means. This system can have a fast supervision and rapid screening to achieve an effective drug sampling and inspection. At present, due to the uneven distribution of resources, some agencies can not carry out the performance well. To improve the effectiveness of drug sampling, we should evaluate testing projects and develop their testing capabilities.

3.2 Evaluation of drug sampling result

In order to enhance the effectiveness of drug sampling, the evaluation of drug sampling result includes six aspects, such as the completion of drug sampling task, pertinence, coverage, overall inspection, hit rate and the investigation of counterfeit drugs.

3.2.1 Completion of drug sampling test

The completion rate of sampling test is a good way to calculate the completion of the task. The rate refers to the percentage of the overall sampling test tasks that are assigned to the drug safety agencies in the provinces and cities to be finished. If the completion rate is high, it means the agency makes full use of limited resources to complete the annual sampling tasks assigned by the superior. The quality of the test can be evaluated with the help of other comprehensive indicators. If the completion rate is low, it means inspection efficiency of the agency cannot meet the requirements of testing, and its sampling test ability is doubtful.

3.2.2 Targeted testing

Drug supervision test is a targeted sampling for suspicious drugs that are found in the process of supervision and inspection by drug regulatory department. Drug screening is an important measure to trace suspicious drugs. It can effectively avoid the serious waste of resources. Therefore, the screening rate should be one hundred percent. Meanwhile, the concept of screening hit ratio is created for testing the effect of drug screening. It refers to the sampling number of counterfeit drugs at the end of screening process[7].

3.2.3 Testing coverage

Testing coverage is the number of testing and supervision for drug production, pharmacy enterprises, management and use in the administration area during a period of time[8]. Wider testing coverage means the extensive drug sampling to crackdown counterfeit drugs. It is hard for the counterfeit drugs to escape from sampling.

It is difficult to cover all links of drug distribution due to the limited sampling resources. Therefore, the drug administration department must allocate the resources to sample drugs selectively in the market. One of the indicators for the evaluation system is to have targeted sampling, reasonable allocation and wide sampling coverage.

3.2.4 Full inspection sampling drugs

Each drug inspection contains a lot of testing items and every item should be tested. It is called full testing. And partial testing means one or some of the items is neglected[9].

In order to accurately determine the authenticity and merits of a drug without missing any check items, itis necessary to increase the percentage of total testing. Meanwhile, in order to make good use of the resources, some special cases should not apply for the full inspection. For instance, if one item of the testing is bad, the whole process can stop. When sampling amounts are insufficient, the main items of sampling drugs should be checked[10](identification and content determination, etc). Therefore, as an index of the effective evaluation system, full testing is of great significance.

3.2.5 Drug sampling hit rate

Drug sampling hit rate is reflected in the unqualified rate. It refers to the ratio between batch numbers of unqualified testing drugs with the total sampling numbers.

Drug supervision test is a targeted sampling to inspect suspicious drugs that have the probability of existential risk, hence the unqualified rate of drug supervision test is not suitable to represent the quality of the whole drugs in the market. Nonetheless, if the unqualified drug rate is high, it means counterfeit drugs are targeted. The higher drug sampling hit rate is, the more sampling resources are utilized. Meanwhile, the technical level of preliminary screening is high, the ability to identify counterfeit drugs is big. Therefore, it is of great significance to take drug sampling hit rate as one index of drug sampling test.

3.2.6 Investigation of the counterfeit drugs

The investigation includes two aspects. The first is the rate of prosecution and settlement. The rate of prosecution is the ratio of the investigation case numbers in the unqualified report to the whole registered cases. The settlement rate is the ratio of closing number to the number of prosecution. The second aspect refers to the case value of the counterfeit drugs. It means the ratio of counterfeit drug value rate to the total number of cases[9].

The prosecution and settlement rate is an important index in the effective evaluation system and its significance reflects the solemnity of law enforcement. It can finally show the utilization of drug regulatory resources.

In addition, the costs of two sampling tests are almost the same, the higher cases rate is, and the counterfeit drugs are more targeted. It means that sampling resources are utilized. Therefore, to maximize the efficiency, we must attach importance to the case numbers of counterfeit drugs.

3.3 The evaluation index framework

As to the goal of evaluation system, three levels are set up according to the importance of each index; they are 15, 10 and 5 points respectively. Unqualified drug sampling rate is a very important index because the unqualified products are targeted in the process of drug supervision. 15 points is assigned for this index, while testing cycle index only requires the inspection authority complete the test within the prescribed period and 5 points is assigned for it. The rest indexes are quite important, so 10 points are given. Therefore, the following evaluation framework is established.

3.3.1 Drug sampling test process evaluation (35 scores in total) (Table 1)

Table 1 The index system of evaluating the effectiveness of sampling test process

3.3.2 Evaluation of drug sampling test results (total 65 scores) (Table 2)

Continued Table 1

Table 2 The index system of evaluating the effectiveness of sampling test result

Continued Table 2

Evaluations results are based on comprehensive and relevant indicators derived from drug testing procedures. The total score is 100 points, 80 points or above are excellent, 70 points or above are good, 60 points or above are qualified. Drugs with level of excellent evaluation results can be awarded; drugs with level of disqualified results should be criticized, and they must rectify in the given period. Meanwhile, the effective evaluation results of drug supervision test can be used as an important basis for arranging drug testing program and testing fund next year.

4 Discussions

4.1 The key factors for the evaluation system are the accurate drug testing data and the data source

The accurate drug testing data and data sources is the cornerstone of the evaluation system. Theoretically, these data are too good to be true. Nevertheless, a variety of factors, including subjective or objective factors in drug supervision test can influence the authenticity of these data. Hence, the evaluation work needs to combine with the information platform of drug sampling to guarantee the validity of the evaluation.

4.2 The evaluation system for the drug supervision test is a whole

An effective evaluation system for drug supervision test must include the process and result of drug sampling work in a comprehensive way; it is an inseparable aspect and would affect the effectiveness of drug testing if any aspect is neglected. In the process of drug testing, it attaches importance to professional quality of the staff and the building of rules and regulations. Meanwhile, it is necessary to invest more in the test device and network information platform to ensure testing results. For drug testing results, some aspects of the indicators can not convey everything, for example, if we just consider the unqualified rate of drug testing while neglecting the full inspection rate and the case number rate, the inspection resources are wasted, and it is not good for the enhancement of drug testing efficiency. If the reverse situation occurs, it will go against the principles of objectivity and impartiality in the process of drug supervision test. Therefore, to evaluate one aspect alone is one-sided and unconvincing. These factors are complementary with each other and they need to be evaluated together to reflect the overall the testing performance.

4.3 The establishment of long-term mechanism for an effective evaluation system

An effective evaluation for drug sampling test is not simply to evaluate the test effectiveness, its final goal is to promote the law enforcement of drug sampling test and make drug sampling to be targeted, applicable, rational and scientific. Therefore, we should take the effective evaluation system as the main content of drug sampling assessment. Meanwhile, the result of the system is directly linked with sampling fund. We should build long-term mechanism for the evaluation system and the evaluation for drug sampling test should be institutionalized, regular and persistent in the long-term.

By combining the theory with practice, we can develop and enhance the assessment of programs and projects, and perfect the evaluation index system continually. The limited inspection resources should be maximized to combat the counterfeit drugs.

[1] CHEN Hong, LI Jian-jun. Explore the Improvement of System of Drug Supervising and Sampling Test [J]. Contemporary Medicine, 2008, 14 (22): 22-23.

[2] BI Yan-feng. Drug Testing Problems and Countermeasures [J]. Northern Pharmacy, 2012, 9 (5): 82-83.

[3] JIANG Qin, HU Shan-lian. Conceptual Framework of Public Health System Performance Assessment [J]. Chinese Health Service Management, 2004, 191 (5): 260-262.

[4] GENG Dong-sheng. Discussion of Determining the Number of Samples in Pharmaceutical Sampling [J]. Chinese Pharmaceutical Affairs, 2011, 25 (11): 1104-1105.

[5] LIU Li, MA Shuang-cheng. Strengthen the Work in Drug Fasttesting Technology Research and Application Management [J]. Chinese Pharmaceutical Affairs, 2012, 26 (8): 857-858.

[6] JIANG Hong. A Pragmatic and Rigorous Evaluation System on an Objective, Scientific Evaluation System [J]. Chinese Pharmacy, 2006, 20 (4): 200-201.

[7] QI Jian-liang. The Imminent Establishment of Evaluation System of Drug Sampling Test [N]. Chinese Medicine News, 2005-03-03.

[8] JIANG Hong. Pragmatic and Rigorous Assessment System and Evaluation System of Objectivity [J]. China Pharmacy, 2006, 20 (4): 200-201.

[9] QI Jian-liang. It is High Time to Establish the Evaluation System [N]. Chinese Medical Journal, 2005-03-03.

[10] Shandong Province Drug Sampling and Testing Work Performance Evaluation Methods [S]. The Food and Drug Administration, No. 263, 2011.

Author’s information: YANG Yue, Professor. Major research area: Pharmaceutical affairs law and regulation, drug policy, pharmaceutical intellectual property. Tel: 024-23986372, E-mail: yyue315@163.com