Comparative research on the prognostic ability of improved early warning and APACHE II evaluation for hospitalized patients in the emergency department*

Yn-Mei Wng,Ting-Ting Wei,Ming Hou,Li Zhng,Aziguli$Mimiti,Ping Li,*

aDepartment of Nursing,Shanghai Gongli Hospital,Shanghai 200135,China

bDepartment of Nursing,The Xinjiang Uygur Autonomous Region People's Hospital,Urumchi 830001,China

Comparative research on the prognostic ability of improved early warning and APACHE II evaluation for hospitalized patients in the emergency department*

Yan-Mei Wanga,Ting-Ting Weia,Ming Houb,Li Zhangb,Aziguli$Maimaitib,Ping Lia,*

aDepartment of Nursing,Shanghai Gongli Hospital,Shanghai 200135,China

bDepartment of Nursing,The Xinjiang Uygur Autonomous Region People's Hospital,Urumchi 830001,China

a r t i c l e i n f o

Article history:

MEWS

APACHE II

Prognosis

Predictive ability

Area under ROC curve

Emergency patients

Objective:To compare the feasibility and applicability of predicting the prognosis of patients using the Early Warning Score(MEWS)system and the Acute Physiology and Chronic Health Evaluation(APACHE II)system in the Emergency Department.

Methods:Using a prospective study method,the APACHE IIand MEWS data for 640 patients hospitalized in the Emergency Internal Medicine Department were collected.The prognoses,two scores to predict the corresponding prediction index of sensitivity,speci fi city and positive predictive value for the prognosis, the negative predictive value and the ROC curve for predicting the prognosis were analyzed for all patients.

Results:In the prediction of the risk of mortality,the MEWS system had a high resolution.The MEWS area under the ROC curve was 0.93.The area under the ROC curve for the APACHE score was 0.79,and the difference was statistically signi fi cant(Z?4.348,P<0.01).

Conclusions:Both the MEWS and APACHE II systems can be used to determine the severity of emergency patients and have a certain predictive value for the patient's mortality risk.However,the MEWS system is simple and quick to operate,making it a useful supplement for APACHE II score.

?2017 Shanxi Medical Periodical Press.Publishing services by Elsevier B.V.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1.Introduction

At present,the most widely used and authoritative evaluation method is the Acute Physiology and Chronic Health Evaluation system(APACHE II).1However,the clinical data acquisition using this system is time-consuming,complicated,and costly;therefore, the lack of medical resources in the region has limited the use of APACHE II.The Modi fi ed Early Warning Score(MEWS)is a new scoring method for patient assessmentand risk strati fi cation and is used in emergency departments or before admission.The system is simple,easy to operate,and was developed in emergency departments and intensive care units(ICUs)of the UK and other developed countries.1,2The APACHE II system helps nurses assess patient conditions according to objective physiological indicators and enhances health care personnel awareness to assess the severity of the patient's condition.3,4Use of the MEWS system by domestic healthcare workers has received increasing attention,but at present,the relevant research on the MEWS system in the Xinjiang Uygur Autonomous Region is sparse.The purpose of this study was to investigate the feasibility and applicability of MEWS and APACHE IIsystems in predicting the prognosis of hospitalized patients in the Emergency Internal Medicine Department in the Xinjiang Uygur Autonomous Region.

2.Objects and methods

2.1.Selection of participants

This study selected patients from January to March in 2014 from the Emergency Department of the Xinjiang Uygur Autonomous Region from a grade three general hospital.The inclusion criteria were as follows:(1)a patient who is hospitalized in the emergency internalmedicine;(2)18 years ofage or older(the APACHE score isdesigned for patients over 18 years ofage);(3)hospitalization time of at least 24 h;and(4)documented APACHE II1score.The exclusion criteria were as follows:cannot continue treatment for other reasons(such as refusing treatmentand referral,etc.).A totalof640 patients were investigated;among these patients,354 cases were male(55.31%),and 286 cases were female(44.69%);the patient age range was 19e93 years(61.34±16.79);283 cases were of Han descent(44.22%);and 357 cases were ethnic minorities(55.78%).A total of 574 patients(89.69%)survived,and 66 died(10.31%).

2.2.Survey method

All of the study subjects who met the inclusion criteria were examined using the APACHE IIand MEWS systems at24 h following admission.Using the patient visit as the observation point,nurses assessed the patient's vital signs,consciousness and MEWS every 4e6 h after the initial assessment.Nurses complete the death MEWS assessment according to the last recorded data regarding the patient's vitalsigns and consciousness.

According to the APACHE II requirements,the worst physiologicaland biochemicalindexes were evaluated and recorded after the patients were admitted to the emergency department at 24 h. Following the patient's prognosis and admission to the hospital (whether the patient was admitted to the Emergency Internal Medicine Departmentofthe ICU),the death and survivalconditions at the time of discharge was the end point of the observation period.

2.3.Survey tool

2.3.1.MEWS score

There are several versions of the MEWS system.This study adopted the MEWS used in Queens Hospital and the Burton Hospitalin the UK.5Respiratory rate,heart rate(atrial fi brillation and the ventricular rate),systolic blood pressure,consciousness and body temperature(axillary temperature)comprise the 5 physiologicalparameters of the assessment and are assigned 0e2 points.Pulse,systolic blood pressure,respiratory frequency and mentalreaction are assigned 0e3 points,and the combined score is added to obtain the MEWS.The lowest possible score is 0 points,the highestpossible score is 14 points,and the higher the score,the more serious the patient's condition.The normalvalues of systolic blood pressure are 90e140 mmHg(1 mmHg? 0.133 kPa),and if the patient blood pressure clearly differed from the normalvalue,calculated according to the basic blood pressure procedure,patient awareness was assessed using the AVPU assessment method.This system includes alert(A),response to voice(V),response to pain(P)and unresponsive(U)scoring.The nurses assessed patients for the fi rst time when before physician evaluation,and the treatment measures were decided according to the patient's condition and the score.According to the related literature,from initial inspection of the patient to the end of the assessment in the Emergency Department,the MEWS score took only 10 min.

2.3.2.APACHE IIscore

The APACHE IIscoring system is a comprehensive assessmentof the age,acute physiology,pathologicalchanges and chronic health status of the patients with severity measured by the doctor.The total APACHE II score consists of 3 parts,with the score ranging from 0 to 71 points.6The higher the score,the more serious the patient's condition and the higher the mortality rate.

According to APACHE II requirements,the worst physiological and biochemical indexes were evaluated and recorded,and the patients were admitted to the Emergency Department at 24 h.An APACHE II score of 20 points is the cut-off point for severity.7,8According to the literature,the patient must be admitted and hospitalized for 24 h in order to obtain allofthe parameters.1Acute physiology and pathology scores,including body temperature, average blood pressure,heart rate,respiratory rate,oxygen saturation,blood pH value or bicarbonate(not recommended for blood gas analysis),serum sodium,serum potassium,serum creatinine, hematocrit and leukocyte count,Glasgow coma score as measured once per hour(score?15)body temperature,blood pressure and heart rate,respiratory rate,oxygen saturation,average blood pressure,and the remaining 11 physiological parameters were obtained 24 h after admission.Each parameter is scored as 0e4 points with a total score of 0e60 points.The age score is responsible for 0e6 points and the chronic health status score for 2e5 points,which indicates the chronic health conditions for organ dysfunction or immune function suppression.9

2.4.Evaluation indicators

2.4.1.Correlation between MEWS and APACHE II score

The correlation analysis was performed between the measured MEWS and APACHE.

2.4.2.Prediction of patient outcome(ROC curve)

According to the area under the ROC curve(the area under the receiver operating characteristic curve,AUROCC,denoted as Az), which indicates the resolution ofthe scoring system,the higher the resolution,the better the effectiveness of the evaluation system.

The area under the ROC curve of the value is Az?0.5e1,and when the area under the ROC curve is less than 0.7,the resolution is relatively low;from 0.7 to 0.9,the resolution is moderate;and more than 0.9,the resolution is high.At present,the ROC curve is regarded as the bestmethod for measuring the quality ofdiagnostic information and decision-making.10

2.4.3.Prognosis predictions(sensitivity and speci fi city)

The percentage of patients with positive MEWS who were positive patients was evaluated along with the percentage of negative patients who were correctly identi fi ed by the MEWS.

2.4.4.Prediction of outcome(positive predictive value,negative predictive value)

The positive predictive value refers to the proportion of truly positive patients(positive)according to the screening test results; the negative predictive value refers to the proportion of truly negative patients according to the screening tests.

2.5.Statistical analysis

Using SPSS17.0 statistical software for statistical analysis,the measurement data are described by mean and the standard deviation(M ±SD).The analysis of variance was used for statistical inference.Categorical data are expressed as a percentage,and the c2testwas used for statisticalinference;the overallaccuracy oftwo score measure with ROC area under the curve,test level a?0.05, P<0.05 for the difference was statistically signi fi cant.

3.Results

3.1.Correlation between MEWS and APACHE IIscores

The overallMEWS score ofthe patients was 3.92±2.58,and the APACHE IIscore was 14.29±6.51.The MEWS and APACHE IIscores were positively correlated,and the correlation coef fi cient was (r)?0.63,P<0.01.

3.2.ROC curve for predicting the prognosis resolution

3.2.1.Area under the ROC curve of the MEWS score and APACHE II score predicted the patient's prognosis

The area under the ROC curve for the MEWS system was Az?0.93(95%CI?0.900e0.969;the best cutoff value for predicting the death of the patient was 7,and the MEWS was used to determine the patient's prognosis;P<0.01).The sensitivity was 76.92%,the speci fi city was 91.70%,the positive predictive value was 51.02%,and the negative predictive value was 97.23%(Fig.1).

The area under the ROC curve in the APACHE score was Az?0.93(95%CI?0.736e0.849);the APACHE score was statistically signi fi cant for judging the prognosis of the patients(P<0.01). The sensitivity was 83.08%,the speci fi city was 62.80%,the positive predictive value was 20.15%,and the negative predictive value was 97.04%(Fig.2).

Fig.1.MEWS score to predict the prognosis of patients using the ROC curve.

Fig.2.APACHE II score to predict the prognosis of patients with the ROC curve.

3.2.2.MEWS and APACHE II scores predicted the area under the ROC curve for the patients

The area under ROC curve for the MEWS system was Az?0.65 (95%CI?0.606e0.691),the best cut-off value to predict a patient was 4 points,and the MEWS was used to determine whether the patient parameters were statistically signi fi cant(P<0.01).The sensitivity was 56.47%,the speci fi city was 70.76%,the positive predictive value was 71.68%,and the negative predictive value was 55.38%.A MEWS score of more than 4 points increased the risk of patient admission to the ICU(Fig.3).

The area under ROC curve for the APACHE score was A?0.68 (95%CI?0.638e0.721),and the use of the APACHE score to determine the patient position on the curve was statistically signi fi cant(P<0.01).The sensitivity was 61.16%,the speci fi city was 66.79%,the positive predictive value was 61.16%,and the negative predictive value was 66.79%(Fig.4).

3.2.3.Comparison of MEWS and APACHE IIscores for the prognosis of patients in the emergency internal medicine

The area under the ROC curve for the MEWS system was greater than that of the APACHE IIscore,as shown in Fig.5.

The difference in the area under the ROC curve between the MEWS and APACHE II scores in predicting the prognosis of the patients was statistically signi fi cant(Z?4.348,P<0.01)(Table 1).

Fig.3.The MEWS system predicts the patient's position on the ROC curve.

Fig.4.APACHE II score used to predict the patient's position on the ROC curve.

Fig.5.MEWS score and APACHE II score in predicting the prognosis of patients with the ROC curve.

Table 1 Comparison of MEWS and II APACHE scores on the ICU and prognosis of patients admitted to the Emergency Internal Medicine Department.

3.2.4.Comparison of MEWS and APACHE II scores in the identi fi cation ability of inpatients in the Emergency Internal Medicine Department

In the identi fi cation of patients who should be admitted to the ICU,the MEWS and APACHE IIscore were used(Fig.6).There was no signi fi cant difference in area under the ROC curve(Z?1.352, P>0.05).

3.2.5.Comparison of the MEWS and APACHE IIscores in predicting the authenticity and bene fi ts of patient outcomes

Fig.6.MEWS and APACHE IIscores to assess the patient's position on the ROC curve.

Two types ofscoring systems were used to assess the speci fi city and the positive predictive value and the prognosis ofpatients,and these methods were signi fi cantly different(c2? 136.207, c2?33.621,P<0.05).The speci fi city and the positive predictive value of the MEWS system was higher than that of the APACHE II score.The predictive value of each system for the sensitivity of negative scores was not different(c2?0.769,c2?0.029,P>0.05) (Table 2).

4.Discussion

4.1.Value of MEWS score in predicting the prognosis ofhospitalized patients in the Emergency Internal Medicine Department

This study showed that the MEWS system predicted the patient's prognosis with high resolution.In 66 cases ofdeath,29 had MEWS of 7 points and above,accounting for 43.94%.In 363 ICU patients,the MEWS were 4 points and above,accounting for 56.47%.Thus,the MEWS system can be used to predict patient prognosis,and MEWS values of 7 points and above have signi ficantly increased risk of death and need to enter ICU monitoring, consistentwith the results ofstudies reported by Chen and Erol.11,12Emergency Department nurses apply MEWS score to assess the severity ofthe patient,and the results should be promptly reported to the doctor.In addition,the frequency of monitoring should be increased to prepare for emergency treatment.

4.2.Value of the APACHE II score in predicting the prognosis of hospitalized patients in the Emergency Internal Medicine Department

The APACHE IIscore is currently the domestic clinical application for criticalillness scoring methods for patients admitted to the ICU.The APACHE IIscore considers more than 20 physiologicaland biochemical parameters measured over 24 h,making the data acquisition time-consuming and laborious.Due to hardware limitations,many biochemical parameters are not conducive to rapid assessment in the case of an emergency,and therefore,the use of this system is limited.This study also showed that the APACHE II score predicts the patient's prognosis with a moderate degree of resolution;in 66 cases of death,34 cases had APACHE IIscores 20 points and above,accounting for 51.52%.The APACHE IIscore can be used to predict the prognosis of patients,but its ability to distinguish among patients is moderate.

4.3.Advantages of clinicalapplication of the MEWS

Through research and related literature reports,the MEWS system can be used to predict the prognosis ofpatients and to help emergency department nurses complete scienti fi c patient assessments.The MEWS uses fi ve life signs to calculate the value of the assignment,does not increase the extra burden of nurses,does not require special equipment,has no site constraints,has a simple operation and calculation,and on average takes 10 min to complete.1Scienti fi c evaluation of patients increases the scienti fi c nature of nursing work and allows doctors and nurses to worktogether to complete the assessment of the patient.13Strong repeatability and timely assessment ofcritically illpatients can aid in their early diagnosis and treatment.

Table 2 MEWS and APACHE II Scores prediction comparison and the effect of the patient's prognosis.

5.Conclusions

This study compared the abilities of the MEWS and APACHE II scores to predict patient prognoses,and the results indicated that the MEWS system can be used to predict the prognosis of patients in the Emergency Internal Medicine Department.The MEWS value can be an effective tool to help emergency nurses scienti fi cally assess patients.

When the patient's MEWS value is seven points or more in an Emergency Internal Medicine Department,and show a poor prognosis,these patients should be further examined.Because the MEWS system is simple and rapid,it can be used as a toolto assess the patient's condition in the Emergency Internal Medicine Department,and itcan be used in clinicaldepartments.However,it is worth noting for researchers that there are multiple versions of the MEWS system,and when choosing the MEWS system,early clinical observation and optimal cut-off values for clinical evaluation must be chosen to ensure the best results.

Conflicts of interest

Allcontributing authors declare no con flicts of interest.

1.Feng HL.Application of early warning score in ICU nursing work.Chin J Nurs. 2011;46:1029e1031(in Chinese).

2.Han C,Ge BL,Zu PJ.Application status of early warning scoring system.Chin Nurs Manag.2012;12:8082(in Chinese).

3.Coombs M,Dillon A.Crossing boundaries,re-de fi ning care:the role of the critical care outreach team.JClin Nurs.2002;11:387e393.

4.McGloin H,Adam SK,Singer M.Unexpected deaths and referrals to intensive care ofpatients on generalwards.Are some cases potentially avoidable?JR Coll Physicians Lond.1999;33:255e259.

5.Paul Fulbrook.Intensive Care Association:A Guide to the Introduction and Promotion Service.London:Intensive Care Association;2002.

6.Knaus WA,Draper EA,Wagner DP,Zimmerman JE.APACHE II:a severity of disease classi fi cation system.Crit Care Med.1985;13:818e829.

7.Jiang XC.Clinical application and signi fi cance of critical illness severity score. Chin Crit Care Med.2000;12:195e197(in Chinese).

8.Hu B,Duan ML,Li A,Liu C.The occurrence of stress ulcer in critically illpatients with APACHE relationship between score and prognosis of patients.Shandong Med.2008;48:6e9(in Chinese).

9.Zheng XZ,Zhu J,Yu Q.The current situation and progress of the clinical application of the strong APACHE II score.Med Rev.2011;17:3297e3299(in Chinese).

10.Sun ZQ.Medical Statistics.Beijing:People's Medical Publishing House;2007: 666(in Chinese).

11.Chen DL.The value of improved early warning scoring system for assessing the severity and prognosis of patients with acute cerebral infarction.Chin J Pract Neuro Dis.2016;18:31e32(in Chinese).

12.Erol A,Yusuf Y,Fatih OO,Gozde S,Bulent CG.Predictive value of the modi fi ed early warning scorer in a Turkish emergency department.Eur J Emerg Med. 2008;15:338e340.

13.Ludikhuize J,de Jonge E,Goossens A.Measuring adherence among nurses one year after training in applying the Modi fi ed Early Warning Score and Situation-Background-Assessment-Recommendation instruments. Resuscitation. 2011;82:1428e1433.

How to cite this article:Wang Y-M,Wei T-T,Hou M,et al. Comparative research on the prognostic ability ofimproved early warning and APACHE II evaluation for hospitalized patients in the emergency department.Chin Nurs Res.2017;4:38e42.http:// dx.doi.org/10.1016/j.cnre.2017.03.009

27 September 2016

*This project is supported by Pudong New Area Health System leadership program(No.PWRd2016-11),and National Natural Science Foundation of China(No. 81360231).

*Corresponding author.

E-mail address:1483746409@qq.com(P.Li).

Peer review under responsibility of Shanxi Medical Periodical Press.

http://dx.doi.org/10.1016/j.cnre.2017.03.009

2095-7718/?2017 ShanxiMedical PeriodicalPress.Publishing services by Elsevier B.V.This is an open access article under the CC BY-NC-ND license(http://creativecommons. org/licenses/by-nc-nd/4.0/).

Received in revised form 3 November 2016

Accepted 23 December 2016

Available online 31 March 2017