Addition of rituximab is not associated with survival benefit compared with CHOP alone for patients with stage I diffuse large B-cell lymphoma

Bo Jia， Yuankai Shi， Suyi Kang， Sheng Yang， Shaoxuan Hu， Yexiong Li， Mei Dong， Weihu Wang，Jianliang Yang， Liqiang Zhou， Peng Liu， Shengyu Zhou， Yan Qin， Lin Gui， Changgong Zhang， Hua Lin， Shanshan Chen， Lin Wang， Xiaohui He

1Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， Beijing 100021， China；2Department of Radiation Oncology，3Department of Medical Record Library， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing 100021， China

Correspondence to： Yuankai Shi. Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， 17 Panjiayuan Nanli， Chaoyang District， Beijing 100021， China. Email： syuankai@cicams.ac.cn； Xiaohui He. Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， 17 Panjiayuan Nanli，Chaoyang District， Beijing 100021， China. Email： xiaohuih2008@163.com.

Addition of rituximab is not associated with survival benefit compared with CHOP alone for patients with stage I diffuse large B-cell lymphoma

Bo Jia1， Yuankai Shi1， Suyi Kang1， Sheng Yang1， Shaoxuan Hu1， Yexiong Li2， Mei Dong1， Weihu Wang2，Jianliang Yang1， Liqiang Zhou1， Peng Liu1， Shengyu Zhou1， Yan Qin1， Lin Gui1， Changgong Zhang1， Hua Lin3， Shanshan Chen1， Lin Wang1， Xiaohui He1

1Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， Beijing 100021， China；2Department of Radiation Oncology，3Department of Medical Record Library， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing 100021， China

Correspondence to： Yuankai Shi. Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， 17 Panjiayuan Nanli， Chaoyang District， Beijing 100021， China. Email： syuankai@cicams.ac.cn； Xiaohui He. Department of Medical Oncology， Cancer Hospital， Chinese Academy of Medical Sciences & Peking Union Medical College， Beijing Key Laboratory of Clinical Study on Anticancer Molecular Targeted Drugs， 17 Panjiayuan Nanli，Chaoyang District， Beijing 100021， China. Email： xiaohuih2008@163.com.

Background： The role of rituximab in combination with CHOP regimen in patients with stage I diffuse large B-cell lymphoma （DLBCL） remains to be defined. We aimed to compare CHOP plus rituximab（R-CHOP） with CHOP alone and determine the value of radiotherapy in these patients.

Methods： Between 2003 and 2009， 140 untreated patients with stage I DLBCL were retrospectively analyzed in this study.

Results： Seventy-eight patients were treated in R-CHOP group and 62 in CHOP group. Ninety-one patients received additional radiotherapy at the end of chemotherapy. The different treatment groups were well-balanced with respect to baseline characteristics. Complete response （CR） rate was 77% both in R-CHOP and CHOP groups （P=0.945）. After a median follow-up period of 56 months， patients received R-CHOP regimen had similar 5-year progression-free survival （PFS） （76% vs. 85%； log-rank P=0.215）and 5-year overall survival （OS） （90% vs. 96%； log-rank P=0.175） compared with those with CHOP alone. Patients with radiotherapy had significantly increased 5-year PFS compared with those who had chemotherapy alone （86% vs. 71%； log-rank P=0.005）. At multivariate analysis， patients who had CR（P=0.008） and received radiotherapy （P=0.003） were signifcantly associated with superior PFS.

Conclusions： CHOP alone could be as effective as R-CHOP regimen and additional radiotherapy would be necessary for stage I or stage I non-bulky DLBCL patients.

Diffuse large B-cell lymphoma （DLBCL）； stage I； R-CHOP； radiotherapy； prognosis

Introduction

Diffuse large B-cell lymphoma （DLBCL） is the most common non-Hodgkin lymphoma （1，2）. Several randomized phase III clinical trials have demonstrated that the addition of rituximab to the traditional cyclophosphamide， doxorubicin， vincristine and prednisone （CHOP） regimen could signifcantly improve survival of DLBCL patients （3-7）.

However， patients with stage I disease were not fully analyzed in any of these phase III clinical trials and whether rituximab contained chemotherapy could improve survivalof these patients remains unclear. MabThera International Trial （MInT） was a phase III clinical trial demonstrated that CHOP plus rituximab （R-CHOP） was associated with better survival compared with CHOP alone in young DLBCL patients with good-prognosis. A total of 824 patients with stage II-IV， or stage I with bulky disease were enrolled. But the number of stage I patients was limited and the stage I patients without bulky disease were not enrolled （5）. The prospective phase II study showed that the addition of rituximab could improve survival， but this result was only compared with historical study （8）. So the role of rituximab for patients with stage I DLBCL in previous studies was not persuasive. Therefore， paralleled controlled study was needed to assess whether CHOP plus rituximab （R-CHOP） can improve outcome compared with CHOP alone for stage I DLBCL patients.

A randomized phase III study compared three cycles of CHOP followed by involved-feld radiotherapy （IFRT） with eight cycles of CHOP alone which demonstrated that three cycles of CHOP followed by IFRT were associated with survival beneft. Three cycles of CHOP followed by IFRT had been the standard therapy for patients with limited stage DLBCL in the pre-rituximab era （9）. In the rituximab era， phase II clinical trial demonstrated that R-CHOP plus IFRT was associated with better survival outcome compared with historical result of CHOP plus IFRT （8）. The results presented in a retrospective study suggested that using six cycles of full-dose R-CHOP treatment alone without radiotherapy could be effective in patients with limited stage （10）. Therefore， whether the addition of IFRT could improve survival of stage I patients especially those received R-CHOP chemotherapy remains to be defned. This study also evaluated the role of IFRT in these patients.

Materials and methods

Patients

Between 2003 and 2009， 140 untreated patients with stage I DLBCL treated in Cancer Hospital， Academy of Medical Sciences （CAMS） & Peking Union Medical College（PUMC） were included in this study. The eligibility criteria for patients in this study included： （I） histological diagnosis of DLBCL based on the WHO classification； （II） stage I according to the Ann Arbor system （11）； （III） no previous therapy； and （IV） received CHOP or R-CHOP in firstline treatment. We excluded patients from the study if they had： （I） severe infection； （II） congestive heart failure； （III） active hepatitis B virus infection； （IV） defcient hematologic，hepatic， and renal functions； （V） previous treatment with chemotherapy or radiotherapy； or （VI） received chemotherapy other than CHOP or R-CHOP in firstline treatment. All pathological specimens were reviewed in order to confirm the diagnosis in our institution. The immunohistochemical stains for CD10， BCL-6 and MUM1 were evaluated to determine whether the patients belong to germinal center or non-germinal center subgroup. A bulky mass was defned as any mass exceeding 10 cm in diameter in a horizontal plane or a mediastinal mass with a maximum diameter exceeding one-third of the maximum chest diameter as observed on CT scans. The study was approved by the Ethics Committee of the Cancer Hospital， CAMS & PUMC， and written informed consent of all eligible patients was obtained.

Chemotherapy

All the patients were scheduled to undergo primary therapy with R-CHOP ［75 mg/m2epirubicin， 750 mg/m2cyclophosphamide， 1.4 mg/m2（maximum 2.0 mg/body）vincristine， 500 mg/body of prednisolone， and 375 mg/m2rituximab every 21 d per cycle］ or CHOP regimen ［75 mg/m2epirubicin， 750 mg/m2cyclophosphamide， 1.4 mg/m2（maximum 2.0 mg/body） vincristine， and 500 mg/body of prednisolone every 21 days per cycle］.

Response and follow-up criteria

According to International Workshop Criteria， initial response includes complete response （CR）， partial response（PR）， stable disease （SD） and progressive disease （PD）. Overall survival （OS） was calculated from the date of initiation of chemotherapy to the date of either the last follow-up or death. Patients who were alive at the last followup visit were referred to as censored data. Progression-free survival （PFS） was calculated from the date of initiation of chemotherapy to the date of disease progression， deaths or last follow-up. Patients who were not progressed or died at the last follow-up were recognized as censored data.

Statistical analysis

Comparisons of clinical characteristics and CR rate between R-CHOP and CHOP groups were performed using χ2test and Wilcoxon Rank-Sum test. Survival was estimated by the Kaplan-Meier method. Log-rank test was usedfor univariate analysis， and P＜0.05 （two-sided test） was considered statistically significant. The Cox proportional hazards model was used for multivariate analysis. P＜0.05（two-sided test） was considered to indicate statistical significance. Statistical analysis was conducted using the SPSS 19.0 software package （SPSS Inc.， Chicago， IL， USA）.

Table 1 Clinical characteristics of 140 stage I DLBCL patients

Results

Patient characteristics

A total of 140 patients were enrolled in our study with 78 patients in CHOP group and 62 patients in R-CHOP group. The R-CHOP and CHOP groups were wellbalanced with respect to baseline characteristics. Fortytwo （68%） and 49 （63%） patients received radiotherapy in R-CHOP group and CHOP group respectively （P=0.554）. Eight （13%） and 7 （9%） patients had bulky mass in R-CHOP and CHOP groups respectively （P=0.455）. The median number of chemotherapy cycles in both R-CHOP and CHOP groups was 4 （P=0.096） （Table 1）. Patients who received combined chemoradiotherapy and chemotherapy alone were also well-balanced with respect to baseline characteristics. Forty-two patients （46%） in radiotherapy group and 20 patients （41%） in no radiotherapy group received R-CHOP chemotherapy （P=0.544）.

Table 2 Response evaluated at the end of primary treatment

Response

At the end of initial treatment， the CR rate was 77% both in R-CHOP and CHOP groups （P=0.945）. The PR rate was 15% in two groups. The SD rate was 3% in R-CHOP group and 4% in CHOP group. PD occurred in 5% of patients treated with R-CHOP and 4% of patients with CHOP regimen （Table 2）.

Follow-up and survival

After a median follow-up period of 56 months， patients received R-CHOP had similar 5-year PFS （76% vs. 85%；log-rank P=0.215） and 5-year OS （90% vs. 96%； log-rank P=0.175） compared with those with CHOP alone （Figure 1）. Concerning the 125 patients with non-bulky stage I DLBCL alone， there was also no signifcant difference of 5-year PFS（76% vs. 83%； log-rank P=0.301） and OS （91% vs. 96%；log-rank P=0.247） rate in patients treated with R-CHOP and CHOP regimen alone （Figure 2）. For 91 patients who received radiotherapy， there was also no difference of 5-year OS （95% vs. 98%； log-rank P=0.503） and 5-year PFS（86% vs. 86%； log-rank P=0.901） in R-CHOP and CHOP groups. Subgroup analysis showed that both in favorablesubgroup ［i.e.， international prognostic index （IPI） =0，no bulk］ and less-favorable subgroup （i.e.， IPI≥1， or bulk，or both）， patients received R-CHOP regimen were not associated with superior 5-year PFS rate compared with those with CHOP alone. All stage I （86% vs. 71%； log-rank P=0.005） and non-bulky stage I （85% vs. 71%； log-rank P=0.009） DLBCL patients who received radiotherapy had signifcantly increased 5-year PFS compared with those who received chemotherapy alone （Figure 3）. For 62 patients received R-CHOP chemotherapy， radiotherapy could also significantly improve 5-year PFS （86% vs. 55%； log-rank P＜0.001） and 5-year OS （95% vs. 80%； log-rank P=0.016）.

Figure 1 PFS （A） and OS （B） for all stage I DLBCL patients. Patients received R-CHOP chemotherapy had similar 5-year PFS （76% vs. 85%； log-rank P=0.215） and 5-year OS （90% vs. 96%； log-rank P=0.175） compared with those with CHOP chemotherapy alone. Patients who did not relapse or progress at the last follow-up （A） and patients who were alive at the last follow-up （B） were referred to as censored data. PFS， progression-free survival； OS， overall survival； DLBCL， diffuse large B-cell lymphoma.

Figure 2 PFS （A） and OS （B） for stage I non-bulky DLBCL patients （N=135）. There was also no signifcant difference of 5-year PFS （76% vs. 83%； log-rank P=0.301） and OS （91% vs. 96%； log-rank P=0.247） rate in patients treated with R-CHOP chemotherapy and CHOP chemotherapy alone. Patients who did not relapse or progress at the last follow-up （A） and patients who were alive at the last follow-up （B）were referred to as censored data. PFS， progression-free survival； OS， overall survival； DLBCL， diffuse large B-cell lymphoma.

Figure 3 PFS for all stage I （A） and non-bulky stage I （B） DLBCL patients. All stage I （86% vs. 71%； log-rank P=0.005） and non-bulky stage I （85% vs. 71%； log-rank P=0.009） DLBCL patients who received radiotherapy had signifcantly increased 5-year PFS compared with those who had chemotherapy alone. Patients who did not relapse or progress at the last follow-up were recognized as censored data. PFS，progression-free survival； DLBCL， diffuse large B-cell lymphoma.

Prognostic factors

Within the whole cohort， the factors signifcantly associated with a superior PFS were patients who had CR after initial therapy （P=0.003）， did not have B symptoms （P=0.023）and received radiotherapy （P=0.005） at univariate analysis. At multivariate analysis， patients who had CR （P=0.008）and received radiotherapy （P=0.003） were significantly associated with superior PFS （Table 3）.

Discussion

Our study evaluated the role of rituximab and IFRT for stage I or stage I non-bulky patients. In our study， the addition of rituximab was not associated with survival benefit compared with CHOP alone and additional IFRT could significantly improve survival for DLBCL patients with stage I or stage I non-bulky disease.

Several randomized phase III clinical trials have evaluated whether the addition of rituximab to the traditional CHOP regimen could improve survival of DLBCL patients（3-7）. Two randomized controlled， multicenter phase III clinical trials， LNH-98.5 and RICOVER-60 studies， only confirmed the benefit of rituximab for elderly patients（4，6）. MInT compared CHOP and R-CHOP regimens in young DLBCL patients with good prognosis. A total of 824 patients who had no risk factors or one risk factor according to age-adjusted IPI （aaIPI）， stage II-IV disease， or stage I disease with bulk were enrolled （5）. But the number of stage I patients was limited and patients with stage I and non-bulky disease were not included in MInT study or any other phase III clinical trials. The prospective phase II study showed that the addition of rituximab could improve survival， but this result was only compared with historical study （8）. So whether rituximab-contained chemotherapy can improve survival of patients with stage I or stage I non-bulky DLBCL remains to be assessed in paralleled controlled study.

In this study， patients received R-CHOP regimen hadsimilar 5-year PFS and OS compared with those with CHOP alone. These results suggested that CHOP alone might be as effective as R-CHOP for patients with stage I or stage I non-bulky DLBCL patients. The survival beneft of rituximab for DLBCL patients was not as significant as reported in previous studies might be for the reason that all cases included in our study were stage I DLBCL patients for whom chemotherapy alone could have favorable outcome.

Table 3 Univariate and multivariate analysis of impact of various clinical variables on 5-year PFS of patients with stage I DLBCL （N=140）

MInT study showed that a favorable subgroup （i.e.，IPI=0， no bulk） could be defined from a less-favorable subgroup （i.e.， IPI=1 or bulk， or both） （3）. The patients in our study were also divided into favorable subgroup （i.e.，IPI=0， no bulk） and less-favorable subgroup （i.e.， IPI=1 or bulk， or both） in the subgroup analysis. It could be found that in both subgroups the addition of rituximab could not be associated with survival benefts. Other prognostic factors such as BCL-2， subtype of pathology， B symptoms and erythrocyte sedimentation rate （ESR） had been reported in previous studies （12，13）. But owing to the limited sample，we did not incorporate these factors into subgroup analysis.

It has been demonstrated that three cycles of CHOP followed by IFRT was associated with survival beneft （9）. In the prospective phase II SWOG 0014 study， three courses of R-CHOP followed by IFRT in high-risk patients was compared with the results of SWOG 8736 study， and the addition of rituximab could improve survival （8）. The results presented in a retrospective study suggested that using six cycles of full-dose R-CHOP treatment alone without radiotherapy could be beneficial and effective in patients with limited stage DLBCL （10）. But other studies demonstrated that patients with limited stage could benefit from radiotherapy （14）. So whether IFRT after chemotherapy could improve survival compared with chemotherapy alone in stage I or stage I non-bulky DLBCL patients especially patients received R-CHOP regimen remains to be assessed. In this study， 91 patients received additional radiotherapy at the end of chemotherapy. Patients with radiotherapy had signifcantly increased 5-year PFS compared with those who had chemotherapy alone. Concerning the patients with non-bulky stage I DLBCL alone， patients received radiotherapy also significantly increased 5-year PFS. To reduce the interference of rituximab， we analyzed 62 patients received R-CHOP chemotherapy， and radiotherapy could also significantly improve 5-year PFS and 5-year OS. These results suggested that additional radiotherapy is still necessary for stage I or stage I non-bulky DLBCL patients even for patients received standard R-CHOP chemotherapy.

In this study， the factors significantly associated with a superior 5-year PFS were patients who had CR after initial therapy， and received radiotherapy at multivariate analysis. The results above were corresponded with previous studies （3，5，12，15-17）. This study also found that patients with more than 4 cycles of chemotherapy were not associated with survival benefit which suggested that 4 cycles of chemotherapy might be enough for stage I DLBCL patients. The results of multivariate analysis in our study further confrmed that R-CHOP chemotherapy might not improve outcome compared with CHOP alone and radiotherapy could signifcant improve PFS for stage I DLBCL patients.

This is a retrospective study so the bias between R-CHOP and CHOP groups exists. Besides， whether the patients received radiotherapy or not could be a confounding factor when comparing the role of rituximab. To reduce the imbalance， the interclass equilibration were compared which showed that the different treatment groups were well-balanced with respect to baseline characteristics. Concerning 91 patients who received radiotherapy， there was also no difference in 5-year OS and 5-year PFS between R-CHOP and CHOP groups. Multivariate analysis was also performed to reduce the bias. Now the studies assessing the role of rituxiamab for stage I DLBCL patients are limited and a long period would be needed to wait for the results of prospective studies. The sample size in this study is not large enough， but with regard to the results in our study， new problems have been raised which needs further evaluation in prospective， randomized， phase 3 clinical trials.

Conclusions

CHOP alone could be as effective as R-CHOP and additional IFRT would be necessary for stage I or stage I non-bulky DLBCL patients. Patients who had CR and received radiotherapy were significantly associated with longer PFS at multivariate analysis. Even though the survival benefit of rituximab for stage I DLBCL patients was not as significant as expected in our study， we still could not deny the importance of rituximab for DLBCL patients of limited stage. New problems have been raised with regard to the results in our study which need further evaluation in prospective， randomized， phase 3 clinical trials.

Acknowledgements

The authors acknowledge the contributions of Departmentsof Medical Oncology and Radiation Oncology for study collaboration， and Department of Medical Record Library for medical record provision， and thank Chinese Society of Clinical Oncology （CSCO） for partial fnancial support.

Footnote

Conficts of Interest： The authors have no conficts of interest to declare.

1. Armitage JO， Weisenburger DD. New approach to classifying non-Hodgkin's lymphomas： clinical features of the major histologic subtypes. Non-Hodgkin's Lymphoma Classifcation Project. J Clin Oncol 1998；16：2780-95.

2. Chen W， Zheng R， Zeng H， et al. Annual report on status of cancer in China， 2011. Chin J Cancer Res 2015；27：2-12.

3. Pfreundschuh M， Kuhnt E， Trümper L， et al. CHOP-like chemotherapy with or without rituximab in young patients with good-prognosis diffuse large-B-cell lymphoma：6-year results of an open-label randomised study of the MabThera International Trial （MInT） Group. Lancet Oncol 2011；12：1013-22.

4. Pfreundschuh M， Schubert J， Ziepert M， et al. Six versus eight cycles of bi-weekly CHOP-14 with or without rituximab in elderly patients with aggressive CD20+ B-cell lymphomas： a randomised controlled trial （RICOVER-60）. Lancet Oncol 2008；9：105-16.

5. Pfreundschuh M， Trümper L， Osterborg A， et al. CHOP-like chemotherapy plus rituximab versus CHOP-like chemotherapy alone in young patients with goodprognosis diffuse large-B-cell lymphoma： a randomised controlled trial by the MabThera International Trial（MInT） Group. Lancet Oncol 2006；7：379-91.

6. Coiffer B， Thieblemont C， Van Den Neste E， et al. Long-term outcome of patients in the LNH-98.5 trial，the frst randomized study comparing rituximab-CHOP to standard CHOP chemotherapy in DLBCL patients： a study by the Groupe d'Etudes des Lymphomes de l'Adulte. Blood 2010；116：2040-5.

7. Coiffer B. State-of-the-art therapeutics： diffuse large B-cell lymphoma. J Clin Oncol 2005；23：6387-93.

8. Persky DO， Unger JM， Spier CM， et al. Phase II study of rituximab plus three cycles of CHOP and involved-feld radiotherapy for patients with limited-stage aggressive B-cell lymphoma： Southwest Oncology Group study 0014. J Clin Oncol 2008；26：2258-63.

9. Miller TP， Dahlberg S， Cassady JR， et al. Chemotherapy alone compared with chemotherapy plus radiotherapy for localized intermediate- and high-grade non-Hodgkin's lymphoma. N Engl J Med 1998；339：21-6.

10. Tomita N， Takasaki H， Miyashita K， et al. R-CHOP therapy alone in limited stage diffuse large B-cell lymphoma. Br J Haematol 2013；161：383-8.

11. Smithers DW. Summary of papers delivered at the Conference on Staging in Hodgkin's Disease （Ann Arbor）. Cancer Res 1971；31：1869-70.

12. Iqbal J， Meyer PN， Smith LM， et al. BCL2 predicts survival in germinal center B-cell-like diffuse large B-cell lymphoma treated with CHOP-like therapy and rituximab. Clin Cancer Res 2011；17：7785-95.

13. Pfreundschuh M， Ho AD， Cavallin-Stahl E， et al. Prognostic signifcance of maximum tumour （bulk）diameter in young patients with good-prognosis diffuse large-B-cell lymphoma treated with CHOP-like chemotherapy with or without rituximab： an exploratory analysis of the MabThera International Trial Group（MInT） study. Lancet Oncol 2008；9：435-44.

14. Ballonoff A， Rusthoven KE， Schwer A， et al. Outcomes and effect of radiotherapy in patients with stage I or II diffuse large B-cell lymphoma： a surveillance， epidemiology，and end results analysis. Int J Radiat Oncol Biol Phys 2008；72：1465-71.

15. Phan J， Mazloom A， Medeiros LJ， et al. Beneft of consolidative radiation therapy in patients with diffuse large B-cell lymphoma treated with R-CHOP chemotherapy. J Clin Oncol 2010；28：4170-6.

16. Ying Z， Wang X， Song Y， et al. Prognostic value of interim（18）F-FDG PET/CT in diffuse large B-cell lymphoma. Chin J Cancer Res 2013；25：95-101.

17. Jia B， Shi Y， Dong M， et al. Clinical features， survival and prognostic factors of primary testicular diffuse large B-cell lymphoma. Chin J Cancer Res 2014；26：459-65.

Cite this article as： Jia B， Shi Y， Kang S， Yang S， Hu S， Li Y， Dong M， Wang W， Yang J， Zhou L， Liu P， Zhou S， Qin Y，Gui L， Zhang C， Lin H， Chen S， Wang L， He X. Addition of rituximab is not associated with survival benefit compared with CHOP alone for patients with stage I diffuse large B-cell lymphoma. Chin J Cancer Res 2015；27（5）：516-523. doi：10.3978/j.issn.1000-9604.2015.10.04

10.3978/j.issn.1000-9604.2015.10.04

Submitted Jul 13， 2015. Accepted for publication Sep 28， 2015.

View this article at： http：//dx.doi.org/10.3978/j.issn.1000-9604.2015.10.04