受噪聲影響的復(fù)擬隨機樣本的STL關(guān)鍵定理

受噪聲影響的復(fù)擬隨機樣本的STL關(guān)鍵定理

杜二玲1，李俊華2

(1.中國地質(zhì)大學(xué)長城學(xué)院 基礎(chǔ)課教學(xué)部，河北 保定071000；2.河北大學(xué) 數(shù)學(xué)與信息科學(xué)學(xué)院，河北 保定071002)

摘要：引入了擬概率空間上復(fù)擬隨機樣本受噪聲影響的復(fù)經(jīng)驗風(fēng)險泛函、復(fù)期望風(fēng)險泛函、復(fù)經(jīng)驗風(fēng)險最小化原則以及嚴(yán)格一致性的定義，提出并證明了擬概率空間上復(fù)擬隨機樣本受噪聲影響的學(xué)習(xí)理論關(guān)鍵定理，為系統(tǒng)建立擬概率空間上基于噪聲影響的復(fù)擬隨機樣本的統(tǒng)計學(xué)習(xí)理論奠定了基礎(chǔ).

關(guān)鍵詞：復(fù)擬隨機樣本；噪聲；復(fù)經(jīng)驗風(fēng)險最小化原則；關(guān)鍵定理

DOI：10.3969/j.issn.1000-1565.2015.05.001

中圖分類號：O29;TP181文獻標(biāo)志碼:A

收稿日期：2014-11-30

基金項目：河北省教育廳科研項目(QN20131055)；河北省高等學(xué)校科學(xué)技術(shù)研究項目(Z2013038)

Key theorem of statistical learning theory with complex

quasi-random samples corrupted by noise

DU Erling1, LI Junhua2

(1. Basic Teaching Department, China University of Geosciences Great Wall College, Baoding 071000,

China;2. College of Mathematics and Information Science, Hebei University, Baoding 071002, China)

Abstract:Some new concepts, such as complex empirical risk functional, complex expected risk functional, complex empirical risk minimization principle, and strict consistency built on quasi-probability space and based on complex quasi-random samples corrupted by noise are introduced. The key theorem of learning theory is given and proved on quasi-probability space and based on complex quasi-random samples corrupted by noise. The investigations will help lay essential theoretical foundations for the systematic and comprehensive development of the complex quasi-random samples corrupted by noise.

Key words: complex quasi-random samples; noise; complex empirical risk minimization principle; key theorem

MSC 2010: 28B99

第一作者：杜二玲(1975-)，女，河北安國人，中國地質(zhì)大學(xué)長城學(xué)院講師，主要從事不確定統(tǒng)計學(xué)習(xí)理論.

E-mail:duerling@126.com

統(tǒng)計學(xué)習(xí)理論(statistical learning theory, SLT)是Vapnik等[1-2]在20世紀(jì)60年代末提出,于90年代中期發(fā)展較成熟, 被學(xué)術(shù)界公認(rèn)為較好地處理小樣本的學(xué)習(xí)理論. SLT是建立在概率空間上且所研究的樣本是實隨機樣本. 而概率的可加性條件非常強, 現(xiàn)實中還存在大量的非實隨機樣本. 為此, 一些學(xué)者已經(jīng)開始從事非概率空間上和復(fù)隨機樣本的統(tǒng)計學(xué)習(xí)理論的研究, 得到了一些重要的成果[3-10].其次, SLT所研究的樣本總是事先假定不受外界干擾.這種假定在實際應(yīng)用中往往得不到滿足. 噪聲是影響樣本的因素之一，也是人們考慮比較多的一種, 有學(xué)者開始了樣本受到噪聲影響的統(tǒng)計學(xué)習(xí)理論的研究[11-13].基于上述考慮, 本文在擬概率空間上引入了復(fù)擬隨機樣本受噪聲影響的一些基本定義, 討論了復(fù)擬隨機樣本受噪聲影響的學(xué)習(xí)理論的關(guān)鍵定理, 從而擴大了支持向量機等應(yīng)用性研究領(lǐng)域的理論基礎(chǔ), 拓展了統(tǒng)計學(xué)習(xí)理論的應(yīng)用范圍.

1基本概念

定義1設(shè)Q′(z,α)=Q(z,α)+ξ是考慮噪聲之后的損失函數(shù),ξ1,ξ2,…,ξl與ξ是獨立同分布的, 定義擬概率空間上復(fù)擬隨機樣本受等均值噪聲影響的復(fù)期望風(fēng)險泛函為

R′(α)=E[Q′(z,α)]=E[Q(z,α)+ξ]=E(Q(z,α))+p=R(α)+p.

擬概率空間上復(fù)擬隨機樣本受等均值噪聲影響的復(fù)經(jīng)驗風(fēng)險泛函為

定義2假設(shè)復(fù)期望風(fēng)險泛函的最小值在Q′(z,α0)上取得,復(fù)經(jīng)驗風(fēng)險泛函的最小值在Q′(z,αl)取得.用Q′(z,αl)逼近Q′(z,α0)的值. 這種在擬概率空間上解決最小化復(fù)期望風(fēng)險泛函問題的方法被稱為復(fù)經(jīng)驗風(fēng)險最小化原則(CERM原則).

定義3對于擬概率空間上的復(fù)可測函數(shù)集Q′(z,α),α∈Λ和擬概率μ,如果對于該函數(shù)集的任何非空子集Λ(c)={α:‖R′(α)‖≥c},c∈(-∞,∞)和任意ε>0,收斂性

(1)

成立，則稱復(fù)經(jīng)驗風(fēng)險最小化原則對于擬概率空間上的復(fù)可測函數(shù)集Q′(z,α),α∈Λ和擬概率μ是嚴(yán)格(非平凡)一致的.

定義4對于擬概率空間上的復(fù)可測函數(shù)集Q′(z,α),α∈Λ和擬概率μ,如果對于任意ε>0,

(2)

則稱式(2)為在擬概率空間中的給定復(fù)可測函數(shù)集上復(fù)經(jīng)驗風(fēng)險泛函到復(fù)期望風(fēng)險泛函的一致單邊收斂性.

2主要結(jié)論

1)對于給定的擬概率μ,復(fù)經(jīng)驗風(fēng)險最小化方法對擬概率空間上的復(fù)可測函數(shù)集Q′(z,α),α∈Λ嚴(yán)格一致成立.

定理第1部分得證.

下面證明充分性. 假設(shè)一致單邊收斂性式(2)成立.

(3)

因此B?(B1∪B2).由大數(shù)定理知μ(B1)→0, 由契比雪夫不等式知μ(B2)→0,所以

μ(B)<μ(B1∪B2)→0，

由式(3)得到μ(A1)→0.

(4)

(5)

根據(jù)式(4)，(5)得μ(A)=μ(A1∪A2)→0.定理得證.

參考文獻:

[1]VAPNIK V N. Statistical learning theory[M]. New York: A Wiley-Interscience Publication, 1998.

[2]VAPNIK V N. An overview of statistical learning theory[J]. IEEE Transactions on Networks, 1999, 10(5):.

[3]SUN Xiaojing, WANG Chao, HA Minghu, et al. The key theorem of learning theory based on hybrid variable [Z]. Proceedings of the International Conference on Machine Learning and Cybernetics, Guilin, 2011.

[4]YAN Shujing, HA Minghu, ZHANG Xiankun, et al. The key theorem of learning theory on uncertainty space[J]. Sixth International Symposium Neural Networks, 2009(6): 27-29.

[5]HA Minghu, BAI Yunchao，WANG Peng. The key theorem and the bounds on the rate of uniform convergence of statistical learning theory on a credibility space[J]. Advances in Fuzzy Sets and Systems, 2006, 1(2): 143-172.

[6]張植明, 田景峰, 哈明虎. 基于復(fù)擬隨機樣本的統(tǒng)計學(xué)習(xí)理論的理論基礎(chǔ)[J]. 計算機工程與應(yīng)用, 2008, 44(9): 82-86.

ZHANG Zhiming, TIAN Jingfeng, HA Minghu. Theoretical foundations of statistical learning theory of complex quasi-random samples[J]. Computer Engineering and Application, 2008, 44(9): 82-86.

[7]HA Minghu, PEDRYCZ W, ZHANG Zhiming, et al. The theoretical foundations of statistical learning theory of complex random samples[J]. Far East Journal of Applied Mathematics, 2009, 34(3): 315-336.

[8]HA Minghu, LI Yan, LI Jia, et al. The key theorem and the bounds on the rate of uniform convergence of learning theory on Sugeno measure space[J]. Science in China, Series F Information Sciences, 2006, 49(3): 372-385.

[9]周彩麗, 哈明虎, 鮑俊艷, 等. 基于模糊數(shù)的模糊學(xué)習(xí)理論的關(guān)鍵定理[J]. 河北大學(xué)學(xué)報：自然科學(xué)版, 2008, 28(5): 449-451.

ZHOU Caili, HA Minghu, BAO Junyan, et al. Key theorem of learning theory based on fuzzy number[J]. Journal of Hebei University：Natural Science Edition, 2008, 28(5): 449-451.

[10]哈明虎, 馮志芳, 宋士吉, 等. 擬概率空間上學(xué)習(xí)理論的關(guān)鍵定理和學(xué)習(xí)過程一致收斂速度的界[J]. 計算機學(xué)報, 2008, 31(3): 476-485.

HA Minghu, FENG Zhifang, SONG Shiji, et al. The key theorem and the bounds on the rate of uniform convergence of learning theory on quasi-probability spaces[J]. Chinese Journal of Computers, 2008, 31(3): 476-485.

[11]李俊華, 高林慶, 李海軍. 噪聲影響的泛空間上的學(xué)習(xí)理論關(guān)鍵定理[J]. 計算機工程與應(yīng)用, 2012, 48(27): 49-52.

LI Junhua, GAO Linqing, LI Haijun. Key theorem of learning theory with samples corrupted by noise on pan-space[J]. Computer Engineering and Application, 2012, 48(27): 49-52.

[12]哈明虎, 李俊華, 白云超，等. 樣本受零均值噪聲影響下的學(xué)習(xí)理論的若干理論研究[J]. 自然科學(xué)進展， 2006(12): 1668-1673.

HA Minghu, LI Junhua, BAI Yunchao, et al. Several influence theoretical research of learning theory with samples corrupted by zero-mean noise[J]. Progress in natural science, 2006(12): 1668-1673.

[13]李俊華, 李海軍. 受噪聲影響的復(fù)hybrid樣本的學(xué)習(xí)理論關(guān)鍵定理[J]. 河北大學(xué)學(xué)報:自然科學(xué)版, 2014, 34(1): 14-18.

LI Junhua, LI Haijun. Key theorem of learning theory with complex hybrid samples corrupted by noise[J]. Journal of Hebei University：Natural Science Edition, 2014, 34(1): 14-18.

(責(zé)任編輯：王蘭英)