一類微分方程兩點(diǎn)邊值問(wèn)題的神經(jīng)網(wǎng)絡(luò)方法應(yīng)用研究

楊云磊,張?zhí)鞓?lè),侯木舟,羅建書(shū)

(1.中南大學(xué) 數(shù)學(xué)與統(tǒng)計(jì)學(xué)院，湖南 長(zhǎng)沙 410083；2.國(guó)防科技大學(xué) 理學(xué)院，湖南 長(zhǎng)沙 410073)

一直以來(lái)，科研人員對(duì)微分方程數(shù)值解法的研究興趣經(jīng)久不衰.物理學(xué)、生物學(xué)、化學(xué)、經(jīng)濟(jì)學(xué)、人口、資源等領(lǐng)域中的很多問(wèn)題會(huì)通過(guò)建立微分方程模型來(lái)描述，進(jìn)而通過(guò)數(shù)值解進(jìn)一步分析.關(guān)于微分方程兩點(diǎn)邊值問(wèn)題數(shù)值解法的研究主要有打靶法和差分法等[1].打靶法通過(guò)變量代換，將二階方程降為一階方程組，利用方程組的數(shù)值解法來(lái)求解；差分法是微分方程數(shù)值解中常用的方法，對(duì)兩點(diǎn)邊值問(wèn)題同樣適用，通過(guò)離散并利用差商代替導(dǎo)數(shù)，從而將微分方程轉(zhuǎn)化為差分方程求解.關(guān)于差分法的研究成果，如離散方法、利用何種差商代替導(dǎo)數(shù)、差分方程求解技術(shù)等均構(gòu)成不同的求解兩點(diǎn)邊值問(wèn)題的差分方法.

隨著人工智能及計(jì)算機(jī)技術(shù)的發(fā)展，越來(lái)越多的研究人員對(duì)神經(jīng)網(wǎng)絡(luò)方法產(chǎn)生了濃厚的興趣.基于神經(jīng)網(wǎng)絡(luò)模型的諸多優(yōu)點(diǎn)，利用神經(jīng)網(wǎng)絡(luò)的函數(shù)逼近能力，人們研究出了很多采用構(gòu)造神經(jīng)網(wǎng)絡(luò)模型求解微分方程的方法，如：多層感知器神經(jīng)網(wǎng)絡(luò)[2-7]、徑向基函數(shù)神經(jīng)網(wǎng)絡(luò)[8-9]、多尺度徑向基函數(shù)神經(jīng)網(wǎng)絡(luò)[10-14]、細(xì)胞神經(jīng)網(wǎng)絡(luò)[15-16]、有限元神經(jīng)網(wǎng)絡(luò)[17]及小波神經(jīng)網(wǎng)絡(luò)[7]等.關(guān)于微分方程問(wèn)題的神經(jīng)網(wǎng)絡(luò)模型構(gòu)造的研究主要有采用硬極限函數(shù)[18]、Legendre多項(xiàng)式[19]、三角基函數(shù)[20]等不同的激活函數(shù)、多層感知器[2]、混合技術(shù)[21]等構(gòu)造試驗(yàn)解；關(guān)于模型算法的研究主要有基于局部自適應(yīng)的RPROP算法[22]、DE進(jìn)化算法[2]、約束積分法[23]、遺傳算法[24]等.

本文主要介紹求解一類線性微分方程兩點(diǎn)邊值問(wèn)題的神經(jīng)網(wǎng)絡(luò)方法，并進(jìn)行數(shù)值實(shí)驗(yàn)分析網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)及樣本量對(duì)計(jì)算結(jié)果的影響.

1 一類線性微分方程兩點(diǎn)邊值問(wèn)題的神經(jīng)網(wǎng)絡(luò)方法

微分方程兩點(diǎn)邊值問(wèn)題的一般形式[1]為：

(1)

在區(qū)間a≤x≤b上的數(shù)值解法問(wèn)題.如果函數(shù)f是y和y′的線性函數(shù)，此時(shí)稱問(wèn)題(1)為線性微分方程兩點(diǎn)邊值問(wèn)題：

(2)

定理1[25]對(duì)任意連續(xù)函數(shù)y:[0,1]→R，存在函數(shù)ψ:[0,1]→R，基函數(shù)φi:[0,1]→[0,1](i=0,1,…,n)及常數(shù)0<ωi<1,0

(3)

且yA(x)是y(x)的近似解.

定理2[25]對(duì)任意連續(xù)函數(shù)y:[0,1]→R，有界函數(shù)ψ:[0,1]→R，及任意給定的ε>0，存在自然數(shù)N及實(shí)數(shù)ωi,vi,bi(i=1,2,…,n)，使得

(4)

即

|y(x)-yA(x)|<ε.

(5)

定理3假定y(x)是問(wèn)題(2)的解析解，由定理1及定理2可知，存在yA(x)是y(x)的近似解.

證明：由定理1及定理2，很容易構(gòu)造y(x)的近似解yA(x).

假定yA(x)是y(x)的近似解，且存在函數(shù)ψ:[0,1]→R，基函數(shù)φi:[0,1]→[0,1](i=0,1,…,n)及常數(shù)0<ωi<1,0

(6)

且

(7)

(8)

求出式(6)在區(qū)間端點(diǎn)處的值yA(a),yA(b):

(9)

假定yA(x)是問(wèn)題(2)的解，則

(10)

將式(6-9)代入(10)，可得:

(11)

選擇確定的激活函數(shù)φi及函數(shù)ψ，構(gòu)造兩點(diǎn)邊值問(wèn)題(2)的拓?fù)渚W(wǎng)絡(luò)，這時(shí)方程(11)是權(quán)值參數(shù)ωi,vi,bi(i=1,2,…,n)的方程組，通過(guò)優(yōu)化算法求出參數(shù)，就得到y(tǒng)(x)的近似解yA(x)，即問(wèn)題(2)的近似解.

選擇合適的激活函數(shù)φi及函數(shù)ψ，可以將方程組(11)表示成矩陣形式：

Hβ=T

(12)

這里，矩陣H及T與權(quán)值參數(shù)ωi,vi,bi(i=1,2,…,n)無(wú)關(guān)，β只與ωi,vi,bi(i=1,2,…,n)有關(guān).

由矩陣分析理論可知，當(dāng)矩陣H為方陣，且是可逆矩陣時(shí)，式(12)存在唯一解β=H-1T，由得出的權(quán)值向量β，代入式(3)可得yA(x)，此時(shí)yA(x)使得式(10)完全成立，是原問(wèn)題(2)的準(zhǔn)確解；

事實(shí)上，很多情況下，矩陣H不是方陣且不可逆，這時(shí)可以通過(guò)計(jì)算[26]

(13)

定義1n×m階矩陣H?是m×n階矩陣H的廣義逆矩陣，若滿足

HH?H=H,H?HH?=H?,(HH?)T=HH?,(H?H)T=H?H,

由定義1可知，存在H?是H的廣義逆矩陣，使得

(14)

使得式(13)成立.

算法步驟：

1)選擇合適的激活函數(shù)φi及函數(shù)ψ，建立兩點(diǎn)邊值問(wèn)題(2)的神經(jīng)網(wǎng)絡(luò)模型，構(gòu)造近似解yA(x).

2)將yA(x)代入問(wèn)題(2)得到方程(11).

3)將方程(11)表示成矩陣形式(12).

4)計(jì)算H?.

2 數(shù)值實(shí)驗(yàn)

為了方便進(jìn)行數(shù)值實(shí)驗(yàn)，這里近似解的構(gòu)造采用Xu等[20]提出的方法，即

(15)

使用IELM算法計(jì)算權(quán)值參數(shù)an(n=0,1,…,N)，bn(n=1,2,…,N).

例1求解線性邊值問(wèn)題

在區(qū)間0

由公式(15)構(gòu)造該問(wèn)題的近似解及神經(jīng)網(wǎng)絡(luò)模型，網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)為1×(2N+1)×1，假定樣本量為m，采用IELM算法計(jì)算權(quán)值參數(shù).圖1與圖2表示網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)及樣本量對(duì)計(jì)算結(jié)果的影響，圖3表示不同情況下計(jì)算10次，每次計(jì)算所需時(shí)間比較，表1及表2表示網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)及樣本量對(duì)計(jì)算精度及計(jì)算速度的影響.

圖1 N取不同值時(shí)數(shù)值解與解析解

圖2 樣本量m取不同值時(shí)數(shù)值解與精確解

圖3 不同情況下每次計(jì)算時(shí)間比較

NN=5N=10max(abs(yA-y))0.358 80.013 1t/s0.023 3980.015 812

注：t/s為統(tǒng)計(jì)計(jì)算10次的平均時(shí)間.

表2 N=10時(shí),樣本量m與算法性能之間的關(guān)系

注：t/s為統(tǒng)計(jì)計(jì)算10次的平均時(shí)間.

由表1實(shí)驗(yàn)數(shù)據(jù)可知，神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)(即隱層神經(jīng)元數(shù)量)對(duì)計(jì)算精度影響較大，對(duì)計(jì)算速度影響較?。寒?dāng)樣本量固定時(shí)，N=10時(shí)誤差最大值比N=5時(shí)降低了一位精度，平均計(jì)算時(shí)間少了0.007 586；由表2實(shí)驗(yàn)數(shù)據(jù)可知，樣本量對(duì)計(jì)算精度及計(jì)算速度的影響均較大，當(dāng)神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)固定時(shí)，增加樣本量，可以提高計(jì)算精度誤差最大值減小，平均計(jì)算時(shí)間也快速增加.

例2求解線性邊值問(wèn)題

由公式(15)構(gòu)造該問(wèn)題的近似解及神經(jīng)網(wǎng)絡(luò)模型，網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)為1×(2N+1)×1，假定樣本量為m，采用IELM算法計(jì)算權(quán)值參數(shù).計(jì)算結(jié)果如下:圖4與圖5表示網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)及樣本量對(duì)計(jì)算結(jié)果的影響，圖6表示不同情況下計(jì)算10次，每次計(jì)算所需時(shí)間比較，表3及表4表示網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)及樣本量對(duì)計(jì)算精度及計(jì)算速度的影響.

圖4 N取不同值時(shí)數(shù)值解與解析解

圖5 樣本量m取不同值時(shí)數(shù)值解與精確解

圖6 不同情況下每次計(jì)算時(shí)間比較

NN=5N=10max(abs(yA-y))0.031 30.013 2t/s0.025 1860.019 834

注：t/s為統(tǒng)計(jì)計(jì)算10次的平均時(shí)間.

表4 N=10時(shí),樣本量m與算法性能之間的關(guān)系

注：t/s為統(tǒng)計(jì)計(jì)算10次的平均時(shí)間.

由表3實(shí)驗(yàn)數(shù)據(jù)可知，神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)(即隱層神經(jīng)元數(shù)量)對(duì)計(jì)算精度影響較大，對(duì)計(jì)算速度影響較小：當(dāng)樣本量固定時(shí)，N=10時(shí)誤差最大值比N=5時(shí)降低0.018 1，平均計(jì)算時(shí)間少0.005 352.由表4實(shí)驗(yàn)數(shù)據(jù)可知，樣本量對(duì)計(jì)算精度及計(jì)算速度的影響均較大，當(dāng)神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)固定時(shí)，增加樣本量，可以提高計(jì)算精度誤差最大值減小，平均計(jì)算時(shí)間也快速增加.

3 結(jié)論

通過(guò)介紹求解一類線性微分方程兩點(diǎn)邊值問(wèn)題的神經(jīng)網(wǎng)絡(luò)方法，建立該問(wèn)題的神經(jīng)網(wǎng)絡(luò)模型.當(dāng)近似解滿足微分方程及邊值條件時(shí),根據(jù)矩陣分析理論，可以采用基于廣義逆矩陣的IELM算法求解近似解中的權(quán)值，為一類線性微分方程兩點(diǎn)邊值問(wèn)題的數(shù)值解提供一種新的方法.數(shù)值實(shí)驗(yàn)結(jié)果表明，采用IELM算法求解神經(jīng)網(wǎng)絡(luò)模型中的權(quán)值參數(shù)，計(jì)算精度及速度只與網(wǎng)絡(luò)拓?fù)浣Y(jié)構(gòu)中隱層神經(jīng)元數(shù)量及樣本量有關(guān)，且只需少量的隱層神經(jīng)元，通過(guò)增加樣本量即可得到相應(yīng)的計(jì)算精度.本文的實(shí)驗(yàn)結(jié)果可以為該方法在應(yīng)用時(shí)隱層神經(jīng)元及樣本量的選擇提供參考，未來(lái)可以研究該算法的推廣能力，如隱層神經(jīng)元及樣本量的最優(yōu)取值區(qū)間等.

參考文獻(xiàn)：

[1] 韓旭里.數(shù)值分析#:#Numerical Analysis[M].北京：高等教育出版社,2011.

[2] LAGARIS I E,LIKAS A C.Artificial neural networks for solving ordinary and partial differential equations[J].IEEE Trans Neural Netw,1998,9(5):987-1000.

[3] LI J Y,LUO S W,QI Y J,et al.Numerical solution of differential equations by radial basis function neural networks[J].Proc Int Jt Conf Neural Netw,2002,1:773-777.

[4] NAM M D,THANH T C.Numerical solution of differential equations using multiquadric radial basis function networks[J].Neural Netw,2001,14:185-199.

[5] CHUA L O,YANG L.Cellular neural networks:theory[J].IEEE Trans Circuits Syst,1988,35:1257-1272.

[6] RAMUHALLI P,UDPA L,UDPA S S.Finite element neural networks for solving differential equations[J].IEEE Trans Neural Netw,2005,16(6):1381-1392.

[7] LI X,OUYANG J,LI Q,et al.Integration wavelet neural network for steady convection dominated diffusion problem[J].IEEE Computer Society,2010,2(2):109-112.

[8] MOODY J,DARKEN C J.Fast learning in networks of locally tuned processing units[J].Neural Comput,2002,35(10):1257-1272.

[9] ESPOSITO A,MARINARO M,ORICCHIO D,et al.Approximation of continuous and discontinuous mappings by a growing neural RBF-based algorithm[J].Neural Netw,2000,13:651-665.

[10] PARK J,SANDBERG I W.Approximation and radial basis function networks[J].Neural Comput,1993,5(2):305-316.

[11] HAYKIN S.Neural networks:a comprehensive foundation[M].N.J.:Prentic Hall,1999.

[12] FRANKE R.Scattered data interpolation:tests of some methods[J].Math Comput,1982,38(157):181-200.

[13] NAM M D,THANH T C.Approximation of function and its derivatives using radial basis function networks[J].Neural Netw,2001,14:185-199.

[14] NAM M D,THANH T C.Numerical solution of differential equations using multiquadric radial basis function networks[J].Neural Netw,2001,14:185-199.

[15] MANGANARO G,ARENA P,FORTUNA P.Cellular neural networks:chaos,complexity and VLSI processing[M].Berlin:Springer,1999.

[16] CHEDHOU J C,KYAMAKYA K,LATIF M A,et al.Solving stiff ordinary and partial differential equations using analog computing based on cellular neural networks[J].ISAST Trans Comput Intell Syst,2009,1(2):38-46.

[17] TAKEUCHI J,KOSUGI Y.Neural network representation of the finite element method[J].Neural Netw,1994,7(2):389-395.

[18] MEADE JR A J,FERNANDEZ A A.The numerical solution of linear ordinary differential equations by feedforward neural networks[J].Math Comput Model,1994,19:l-25.

[19] MALL S,CHAKRAVERTY S.Application of legendre neural network for solving ordinary differential equations[J].Applied Computing,2016,43:347-356.

[20] XU L Y,HUI W,ZENG Z Z.The Algorithm of Neural Networks on the Initial Value Problems in Ordinary Differential Equations[C].2007 2nd Conference on Industrial Electronics and Applications,2007:813-816.

[21] TSOULOS I G,GAVRILIS D,GLAVAS E.Solving differential equations with constructed neural networks[J].Neurocomputing,2009,72:2385-2391.

[22] REIDMILLER M,BRAUN H.A direct adaptive method for faster back propagation learning:the RPROP algorithm[J].IEEE International Conference on Neural Networks,1993,1(A8):586-591.

[23] RUDD K,FERRARI S.A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks[J].Neurocomputing,2015,155(C):277-285.

[24] AARTS L P,VEER P V.Neural network method for partial differential equations[J].Neural Process Lett,2001,14:261-271.

[25] BEIDOKHTI R S,MALEK A.Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques[J].Franklin Inst,2009,346:898-913.

[26] HUANG G B,ZHU Q Y,SIEW C K.Extreme learning machine:theory and applications[J].Neurocomputing,2006,70(1/2/3):489-501.