Optimal LQG control for discrete time-varying system with multiplicative noise and multiple state delays

Xiao Lu ·Qiyan Zhang·Xiao Liang·Haixia Wang·Chunyang Sheng·Zhiguo Zhang

Abstract This paper is concerned with the optimal linear quadratic Gaussian (LQG) control problem for discrete time-varying system with multiplicative noise and multiple state delays.The main contributions are twofolds.First,in virtue of Pontryagin’s maximum principle,we solve the forward and backward stochastic difference equations (FBSDEs) and show the relationship between the state and the costate.Second,based on the solution to the FBSDEs and the coupled difference Riccati equations,the necessary and sufficient condition for the optimal problem is obtained.Meanwhile,an explicit analytical expression is given for the optimal LQG controller.Numerical examples are shown to illustrate the effectiveness of the proposed algorithm.

Keywords LQG control·Discrete time-varying system·Multiplicative noise·Multiple state delays

1 Introduction

The control problem for time-delay systems have received extensively attention since 1950s because of its wide applications in networked control system,intelligent cruise control system,finance,cable-driven manipulators and so on[1–6].There have been a lot of researches on the optimal control and stabilization problem of time-delay systems in recent years,and many results have been surveyed,which concern with single input/state delay or multiple input/state delays [7–10].For example,Yue et al.[7] proposed a Lyapunov–Krasovskii functional approach to design the delayed feedback controller of uncertain systems with time-varying input delay,by introducing some relaxation matrices and turning parameters.Lee et al.[9] studied the robust H∞control problem for uncertain linear systems with a state-delay.Based on the obtained delay-dependent bounded real lemma,the delay-dependent condition for the existence of a robust controller was presented.Due to the existence of multiple delays,the optimal controller is related to the past variables,which makes the control problem more challenging.

On the other hand,stochastic uncertainties exist in many control processes,and some results have been shown in[11–14].Qi et al.[12] presented the optimal estimation and the optimal output feedback controller of the discrete-time multiplicative noise system with intermittent observations by virtue of coupled Riccati equations.The stabilization condition for this system was developed in the mean square sense.Rami et al.[13] considered the discrete-time stochastic LQ problem subject to state and control-dependent noises.A necessary and sufficient condition for the existence of the optimal control was identified in terms of the solution to the proposed difference Riccati equation.As to meet the actual demand in different areas,the control systems with both stochastic uncertainties and time delay(s) have been thoroughly studied [15–19].Zhang et al.[15] obtained the optimal linear quadratic regulation (LQR) controller for discrete-time system with input delay and multiplicative noise via the Riccati-ZXL difference equation,while the additive noise was not considered in this reference.In [16],Liang et al.took the state-and control-dependent noise,additive noise and input delay into account,and the optimal controller and the suboptimal linear state estimate feedback controller for the linear quadratic Gaussian (LQG) system were both derived,with only single time delay in the input.Besides,when there are multiplicative noise and multiple delays in the input,Li et al.[19] presented the optimal controller and the optimal cost under the necessary and sufficient condition.However,additive noise was not considered.It is obviously shown that the system models described in the above literatures are all discrete time-invariant and input-delay(s) systems.Moreover,the optimal problem involving simultaneously multiplicative noise,additive noise and multiple state delays are not mentioned.In addition,when the additive noise is related to the multiplicative noise,the analysis and synthesis for the control problem remain challenging.

Different from the existing research systems,the system considered in this paper contains simultaneously multiplicative noise,multiple state delays and additive noise,which is more complex than before.It should be emphasized that the additive noise and the multiplicative noise are dependent,and the coefficients in this paper are time-varying.The LQG control for our paper is much more sophisticated and unsolved.The main contributions of this paper are as follows:(1) The relations between the state and the costate in terms of the discrete time-varying LQG problem is given by lots of inductive calculations,which is also the solution to the forward and backward stochastic difference equations(FBSDEs).(2) If and only if a sequence of matrices are all positive definite,the optimal controller and the associated cost function will be obtained via the coupled difference Riccati equations,and the explicit expression of the unique controller is presented,which is obviously more complicated than LQR controller in [19].Our approach is based on the stochastic maximum principle,and the key technique is the solution to the FBSDEs.

The rest of this paper is organized as follows.In Sect.2,the discrete time-varying stochastic LQG control problem is described.In Sect.3,the key tool to the solution is presented,and the necessary and sufficient condition for the optimal LQG control problem is shown.The solutions to the general LQG problem are derived.Numerical examples are shown in Sect.4.Conclusionsare provided in Sect.5.Proofs of the Lemma and the Theorem are described in Appendixes.

Notation?ndenotes then-dimensional real Euclidean space.Ipresents the unit matrix of appropriate dimension.The superscript′denotes the transpose of the matrix.{Ω,F,P,{Fk}k≥0} denotes a complete probability space on which random variableνkandμkare defined such that{Fk}k≥0is the natural filtration generated byνkandμk,i.e.,Fk=σ{ν0,…,νk,μ0,…,μk},augmented by all the P-null sets in F .A symmetricA>0(≥0) means that it is a positive definite (positive semi-definite) matrix.θa,bis the usual Kronecker function,i.e.,θa,b=0 ifa≠b,andθa,b=1 ifa=b.Tr(P) represents the trace of matrixP.

2 Problem formulation

Consider the discrete time-varying stochastic LQG system with state delays and multiplicative scalar noise:

Problem 1Find the unique Fk?1-measurable state feedback controlleruk,k=0,…,N,for system (1) such that the cost function (2) is minimized.

3 Main results

For simplicity,we make the system (1) to be

Following the similar approach in [19],we apply stochastic Pontryagin’s maximum principle [20] to system (3) with the cost function (2) to yield the costate equations:

whereζkis the costate variable withζk=0 fork>N.

For further study,we define the following Riccati coupled equations and make the backwards recursion fork=N,N?1,…,0:

It should be emphasized that the recursion will stop unless assuming thatΩkis invertible.To give the main results of Problem 1,we need to obtain the solution to the FBSDEs(3) and (4)–(6),and then the following lemma is proposed.

Lemma 1Assuming that Ωk are positive definite,i.e.,Ωk >0 ,for k=0,…,N,then the following equation

with the terminal value ΦN+1=0 ,andsatisfy thecoupled equations(7)–(10).

ProofThe proof of Lemma 1 is in Appendix A. ?

Remark 1It is noted that the system model in this paper is discrete time-varying,and contains simultaneously multiplicative noise,additive noise and multiple state delays.Meanwhile,the multiplicative noise is related with the additive noise.Thus,the problem of optimal LQG control is particularly difficult.

Now,we are in the position to present the solution to Problem 1.The results are stated in the following theorem.

Theorem 1Problem1has a uniqueFk?1-measurable uk if and only if Ωk,for k=0,…,N,are positive definite.In this context,the optimal controller uk is calculated by

Remark 3Different from the existing work [19],the diffi-culties caused by the additive noise are mainly as follows.First,this paper considers the optimal LQG control problem with both multiplicative noise and correlated additive noise,which is more challenging than [19].Second,due to the existence of the additive noise,the key technique to this optimal control problem,i.e.,the solution to the FBSDEs (11)is quite more difficult than that of [19].Besides,the optimal controlleruksatisfying (13) and the associated optimal cost(14) are more difficult to obtain,and the expression ofukandare more complicated than [19].

Remark 4For a stochastic discrete-time system with no state delays,i.e.,d=0 in system (3),it is obviously obtained that the coupled Riccati difference equations:

Remark 5From Theorem 1,when the disturbance termμkand the multiplicative noiseνkare independent,i.e.,τ=0,and when the additive noise is Gaussian white noise,i.e.,the Riccati difference equations are as (7)–(10),and the matrices can be rewritten as

As the terminal valueΦN+1=0,it is obviously obtained thatΣkandΦkin (16),(17) always equal to be zero fork=0,…,N.Then,the optimal controller reduces to

In view of obtaining the scalar case of optimal LQG control system (3),we derive the results to the general system with multiple delays and multiplicative noise.

Consider the following case of discrete time-varying system:

where Vk=(νk(1)…νk(f))′is af-dimensional white noise defined on a complete probability {Ω,P,F} .Vksatisfies the varianceγ,i.e.,

Here Fkis the natural filtration generated by Vkandμk,i.e.,Fkis theσ-algebra generated by {V0,…,Vk,μ0,…,μk} .Then,the general case of discrete time-varying LQG control problem is stated as follows.

Problem 2Find the unique Fk?1-measurable state feedback controlleruk,k=0,…,N,for system (18) such that the cost function (2) is minimized.

To solve Problem 2,we derive the definition as

and the coupled difference Riccati equations (7)–(9) extend to

Based on the above definitions,the solution to Problem 2 is derived in the following theorem.

Theorem 2Problem2has a uniquemeasurable uk ifand only if Ωk,for k=0,…,N,are positive definite.In this case,the optimal controller uk is given by

Remark 6It is obviously that the multiplicative noise Vkin the general LQG control system (18) is expanded by multiple dimensions of white noises.The existence of multidimensional white noise has no essential influence on the optimal control problem,and we can treat it as a whole.Then,the approach of Theorem 1 is also applied to the general situation.Thus,combining the mathematical characteristics of Vk,Theorem 2 is derived as the above.

4 Numerical examples

Example 1Consider the scalar case of time-varying LQG control system (3) in Theorem 1,as the additive noiseμkcorrelated withνk.Let the associated parameters be as

It is obviously known thatΩkis positive definite fork=0,1 .Thus,from Theorem 1,there exists a uniqueuk,which is given by

Obviously,Ωk >0 fork=0,…,N,therefore,there exist a optimal controller for Problem 1.In addition,when the initial values are

5 Conclusions

In this paper,the discrete time-varying LQG control problem with both multiplicative noise and multiple state delays has been studied.We obtain the solution to the FBSDEs for the discrete time-varying systems.A necessary and sufficient condition for the existence of a unique optimal controller is proposed.The basis of this approach is the stochastic maximum principle and the key is the relationship between the state and costate.In the future works,we expect that the results in this paper shall pave new ways for networked control system with both state delays and packet dropout.

Appendix A

With the stochastic maximum principle (4)–(6) to LQG control system (3) involving multiple state delays and multiplicative noise,we can obtain fork=N,

whereΦNsatisfied (12) with the terminal values being zero.

Now,we have verified (11) fork=N.Supposing thatζk?1are as (11) for allk≥n+1,we will show that (11) also holds fork=n.Fork=n+1,with (3) and (11),ζncan be calculated as

Plugging (3) and (23) into the above equation for timesd,we can calculateζn?1as follows:

Appendix B

(Necessity) Suppose that there exists the unique Fk?1-measurableukto make the cost function (2) minimized.We will show thatΩk,k=0,…,Nare positive definite by induction and the optimal controller can be designed as (13).Define

wherexN=0 andxN?j=0 forj=0,…,das the uniqueness of the optimal controller is unrelated withxk.

AsJ(N) can be expressed as a quadratic function ofuN,and the performance index must be positive,it can be obviously know thatΩN >0,i.e.,Ωkis positive definite fork=N.AssumingΩk >0 for allk≥n+1,we will prove thatΩn >0 .With (3),(5) and (6),fork≥n+1,we construct that

Similarly to the caseΩN >0 above,we obviously getΩn >0 for allk=0,…,N.This completes the proof of necessity.

(Sufficiency) Suppose thatΩk >0 fork=0,…,Nis ture,we will show the existence of the uniquemeasurableukto minimize (2).Make the definition: