A Stackelberg strategy for continuous-time mixed H2/H∞control problem with time delay

Xiaoqian LI,Wei WANG,Juanjuan XU,Huanshui ZHANG

School of Control Science and Engineering,Shandong University,Jinan Shandong 250061,China

Abstract This paper is concerned with the mixed H2/H∞control with linear continuous time system and time delay.To deal with this,we presents a Stackelberg strategy by treating the control input and the disturbance as leader and follower,respectively.The leader’s control strategy minimizes the cost function which is in H2 norm and the follower’s control strategy maximizes the cost function which is in H∞norm.The main technique of this paper is deal with the noncausal relationship of the variables caused by time delay in the control input by introducing two costates to capture the future information and one state to capture the past information.Through theory analyzing,the Stackelberg strategy exists uniquely.Moreover,with the assistance of the extended state space expression,the explicit expression of the strategy is obtained.

Keywords:Mixed H2/H∞control,continuous-time system,Stackelberg strategy,time delay

1 Introduction

In control theory and engineering,two main performance measures known as H2-norm and H∞-norm have been considered by several researchers[1–5].In most practical cases,a desired performance index(H2norm)is need to be minimized while certain worst-case performance(H∞norm)with respect to the external disturbances is guaranteed[5].The mixed H2/H∞control not only satisfies the H∞cost function,but also minimizes the H2performance index,simultaneously,so it has been treated extensively by researchers in the past decades[6–14].The Nash game approach to H2/H∞controller design led to a breakthrough[9],in which the two pay-off functions associated with a two-player Nash game were used to represent the H2performance and H∞performance separately.However,it is also desirable to develop a systematic design approach that combines significant aspects of both H2and H∞methods.Encouraged by the time domain game approach,two multi objective control design problems were formulated and solved in[11].In[12],the solution of finite horizon H2/H∞control was equivalent to the solvability of four coupled difference matrix-valued equations,however,the four coupled difference equations may be solved recursively.As an active area of research,Stackelberg strategy has been widely used in economics and other control fields due to its significant performance[15–19].In[15],the H2/H∞r(nóng)obust control problem was reformulated as a Stackelberg differential game.Stackelberg game strategy was also applied in stochastic H2/H∞control for systems with state and external disturbance dependentnoise[16],the required strategy set had been obtained by solving cross-coupled algebraic nonlinear matrix equations.

Due to the properties of the mixed H2/H∞control,it has been widely used in parameter perturbation control[20],suboptimal control[21],nonlinear control[6,22]and time delay systems control[23–25].The time delay systems have received much attention in recently years for the widely applications in power electronics and communication technology[26–28].In order to get more efficient and explicit solutions for systems with input or output delay,many efforts had been made by researchers.Sufficient conditions were given for teamoptimal closed-loop Stackelberg strategies in linear differential games with time delay[29].The standard H∞control problem with multiple input-output delays had been studied by Meinsma and Mirkin[30].Necessary and sufficient conditions presented in[31]guaranteed the existence and uniqueness of the open-loop Stackelberg strategies in a linear quadratic differential game with time delay.However,there are difficulties remained when solving mixed H2/H∞problem with time delay.First,the non-causality of the control caused by time delay is ubiquitous and need to be solved.Second,the controller is characterized by cross-coupled Riccati equations which may be solved by tedious recursively.

In this paper,the mixed H2/H∞problem will be formulated as a linear quadratic differential game with input delay,and it will be solved by introducing an open-loop Stackelberg strategy.Considering the hierarchy relationship between leader and follower,the leader declares its strategy first while the follower’s strategy is subjected to leader’s.In another word,the mixed H2/H∞control is concerned with the designment of a pair of controllers.The main difficulty is the noncausal relationship of the variables caused by time delay.In order to deal with this problem,two costates and one state are introduced to capture the future information and the past information,respectively.With the assistance of the decoupled Riccati equations and the extended state space expression,the explicit expressions of the controllers are obtained.

Here is the outline of this paper.Section 2 is the statement of the mixed H2/H∞control problem.The main results of the paper are given in Section 3.In this section,Stackelberg game approach is introduced to deal with the mixed H2/H∞control problem.Concluding remarks are formulated in Section 4.

Throughout this paper,the following notations will be used.Superscript′means the transpose of a matrix,x′means the transpose of x.Rnsignifies the family of ndimensional column vectors.D?1denotes the inverse of real matrix D.I means the unit matrix with compatible dimension.

2 Problem formulation

Considering the dynamic equations of the continuous-time system with input delay

where x(t)∈Rnis the system state;u(t)∈Rsis control input which is also the control of the leader;w(t)∈Rmis disturbance which is also the control of the follower.z2(t)∈ Rrand z∞(t)∈ Rlare control outputs.n,s,m,r and l are finite positive integers.τ is a bounded positive integer reflecting the aftereffect of the leader’s strategy.A,Bi,Cjand Dj,i(i=1,2；j=2,∞)are constant matrices with appropriate dimensions.

The performance indexes are given as follows:

To be specific,combining the dynamic equations presented in the above,the mixed H2/H∞problem being considered here can be described as follows:

For a standard H∞control problem

if γ ＞ γ0(γ0is the norm of transfer function from output z∞to disturbance w when the optimal control law u is invoked),then there exists a unique solution[5].In the mixed H2/H∞control,the optimal strategies u and w are optimal for the given γ.The Stackelberg strategy is introduced to deal with this kind of constrained optimization problem,in which the leader’s strategy u minimizes H2norm while the follower’s strategy w makes the H∞norm meeting the constraint condition.Because of the existed hierarchy in the Stackelberg strategy,the leader selects his strategy first.Under physical background,the disturbance should react in a manner which is the optimal reaction to the declared input strategy.It is the basic idea of a unique Stackelberg game approach when being used in mixed H2/H∞problem.From the definitions,we can see that Stackelberg game and mixed H2/H∞are considering the same problem,but the indexes of them are different,the later is more complex.

Although Stackelberg strategy has been studied from last century,there are many problems need to be resolved,such as open-loop control,for the reason that in the previous results the asymmetric coupled Riccati equations need to be solved.Recently,the research of Stackelberg strategy has made substantial progress,on the basis of the research achievements,we consider the similar but more complex mixed control in this paper.It should be mentioned that the closed-loop control in Stackelberg strategy is with much difficulty and has little substantial progress now.Based on the above analysis,we adopt the open-loop Stackelberg strategy to solve the mixed H2/H∞problem.

Problem formulationFor mixed H2/H∞control problem with time delay,find a pair of strategies u and w,such that the performance index J2is minimized by controller u while the performance index J∞is maximized by controller w.

3 Main results

In this section,we consider the synthesis systems with two types ofperformance indexes.In mixed H2/H∞controlproblem,the controllers share properties ofboth H2control and H∞control.Due to the hierarchy relationship between leader and follower,we should first figure out the controller of the follower,then find the controller of the leader.Accordingly,the mixed problem is reduced to optimize H∞problem and H2problem.An extended sate space formulation contributes to the explicit expression of the optimal strategy.

3.1 Hierarchy optimization

The mixed H2/H∞control problem described in Section 2 is reformulated as a leader-follower game by denoting u as the leader and w as the follower.In this section,we derive the strategies u and w and give necessary conditions when the solution of the problem existed uniquely.

First,we consider the optimization problem for the H∞case

By applying Pontryagin’s maximum principle,we formulate the Hamiltonian function as follows:

where p(t)is the costate with compatible dimension.

The following necessary conditions for this optimization problem are as

Since the final state is free,the transversality condition leads to p(T)=P1(T)x(T).Otherwise,the leader’s control input u(t?τ)≠ 0,t≥ τ is involved in(6),so there exists a nonhomogeneous relationship between the state x(t)and costate p(t)

where η1(t)and P1(t)will be determined in Lemma 1.

Combining(8)and(9),the equilibrium condition(8)can be rewritten as

Assumption 1For given values D∞,2and γ,the matrix λFis negative definite.

From the analysis of standard H∞problem in Section 2,the negative definiteness of matrix λFin equilibrium condition(10)makes the existence and uniqueness of the solution for the follower’s optimal control.The optimization problem for the follower has an unique solution for any fixed u.

Under Assumption 1,the explicit expression of the follower strategy can be presented as follows:

Substituting(11)into(1),one has

Lemma 1The matrix P1(t)satisfies the following Riccati equation:

and the co-state η1(t)can be presented as

ProofWhen at the terminal time T,p(T)=P1(T)×x(T),the terminal value follows.Suppose that(9)holds at time t,η1(t)=p(t)?P1(t)x(t).Substituting(9)into(7),one has

Taking derivative to(9),one has

By comparing the above equations,we can see that the right sides of them are identical.It implies that all correlation coefficients of x(t)are identical,which leads to the following equality:

By transposing,the Riccati equation(13)is obtained.

By using the same method,we obtain the expression of˙η1(t).The proof is completed.□

The H∞case analyzed in the above suggests that the optimality of the follower is in terms of a given γ.Section 2 gives the restriction that γ is greater than the corresponding H∞optimal γ0level.Therefore,optimal controller means that it is optimal for a given γ level,which in turn implies that any mixed optimal controller is a sub-optimal pure H∞controller,but the converse of it may not be true.If a suboptimal controller of pure H∞exists,it is an admissible mixed controller,thus an optimal mixed controller exists.

Now,applying the similar lines,we consider the optimization problem for the H2case

The performance index J2can be rewritten as

Define the Hamiltonian function.

where and α(t)and β(t)are co-states with suitable dimension.

By applying the maximum principle again,the optimal strategy u satisfies

with α(T)=P2(T)x(T),β(0)=0 and t＞ τ.

By introducing a new costate η2(t)to describe the nonhomogeneous relationship between α(t)and x(t),we have

where η2(t)and P2(t)are needed to be determined.

Substituting equation(21)into equation(20),one has

Assumption 2The matrixis positive definite,thenis invertible.

By analysing the standard H2problem in Section 2,the positive definiteness of the matrixin equilibrium condition(20)guarantees that there existing a unique optimal controller for the leader.

From the above consideration,the explicit expression of u(t? τ)is as follows:

Lemma 2The matrix P2(t)in(21)is the solution of the Lyapunov equation which can be obtained with the assistance of terminal value P2(T)

and η2(t)in(21)is given as

where η2(T)=0.

ProofThe proof process is similar to Lemma 1.Therefore,we omit it. □

Note that from(11)and(22),the controllers w(t)and u(t?τ)have been obtained.But the two backward variables η1(t)and η2(t)are involved in the equilibrium conditions which make the derived controllers noncausal.In order to derive a causal strategy,we shall introduce a new state to capture the past information and establish the relationship between the new co-states and the new state.

3.2 Explicit expression for augmented variables

In this section,an extended state space for backward variables and another extended state space for forward variables will be given in order to obtain the explicit formulate of control strategies.A new state will be introduced to capture the past information caused by input delay.

From(12),(14),(19)and(24),we have

where

Lemma 3Let Φ(t)and(Φ?1(t))′are transmission functions of equations(25)and(26)respectively.Equation(25)can be rewritten as

where

The variable ξ(t)is termed as a new state to capture the past information.The dynamic equation of ξ(t)is as follows:

where t≥ τ.

ProofSee Appendix A. □

Considering the the definitions of δ,η and with the aid of(21)and(27),the equilibrium condition(20)can be rewritten as

where ?！?t)=E′+F′(t)G(t? τ,t).

The optimal controller of the leader is as follows:

where t≥ τ.

From Assumption 2,it is known that the control strategy exists and is unique.

Substituting(30)into(28)and(26),we obtain the following equations:

with the initial value δ(0)=[0 x′(0)]′and the terminal value η(T)=0.

Remark1As shown in above,the new costatesη1(t)and η2(t)in(9)and(21)play important roles in deriving the unique control strategies w(t)and u(t?τ).With the costate η1(t),the equilibrium condition(8)has been rewritten as equation(10).With the costate η2(t),the equilibrium condition(20)has been reduced to equation(29)which is formulated by three parts,past information F′(t)ξ(t),current informationD2,1u(t? τ),and future information ?！?t)η(t).

Lemma 4Denoting a symmetric matrix M(t)and it satisfies

where M(t)is a symmetric matrix characterized by the following Riccati equation:

where

The dynamic of ξ(t)is given as

ProofConsidering the zero terminal value of η(T),it yields that η(T)=M(T)ξ(T)where M(T)=0.For the fact that(31)allows to assign an arbitrary initial value to ξ(t),then a unique solution is admitted in the Hamilton-Jacobi-Bellman system(31)and(32).Taking derivative on the both sides of η(t)=M(t)ξ(t),then

Combining(32)and(36),it is easy to get the equation(34).Substituting(33)into(31)yields that

The proof is completed. □

Remark 2As viewed from mathematics,a major problem is how to solve the coupled Riccati equations.Homotopic methods developed by[32]and[33]may be proved useful.Since the decoupled equations(13),(23)and(34)derived in this paper are much simpler than those appearing in the oblique projection method,it is more computation saving to be solved.Therefore,it is an efficient algorithm with special properties in future application.

Remark 3The introduction of the appropriate concepts including δ and η makes a breakthrough for it is helpful to obtain the explicit formulate of the control strategies.δ is defined to establish the relationship between state and costate,and η is defined to capture the effects of future information.

3.3 Main result

Based on the above discussions,we state the main results of the mixed H2/H∞control problem with time delay.

Theorem 1 The mixed H2/H∞optimization problem with the performance indexes in the form of H2and H∞(1)–(5)can be treated as a leader-follower game.In view of the negative definiteness of λFand the positive definiteness ofthe open-loop Stackelberg strategies exist uniquely.

where

and at the same time the dynamic of˙δ(t)is as

the initial values of the above equation isδ(0)=[0 x′(0)]′.

ProofSee Appendix B. □

Remark 4We can see that there are parameters such as F′(t+ τ),Γ′(t+ τ)and M(t+ τ)in the structure of strategies.From the definition of F(t),we will know that it is closely related to P1(t),P2(t)and time-invariant parameters B1and B2.With time going,F(t+τ)is only affected by P1(t+τ)and P2(t+τ).As shown in equations(13)and(23),P1(t+τ)and P2(t+τ)can be calculated out of line.Similar analysis will be adopted when it comes to ?！?t+τ)and M(t+τ).For the reason that we considering a time-invariant system and the outline calculations of P1(t)and P2(t),the parameters F(t+ τ),?！?t+ τ)and M(t+τ)can be calculated out of line.

In order to guarantee the effectiveness of the strategy obtained above,we give the calculation of the optimal performance indexes.

Theorem2The performance indexes are as follows:

ProofSee Appendix C. □

Remark 5Although the Lagrange multiplier technique introduced to prove the necessary conditions in[34]is similar to this paper,the two derivation procedures are completely different,they also result in different sets of forward backward differential equations(FBDEs)and cross-coupled stochastic algebraic equations(CASEs)[34],respectively.The derivation of the set(FBDEs)in this paper is decoupled and is more computation saving to be solved,even though it is an explicit term.

3.4 An example

This section provides a brief presentation of the above algorithm.

Step 1For t=T,equations(13)and(23)will be solved accurately by final conditions P1(T)=0 and P2(T)=0.

Step 2Substitute the obtained P1(t)and P2(t)into an extended state space model(25)and(26),which is formulated under the initial and final conditions δ(0)=[0 x′(0)]′and η(T)=[0 0]′.

Step 3After iterative calculations of each processing step in(25)and(26),a new state ξ is introduced to capture the past information of δ,which is determined by initial state.

Step 4Equation(33)represents a homogeneous relationship between η and ξ,with final condition M(T)=0.

Step 5Repeat the above procedures,the strategies u and w are obtained.

Now,we show the efficiency of the above algorithm.The following example illustrates more details of the above procedures in finding final solutions.Consider a system with the following parameters:

Let T=6 s;the step size is 0.1 s;the time-delay in control input will be chosen as 0.3 s and 0.5 s,respectively.The initial values of states are x1(0)=19.20 and x2(0)= ?23.60.γ=0.7.The terminal conditions P1(T)=0 and P2(T)=0 are set in advance to solve the other values of the Riccati operators P1(t)and P1(t).

Due to the two dimensions of the system,there are two controllers named as u1(t)and u2(t)in control strategy u(t)obtained in the previous section,which are shown in Fig.1.From the curves,it is obviously that the values of u1(t)and u2(t)are tending to 0 in finite time.The trajectories presented in these figures imply that the mixed H2/H∞control problem is asymptotically stable.

Fig.1 The curve of control input u(t).

Fig.2 shows the trajectories ofstates x1(t)and x2(t).In order to show the effect of the time delay,we draw the trajectories when τ =0.3 s and τ =0.5 s(which are presented as d=0.3 s and d=0.5 s in Fig.2),respectively.The curves show that with time going on,the values of them are tending to 0 in finite time.

The performance index J2should be minimized by control input and the value is J2=4.4972×104,it is sum of 2-norm of z2(t)from t=0 s to t=6 s.J∞should be maximized by disturbance and the value is J∞= ?3.6134 × 104,it is sum of‖z∞(t)‖2? γ2‖w(t)‖2from t=0 s to t=6 s.Figs.3 and 4 show the trajectories of H2-norm and H∞-norm,from the curves we can see how they change over time and arrive the final value.

Fig.2 The curve of state x(t).

Fig.3 The curve of H2-norm of‖z2(t)‖2.

Fig.4 The curve of H∞-norm of‖z∞(t)‖2 ? γ2‖w(t)‖2.

Remark 6In order to guarantee the presented recursive algorithm is proceed backward,the priori conditions λF＜ 0 andshould be guaranteed.Otherwise,the iterative should be stopped.It is noted that P1(t)and P2(t)can be computed,provided that λFandare known.

4 Conclusions

In this paper,the mixed H2/H∞control problem described in linear differential time dynamic setting with time delay has been converted into leader-follower game.The linear quadratic performance indexes are presented in H2norm and H∞norm,the leader minimizes the H2norm while the follower guarantees the H∞constraint.In order to deal with the noncausal relationship of the variables,costates and state were introduced to capture the future information and the past information,respectively.An extended state space expression is introduced to establish an homogeneous equilibrium between costates and thus obtain the explicit expression of the strategy.The optimal control strategy shows that the leader announced his strategy firstly and the followerdecided his strategy while considering the leader’s strategy.

Equation(26)can be rewritten as

To simplify the expression of(a1),we will give some definitions of ξ(t+ τ)and G(t,t+ τ),which have been shown in Lemma 3.

Therefore,(a1)can be rewritten as(27).

Note that

We can see that ξ(τ)is determined by the initial data,and it captures the information of x(τ).The introduction of the state ξ(t)is helpful to obtain causal strategy.From equation(27),we have

where t≥τ and˙G(t?τ,t)is obtained by derivation on both sides of˙G(t?τ,t),it follows that

Substituting(26)and(a3)into(a2),˙ξ(t)can be rewritten as equation(28).

The proof is completed. □

Appendix B

ProofHere,we state the dynamic for δ.Combining(26)and(33),then

Combining(25)and(a4),equation(a5)can be obtained.

Therefore,the dynamic of˙δ(t)is obtained.

In view of(30)and(33),we have

then the expression of u(t)is obtained easily and has been presented in(37).

Substituting u(t)and(27)into w(t)yields that

Then the control strategy w(t)is obtained.

Thus,the proof is completed. □

Appendix C

ProofIn order to get a reduced equation of the performance index,we first take derivative to x′(t)p(t)

Taking integration from 0 to T,it yields that

Therefore,it is easy to obtain the presentation of J∞.

Similar to J∞,we calculate the performance index J2,take derivative to x′(t)α(t).

Taking integration from 0 to T,one has

The proof is completed. □