Stability in Probability and Inverse Optimal Control of Evolution Systems Driven by Lévy Processes

Khac Duc Do

Abstract—This paper first develops a Lyapunov-type theorem to study global well-posedness (existence and uniqueness of the strong variational solution) and asymptotic stability in probability of nonlinear stochastic evolution systems (SESs) driven by a special class of Lévy processes, which consist of Wiener and compensated Poisson processes. This theorem is then utilized to develop an approach to solve an inverse optimal stabilization problem for SESs driven by Lévy processes. The inverse optimal control design achieves global well-posedness and global asymptotic stability of the closed-loop system, and minimizes a meaningful cost functional that penalizes both states and control. The approach does not require to solve a Hamilton-Jacobi-Bellman equation(HJBE). An optimal stabilization of the evolution of the frequency of a certain genetic character from the population is included to illustrate the theoretical developments.

I. Introduction

WHILE Lévy processes are stochastic processes with stationary and independent increments in general, this paper addresses stochastic evolution systems (SESs) in Hilbert spaces driven by a special class of Lévy processes, which consist of diffusion (standard Brownian motion or Wiener process) and compensated Poisson processes according to the It? decomposition [1] because these SESs can represent many dynamical systems governed by ordinary differential equations(ODEs) and/or partial differential equations (PDEs) in Euclidean spaces subject to complex disturbances. The paper is restricted to the above special class of Lévy processes are considered because the proof of our results on well-posedness,stability, and control design rely on their martingale property.Well-posedness and stability, and control of the solutions of the aforementioned SESs are of great interest.

Excellent results have been obtained for well-posedness and stability analysis of SESs driven by diffuses, see [2]–[10] and references therein. Recently, well-posedness and stability of SESs driven by Lévy processes have received a lot of attention,see for example [11]–[14] and also [15]–[18] for results on SESs in Euclidean space. The monograph [14] can be used for further expositions. The aforementioned works are usually concerned with well-posedness analysis for SESs with respect to sample paths or moments under different conditions on the system functions. Moreover, stability in moment and stability in probability are often considered. Motivated by the fact that Lyapunov’s direct method has been extensively used for deterministic dynamical system [19], this method is also the most powerful tool to investigate stability of SESs. While both stability in moment and stability in probability of SESs driven by diffuses, and stability in moment of SESs driven by Lévy processes are extensively studied in the above works, where different criteria on a Lyapunov functional are imposed.Asymptotic stability in probability of SESs driven by Lévy processes is yet to be addressed.

Although optimal control is desirable because it avoids control effort wasted and increases stability margin [20], [21],classical design of an optimal control for both deterministic and stochastic nonlinear systems is extremely difficult due to formidable task of solving a Hamilton-Jaccobi-Belman equation (HJBE). This difficulty was eased by the inverse approach [21]–[23], where the cost functional properly penalizing both states and control is specified by the control designer. In this approach, a stabilizing control law is first designed and must be of a special form such as the one based on Sontag’s formula [24] or its variant [25]. An inverse optimal control is then designed based on Legendre-Fenchel (LF)transform. While inverse optimal control has been well developed for systems governed by stochastic ordinary differential equations (SODEs) driven by both diffuses and(recently) Lévy processes (e.g., [22], [25], [26]) inverse optimal control of SESs driven by diffuses or Lévy processes has not been considered except for the work in [27] on inverse optimal control of Burger equations and [23], [28] on inverse optimal control of deterministic evolution systems.

The above discussion motives new contributions of this paper.New results on sufficient conditions based on Lyapunov’s direct method are developed for global well-posedness and asymptotic stability in probability of SESs driven by Lévy processes. These conditions are analogous to those for deterministic evolution systems in [23], [28] or ODEs in [19],and hence are easy to be verified. A novel approach is developed to design optimal stabilizers for SESs driven by Lévy processes without having to solve a HJBE. Compared with the existing results on inverse optimal control of ODEs in Euclidean spaces [22], [25], [26], the results in this paper are much more general since SESs cover ODEs in Euclidean spaces but not vice versa, i.e., SESs cover both lumped- and distributedparameter systems while ODEs in Euclidean spaces cover only lumped-parameter systems. In comparison with the existing results on inverse optimal of evolution systems in Hilbert spaces [23], [27], [28], our results are also much more general since the SESs include stochastic disturbances, which include Wiener and compensated Poisson processes. It is noted that when all the stochastic disturbances are removed, our results reduce to those in [23], [28]. The main contributions are detailed as follows.

First, a Lyapunov-type theorem is developed to study wellposedness and global asymptotic stability in probability for SESs driven by Lévy processes. Standard local conditions on the system functions are first imposed for local well-posedness.Then, a growth condition on the infinitesimal generator of a Lyapunov functional is imposed to extend the local wellposedness to global. Criteria for global asymptotic stability in probability are very analogous to those for ODEs in [19].

Second, an inverse optimal stabilization problem is posed and solved by an inverse approach for SESs driven by Lévy processes. This approach proceeds the design of optimal stabilizers in an inverse way in the sense that the Lyapunov functional candidate is first chosen, an inverse pre-optimal control law is next designed in an appropriate form, then the inverse optimal control is computed based on the LF transform.The approach avoids the formidable task of solving a HJBE, and minimizes a cost functional, which is positive definite in both states and control.

Finally, the above theoretical developments are illustrated via a design of an inverse optimal stabilizers for application in the evolution of the frequency of a certain genetic character from the population.

Notations:The symbolsanddenote the infimum and supremum operators, respectively. The symboldenotes the expected value whiledenotes probability.

II. Stability in Probability

This section derives sufficient conditions on global wellposedness and stability in probability of SESs. LetHbe a separable Hilbert space identified with its dualby the Riesz isomorphism. LetVbe a real reflexible Banach space such thatVcontinuously and densely. From the definitions ofH,Vandthe embeddingis continuous and dense. We denote bythe norms inH,V, andV?, respectively; bythe duality product betweenVandV?; and bythe inner product inH. The duality product betweenVands atisfies=Letbe a cylindricalQ-Wiener process defined on a separable Hilbert spaceKwith respect to a probability spaceequipped with a filtrationsuch that it is right continuous andcontains all P-null sets.which is the space of all Hilbert-Schmidt operators fromintoH. The spaceis a separable Hilbert space equipped with the normforwhere t r(·) denotes the trace operator and?denotes the adjoint operation. Letbe the compensated Poisson process associated with the Poisson random processwhich is independent fromand the characteristicThe reader is referred to [1], [29] for details on Poisson processes. We denote bythe space of all càdlàg path fromintoH, and bythe space of all V-valued processwhich arefromtoVandsatisfyConsider the following nonlinear SES on the spaceH:

We give a definition of asymptotic stability in probability of(1), which is an extension of the one in [26] to a SES.

Definition 1:Letbe a positive constant. The strong variational solution of (1) is said to be:

1) globally stable in probability if there exists a classfunction α such that

2) globally asymptotically stable in probability if there exists a classsuch that

The reader is referred to [7], [30] for a concept of strong variational solution of a SES. Letdenote the space of all real-valued nonnegative functionsonHsuch thatis twice (Fréchet) differentiable inXand is differentiable int; bothandare continuous inHandis continuous inWe denote the infinitesimal generatorofalong the variational strong solution of (1) by

Next, we impose several standard local conditions on the system functions of (1) to ensure its local well-posedness(existence and uniqueness of the strong variational solution),see [7, Chapter 4].

Assumption 1:The following local conditions hold for alland a constant ε >0.

H1 (Continuity):The following mappings are continuous:

H2 (Monotonicity):For anywiththere exists a constantsuch that

H3 (Coercivity):The infinitesimal generatorofsatisfies

H4 (Local growth):There exists a constantsuch that for allwith

Global well-posedness and stability in probability of (1) are given in the following theorem.

Theorem 1:Under Assumption 1, suppose that there exist a functionand classα1and α2such thatand a constant

1) Well-posedness:Assume that the infinitesimal generatorsatisfies

2) Asymptotic convergence:Assume that

3) Asymptotic stability:Assume that

Proof:See Appendix I.■

Theorem 1 has applications in control design. In particular,Item 2) on asymptotic convergence has applications in adaptive control design. Item 3) on asymptotic stability has applications in control design for systems, where saturated control input is required.

III. Inverse Optimal Stabilization of SESs

This section considers the nonlinear SES on the spaceH

Definition 2:The problem of inverse optimal stabilization for (14) is solvable if there exist a classwhose derivativeis also a classa symmetric positive definite matrix-valued functiona positive definite functionand a feedbackcontinuous away from the origin, which guarantees global asymptotic stability in probability of the trivial solution of the closed-loop system, and minimizes the cost functional

The above definition indicates that there are three differences between the inverse optimal control and the direct optimal one.First, in the inverse optimal control design, we search forand the optimal feedback controlSecond, the functionsandare not necessary to be quadratic. Third,is resulted after the control design instead of being specified a priori as in direct optimal control. However, as it will be seen later that the inverse optimal control design will not require to solve a HJBE as in the direct optimal control design.Moreover, the inverse optimal control problem penalizes on both the state and control while Lyapunov’s direct method penalizes only the state. The inverse optimal control design is stated in the following theorem.

Theorem 2:Letbe a Lyapunov function candidate satisfying (9),denote the LF transform [31] of the classwhose derivativeis also aclassi.e.,be a symmetric positive definite matrix-valued function. Suppose that the following control law (referred to as an “inverse pre-optimal control” as design of an inverse optimal control is based on this control law, see also Procedure 1)

asymptotically stabilizes the system (14) in probability at the origin with respect toThen, the control law

where

The Lyapunov function candidatesolves the family of HJBEs parameterized by

Proof:See Appendix II. ■

The inverse optimal controlin (17) has an infinite gain margin becauseLet the inverse pre-optimal controluin(16) and the inverse optimal controlin (17) be rewritten as

It is seen that the matricesandare positive definite becauseandare classandis a positive definite matrix. Therefore, the practical steps for design of an inverse optimal control for the system (14) are given in the following procedure.

Procedure 1:

Step 1:Use Lyapunov direct method [19] extended to SESs in Hilbert spaces to search for an inverse pre-optimal control of the form(analogous to aform in a classical sense [21]). Thus, Sontag’s formula [24] or its variant [25] can be extended to design an inverse pre-optimal control. The essential point is to choose an appropriate Lyapunov functionsuch that a pre-optimal control law of the above form can be designed to make the infinitesimal generatornegative definite. Choosing an appropriate Lyapunov function is non-trivial but the system energy can be a good candidate.

Step 2:Proceed either one of two ways. In the first way, we choose a classwhose derivativeis also a classand a positive definite matrixThis task can be done by picking a priori classthen determining the matrixfrom the expression ofDifferent choices ofare possible. These choices will result in different matricesdifferent inverse optimal control law,and hence different desired cost functionals. This flexibility is offered to the designer. In the second way, we specifythen determinefrom the expression ofThe second way is generally much more complicated than the first way but might result in a constant positive definite matrix

Step 3:Obtain an inverse optimal control from the inverse preoptimal control via the LF transform, i.e.,

IV. An Application Example

In this section, we present an application of the theories developed in previous sections to design an inverse optimal stabilizer for the evolution of the frequency of a certain genetic character from the population in an open and bounded domainwith regular boundary. Letbe the frequency at the instanttand the pointThe discrete models in time and space in [32] can be generalized to the following PDE for modelling the aforementioned phenomenon

with a zero boundary condition, whereis the distributed control input andThe termdepends on the migratory movements of the population in O following a normal law of dispersion, and is given by

A. Control Design

We consider the Lyapunov functional candidate

Applying Poincaré’s inequality and using assumptions on the system functions, it can be deduced that

1) Stabilization Control Design: From (30), a stabilization control that cancels all the destabilizing terms can be designed as follows:

which by Theorem 1 guarantees that the strong variational solution of the closed-loop system under the stabilization controlgiven by (31) is globally stable in probability and globally asymptotically stable in probability.

2) Inverse Optimal Control Design:Now, we follow Theorem 2 or Procedure 1 to design an inverse pre-optimal control law. From (30), we choose an inverse pre-optimal control as

which shows that the inverse pre-optimal control (33) globally asymptotically stabilizes the system (26) in probability according to Section II. Now, we determine the classfunctionand the positive definite matrixComparing(33) with (16) shows that the inverse pre-optimal control (33)is of the form (16) with

First choice ofWe choose

which is clearly positive definite for allWith the abovechosen as in (37) andcalculated as in (38), the inverse optimal control is computed according to (17) as follows:

where

where we have used the calculation in obtaining (30). It is clear thatis positive definite becauseandsatisfy(34), andThis means thatis positive definite inX.Moreover,is positive definite inu, see the first equation in (41). However, the weight thatputs onudepends onX(this is becausedepends onX, see (38)).

Second choice ofFor this particular application, we are able to findsuch thatis a positive constant in what follows. We choose

which is substituted into (36) to obtain

B. Simulations

In this section, we perform two simulations: one on the stabilization controlgiven by (31) and one on the inverse optimal controlgiven by (45). The spatial dimension is taken to be 1 in the interval [ 0,10]. The system parameters are taken asThe boundary conditions are chosen asThe initial condition isThe Lévy measuresare taken aswhere λ1=3 and λ2=5,=which is the density function of a lognormal random variable, andTo generate the Lévy process, we first generate anvectorof the standard Brownian motion step, andn-element Poisson process vectorsand[29]. Next, we the f irst-order Euler method to obtain the integrationfor eachdtbecausedoes not have an analytical solution.

Simulation results of the stabilizing controlwith=0.5 andare given in Fig. 1, where the stateis plotted in Fig.1(a) while the controlis plotted in Fig.1(b).Simulation results of the inverse optimal controlwithandare given in Fig. 2, where the stateis plotted in Fig. 2(a) while the controlis plotted in Fig. 2(b). It is seen from Figs. 1 and 2 that both stabilization controland inverse optimal controlare able to drive the system stateto zero. However, the inverse optimal controlyields better performance than the stabilization controldoes in the sense that the stateunderconverges to zero faster than that underwhile the controlis not much larger thanuc. This can be seen from Fig.3, where the cost functional differenceis plotted, thatis always positive. This is consistent with the property of the inverse optimal control that it minimizes the cost functionalIn fact, it can be proved thatalways positive provided thatis a stabilization control by using the same argument as in [33, Remark 3.2].

V. Conclusions

A Lyapunov-type theorem was developed to study global well-posedness and asymptotic stability in probability of SESs driven by Lévy processes. An inverse optimal control approach was proposed to design stabilizers, which are optimal with respect to a cost functional that penalizes both states and control without having to solve a HJBE. A procedure was also presented to apply the theoretical developments to solve an inverse optimal stabilization for practical systems.

Appendix I Proof of Theorem 1

A. Preparing Lemma

Fig. 1. Results under the stabilization optimal control.

Fig. 2. Results under the inverse optimal control.

Fig. 3. Cost functional difference under two controls.

We first prove the following lemma, which requires a global version of conditions in Assumption 1. Then, this lemma is extended to the case where only local conditions stated in Theorem 1 in Section II are required in the proof of Item 1) of Theorem 1.

Lemma A.1: Suppose that Assumption 1 holds for arbitrarily largei.e., all the local conditions (6)–(8) are replaced by global ones. Then, there exists a unique global variational strong solution to (1).

Proof:1)Existence:Proof is based on the approach in [7],[34]. Letbe orthonormal basis inHobtained by the Gramm-Schmidt orthonormalization process from a dense linearly independent subset ofDefineby

Then by the assumption of the Gelfand triplet,is just the orthogonal projection ontoinH, and note thatis a finite dimensional space. Moreover, we have

which under conditions of Lemma A.1 [35], [36] has a unique solutionsatisfying

We now show that this solution is bounded in properspaces. Identifyingwith anprocess and applying the It? formula toyields

where

Applying the Gronwall inequality [37], to (56) yields

we obtain

On the other hand, it is obtained from (51) that

Substituting (62) into (61) and using (58) yield

Thus, we arrive at

Moreover, from (58) we also have

The global version of the condition (6) ensures that the first integral of the last equation in (68) is non-positive. Therefore,we can simplify (68) as follows:

Because of (65) and square integrability ofandS, we can pass (71) to the limit using the Lebesgue dominated convergence theorem (DCT) to obtain

which further yields

Substituting (73) into (70) gives

2) Uniqueness:Supposeandare the solutions of (1)with initial conditionsandApplying the It? formula togives

B. Proof of Item 1) of Theorem 1

Proof:Letbe the boundedness of the initial data, i.e.,For any integerlet us define

which implies that

This together with (82) means thatis the unique strong solution of (1) forWe need to show thatApplying the It? formula towiththe strong solution of (80) gives

where

which also implies that

Taking expectation of (86), using (90) and (10) give

Applying the Gronwall inequality [37] to (91) yields

which further gives

Therefore

C. Proof of Item 2) of Theorem 1

Proof: 1) Global Stability in Probability:Since all conditions of Item 1) hold, there exists a global unique solution to (1). The technique in [26] is used at places in our proof. Applying the It? formula toalong the solutionof (1), and using (11) gives

where

i.e., globally stability in probability of (1) is proved.

2) Asymptotic Convergence:We first decompose the sample space into three mutually exclusive events

In what follows, we will show thatandApplying the It? formula toalong the solutionof(1) and using (11) and (9), then taking the expectancy gives

Then the local conditions of Theorem 1 imply that there exists a constantsuch that

Therefore, it holds that

where

We now compute the upper-bound of

i) Upper-bound ofUsing (105), we can derive the upper-bound ofas

ii) Upper-bound ofTo compute the upper-bound ofwe use the Burkholder-Davis-Gundy (BDG) inequality [3] to obtain

iii) Upper-bound ofWe denote bythe quadratic variation ofThen, by the BDG inequality [38]there exists a positive constantsuch that

Substituting (109), (110), and (113) into (107) gives

Now, by the strong Markov property of solutions of (1), onsee [1, Theorem 6.4.5, p. 387] and [39, Theorem 32, p. 294], the vectorunder the conditional distributionis the same as that of a solution of (1)withtbeing replaced byand the initial data satisfyingSince (11) holds for allthe estimate (117)applies within place ofandin place ofon. Pickinggives, see [26, page 1241]

This follows from the Borel-Canelli lemma that

Since

Thus,

Now, from (9) we have

which implies that

This contradicts (103). Thus, we must haveSince we have proved thatandit holds from(100) thati.e., (12) holds.

D. Proof of Item 3) of Theorem 1

Proof:According to direct consequence of Item 2) of Theorem 1, becausewhereis a classfunction, is positive definite and implies thatThis together with (99) yields

which means that there exists a classsuch that

i.e., global asymptotic stability in probability of (1) is proved. ■

Appendix II Proof of Theorem 2

A. Proof of J(u) Being a Meaningful Cost Functional

Proof:According to Theorem 1, the control (16) globally stabilizes the system (14) in probability at the origin implies that there exists a classsuch that

Combining (19) and the last inequality in (131) yields

B. Proof of u? Stabilizing (14)

Proof:The infinite generatoralong the control (17) is

Using definition of the LF transform [31], i.e.,we have

Substituting the last equality in (134) into (133) yields

C. Proof of Optimality

Proof:Applying the It? formula toalong the solutions of (14) under the stabilizing control (16) yields

Since we have already proved stability of the closed-loop system consisting of either (14) and (16) or (14) and (17), the right hand-side of (136) is a martingale. Thus, taking the expectation of both sides of (136) results in

Note that with the stabilizing control (16), we can writegiven in (19) as follows:

Substituting (138) into (18) and using (137) yields

To further consider the last equality in (139), we need the following identities. First, we usein (16) to obtain

With the last equality in (140)–(142), we can write (139) as

Applying the general Young inequality to (143) gives

where the equality holds if and only if

D. Proof of Continuity

Proof:Sinceandare continuous functions andis a classis continuous awayfrom Ifcontinuity ofcan be proved as follows: