Mean Field Games with Common Noises and Conditional Distribution Dependent FBSDEs?

Ziyu HUANG Shanjian TANG

Abstract In this paper, the authors consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way. They assume that the cost function satisfies a convexity and a weak monotonicity property. They use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent forward-backward stochastic differential equation (FBSDE for short). They prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state on the conditional distribution is sufficiently small, or when the convexity parameter of the running cost on the control is sufficiently large. Two different methods are developed. The first method is based on a continuation of the coefficients, which is developed for FBSDE by [Hu, Y. and Peng, S., Solution of forward-backward stochastic differential equations, Probab. Theory Rel., 103(2), 1995,273–283]. They apply the method to conditional distribution dependent FBSDE. The second method is to show the existence result on a small time interval by Banach fixed point theorem and then extend the local solution to the whole time interval.

Keywords Mean field games,Common noises,FBSDEs,Stochastic maximum principle

1 Introduction

Mean field games (MFGs for short) were proposed by Lasry and Lions in a serie of papers(see [14–16]) and also independently by Huang, Caines and Malham′e [10], under the different name of Nash certainty equivalence. They are sometimes approached by symmetric, noncooperative stochastic differential games of interacting N players. To be specific, each player solves a stochastic control problem with the cost and the state dynamics depending not only on his own state and control but also on other players’ states. The interaction among the players can be weak in the sense that one player is influenced by the other players only through the empirical distribution. In view of the theory of McKean-Vlasov limits and propagation of chaos for uncontrolled weakly interacting particle systems (see [22]), it is expected to have a convergence for N-player game Nash equilibria by assuming independence of the random noise in the players’ state processes and some symmetry conditions of the players. The literature in this area is huge. See [3] for a summary of a series of Lions’ lectures given at the Coll′ege de France. Carmona and Delarue approached the MFG problem from a probabilistic point of view(see[4–6]). There are rigorous results about construction of ε-Nash equilibria for N-player games, see for example [4, 8, 11–13].

In most studies mentioned above, the noises in each player’s state dynamic are assumed to be independent and the empirical distribution of players’ states is deterministic in the limit.See [7] on a model of inter-bank borrowing and lending, where noises of players are dependent.

The presence of a common noise clearly adds extra complexity to the problem as the empirical distribution of players’ state becomes stochastical in the limit. Following a PDE approach,Pham and Wei [21] studied the dynamic programming for optimal control of stochastic McKean-Vlasov dynamics; in particular, Pham [20] solved the optimal control problem for a linear conditional McKean-Vlasov equation with a quadratic cost functional. Carmona and Delarue [4] consider the mean field game without common noises. They use a probabilistic approach based on the stochastic maximum principle (SMP for short) within a linear-convex framework. Nonetheless, their arguments of using Schauder fixed-point theorem to a compact subset of deterministic flows of probability measures, is difficult to be adapted to the case of common noises. Yu and Tang [24] considered mean field games with degenerate state- and distribution-dependent noise. Ahuja [1] studied a simple linear model of the mean field games in the presence of common noise with the terminal cost being convex and weakly monotone.The statistics of the state process occurs in the McKean-Vlasov forward-backward stochastic differential equation (FBSDE for short) arising from the stochastic maximum principle as the distribution conditioned on the common noise. Ahuja et al. [2] further consider a system of FBSDEs with monotone functionals and then solve the mean field game with a common noise within a linear-convex setting for weakly monotone cost functions. However, their state dynamics do not depend on the statistics of the state. The monotone condition usually fails to hold for the conditional distribution dependent FBSDE if the state dynamic depends on the conditional distribution of the state.

In this paper, we consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way. We use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent FBSDE. We prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state coefficient on the conditional distribution is sufficiently small, or when the convexity parameter of the running cost on the control is sufficiently large. To accomplish this, we assume that the terminal cost and the running cost are convex and weakly monotone.We develop two different methods to show the existence and uniqueness result.

The first method is based on a continuation of the coefficients,which is developed for FBSDE by Hu and Peng [9]. With this method, Carmona and Delarue [5] solve a linear case without common noises and Ahuja et al. [2] solve that mean field games with common noises within a linear-convex setting when the state dynamic is independent of the conditional distribution of state.

The second method, inspired by [1], is first to show the existence result on a small time interval by Banach fixed point theorem and then to extend the local solution to the whole time.Ahuja [1] showed the existence and uniqueness result for the particular MFG with common noises for the linear state dXt= αtdt+σdWt+where αtis the control, (W,) is a two-dimensional standard Brownian motion and (σ,) is constant. We shall consider a more general model. All the coefficients of our state equation are allowed to depend on the control,the state and the conditional distribution of state. More assumptions in the second method are required to derive the existence result, while the probabilistic properties as well as the sensitivity of the FBSDEs have their own interests.

The paper is organized as follows. In Section 2, we introduce our model and formulate the main problem. In Section 3, we use the sufficient Pontryagin principle for optimality to transform the control problem into an existence and uniqueness problem of a conditional distribution dependent FBSDE. The existence and uniqueness result of the conditional distribution dependent FBSDE is stated and proved with different methods in Sections 4–5. Appendices containing the proofs of main lemmas are attached.

2 Problem Formulation

In this section, we describe our stochastic differential game model, and then formulate the limit problem of the N-player game as a MFG with a common noise.

2.1 Notations

Let P(R) denote the space of all Borel probability measures on R, and P2(R) denote the space of all probability measures m ∈P(R) such that

The Wasserstein distance is defined on P2(R) by

where Γ(m1,m2) denotes the collection of all probability measures on R2with marginals m1and m2. The space (P2(R),W2) is a complete separable metric space. Let M2(C[0,T]) denote the space of all probability measures m on C[0,T] such that

The measure on it is defined by

The space (M2(C[0,T]),D2) is a complete separable metric space.

2.2 N-Player stochastic differential games

are assumed to be identical for all players.

Note that the strategies of other players have an effect on the i-th player throughwhich is the main feature that makes this set up a game. We are seeking a type of equilibrium solution widely used in game theory setting called Nash equilibrium.

Definition 2.1A set of strategies (ui)1≤i≤Nis a Nash equilibrium if uiis optimal for the i-th player given the other players’ strategies (. In other words,

Solving for a Nash equilibrium of an N-player game is impractical when N is large due to the curse of dimensionality. So we formally take the limit as N →∞and consider the limit problem instead.

2.3 Formulation of the problem

We now formulate the MFG with a common noise by taking the limit of N-player stochastic differential games as N →∞. When considering the limiting problem, we assume that each player adopts the same strategy. Therefore, the players’ distribution can be represented by a conditional law of a single representative player given a common noise. In other words, we formulate the MFG with a common noise as a stochastic control problem for a single player with an equilibrium condition involving a conditional law of the state process given a common noise.

3 Stochastic Maximum Principle

In this section,we discuss the stochastic maximum principle for MFG with a common noise.The stochastic maximum principle gives optimality conditions satisfied by an optimal control.It gives sufficient and necessary conditions for the existence of an optimal control in terms of solvability of the adjoint process as a backward stochastic differential equation (BSDE for short). For more details about stochastic maximum principle, we refer to [19] or [22]. In our case, Problem 2.1 is associated to a conditional distribution dependent FBSDE with the help of the sufficient Pontryagin principle for optimality.

We begin with discussing the stochastic maximum principle given anprogressivelymeasurable stochastic flow of probability measures m = {mt,0 ≤t ≤T} ∈M2(C[0,T]).We define the generalized Hamiltonian

Now we state the first set of assumptions to ensure that the stochastic control problem is uniquely solvable given m. For notational convenience, we use the same constant L for all the conditions below.

(H1) The drift b and the volatility σ,are linear in x and u. They read

for some measurable deterministic functions φ0: [0,T]×P2(R) →R satisfying the following linear growth:

and φ1,φ2: [0,T] →R being bounded by a positive constant L. Further, (σ2,is bounded by a positive constant Bu. For notational convenience, we can assume that Bu≤L by setting L=max{L,Bu}.

(H2) The function f(t,0,0,m) satisfies a quadratic growth condition in m. The function f(t,·,·,m) : R×R →R is differentiable for all (t,m) ∈[0,T]×P2(R), with the derivatives(fx,fu)(t,x,u,m)satisfying a linear growth in(x,u,m). Similarly,the function g(0,m)satisfies a quadratic growth condition in m. The function g(·,m) : R →R is differentiable for all m ∈P2(R), with the derivative gx(x,m) satisfying a linear growth in (x,m). That is,

(H3) The function f is of the form

The function f0is differentiable with respect to (x,u) and the function f1is differentiable with respect to x. The derivatives (f0x,f0u)(t,·,·) : R×R →R×R are L-Lipschitz continuous uniformly in t ∈[0,T]. The derivative f1x(t,·,m):R →R is L-Lipschitz continuous uniformly in (t,m)∈[0,T]×P2(R). The derivative gx(·,m):R →R is L-Lipschitz continuous uniformly in m ∈P2(R).

(H4) The functions f1(t,·,m) and g(·,m) are convex for all (t,m)∈[0,T]×P2(R), in such a way that

The function f0(t,x,u) is jointly convex in (x,u) with a strict convexity in u for all t ∈[0,T],in such a way that, for some Cf>0,

The linear growth condition(H2)and Lipschitz condition(H3)are standard assumptions to ensure the existence of a strong solution. The linear-convex conditions (H1) and (H4) ensure that the Hamiltonian is strictly convex, so that there is a unique minimizer in the feedback form. The separability condition in (H3) ensures that the feedback control is independent of m. The following result is borrowed from [4, Lemma 1].

Using the convex assumption (H4), we have that

which imply

The above estimate and assumption (H2) show that

We are ready to state the stochastic maximum principle for a given stochastic flow of probability measures m={mt,0 ≤t ≤T}∈M2(C[0,T]). We define the control problem Pm:

ProofThe proof is standard and we refer to [19, Theorem 6.4.6]. The estimate (3.5)requires strict convexity in u of f0. The proof can be found in [4, Theorem 2.2].

We now show FBSDE(3.3)is uniquely solvable,which implies that problem Pmis uniquely solvable. We state the slightly more general result for a random terminal function and an arbitrary initial and terminal time, which will arise in a subsequent section. It is an immediate consequence of [18, Theorem 2.3], concerning the existence and uniqueness of a solution to a monotone FBSDE.

for some constant C depending on (L,T,Cf,Cv).

Plugging this into Theorem 3.1, we have the stochastic maximum principle for Problem 2.1.

Equivalently,for any square-integrable random variables ξ and ξ′on the same probability space,

4 Solvability of FBSDE (3.8): Method One

In this section, we give the existence and uniqueness result of the solution to FBSDE (3.8)by the method of continuation in coefficients. We have the following main result.

We call an input for FBSDE (3.8) a five-tuple

Our aim is to show (S1) holds true. The following lemma is proved in Appendix A.

Lemma 4.1Suppose that assumptions (H1)–(H6) hold. Let γ ∈[0,1] such that (Sγ) holds true. Then, there exist δ > 0 depending only on (L,T), and a constant C independent of γ,such that for any ξ1,ξ2∈and I1,I2∈I, the respective extended solutions Θ1and Θ2of E(γ,ξ1,I1) and E(γ,ξ2,I2) satisfy

when Lm≤δ.

We now give the following lemma, which plays a crucial role in the proof of Theorem 4.1.

Lemma 4.2Suppose that assumptions (H1)–(H6) hold. There exist δ > 0 depending only on (L,T) and η0> 0 such that, if Lm≤δ and (Sγ) holds true for some γ ∈[0,1), then(Sγ+η) holds true for any η ∈(0,η0] satisfying γ+η ≤1.

ProofThe proof follows from the contraction of Picard’s mapping. Consider γ such that(Sγ) holds true. For η > 0, any ξ ∈and any I ∈I, we aim to show that the FBSDE E(γ+η,ξ,I) has a unique extended solution in S. To do so, we define a map Φ:S →S, whose fixed points are solutions of E(γ+η,ξ,I).

The definition Φ is as follows. Given a process Θ ∈S,we denote by Θ′the extended solution of the FBSDE E(γ,ξ,I′) with

From the assumption that (Sγ) holds true, Θ′is uniquely defined, and it belongs to S, so that Φ : ΘΘ′maps S into itself. It is then clear that a process Θ ∈S is a fixed point of Φ if and only if Θ is an extended solution of E(γ +η,ξ,I). So we only need to illustrate that Φ is a contraction when η is small enough.

In fact, for any Θ1,Θ2∈S, we know from Lemma 4.1 that

where C is independent of γ and η. So when η is small enough, Φ is indeed a contraction.

Proof of Theorem 4.1In view of Lemma 4.2, we only need to prove that (S0) holds true, which is obviously true since there is no coupling between the forward and the backward equations when γ =0.

5 Solvability of FBSDE (3.8): Method Two

In this section, we prove the existence and uniqueness of the solution of FBSDE (3.8) with an alternative method. In the first subsection, we use the weak monotonicity assumption to deduce the uniqueness result. And in the second subsection, we first show the existence result on a small time interval [τ,T] and then extend the local solution to the whole time interval[0,T]. More assumptions are required than the first method. However, the intermediate result can better demonstrate the probabilistic properties as well as the sensitivity, which are worthy of study.

5.1 Uniqueness

We have the following uniqueness of the solution of FBSDE (3.8).

We know from (5.1) that

From the strict convexity of f0as assumed in (H4), we have that for t ∈[0,T],

From (5.4), we have

From the weak monotonicity assumption (H6), we know that

Plugging (5.3), (5.5) and (5.6) into (5.2), using the Lipschitz continuity assumption (H5) and the average inequality, we have

where we have used the following estimates

By standard estimates for SDEs and BSDEs, there exist two constants C1> 0 and C2> 0 depending only on (L,T), such that

From (5.7)–(5.9), we have

The constant

5.2 Existence

Next, we prove the existence result of the solution of FBSDE (3.8). The idea is to show the existence result on a small time interval [τ,T] firstly and then extend the local solution to the whole time interval [0,T]. In this subsection, we always suppose that assumptions (H1)–(H6)hold.

The lemma below is an immediate consequence of [2, Theorem 3]. Similar results can be found in [17, Theorem 6.7] and [22, Theorem 1.1].

and

with the optimality condition

or equivalently,

such that

We set

Conditions (5.11)–(5.13) and Theorem 3.2 ensure that u is uniquely defined. Moreover, both inequalities (5.15) and (3.2) and Lemma 3.1 yield that u ∈(s,τ). Thus, Φs,τ,η,v:u maps(s,τ) into itself. Furthermore, the fixed point of Φ0,T,ξ0,gxis the solution of FBSDE(3.8). The lemma below gives a solution on a small time interval.

The proof is given in Appendix B. Assumptions(H2)–(H5)ensure that gxsatisfies conditions(5.11)–(5.13). Suppose that

the solution of the following FBSDE,

where

and with the optimality condition

such that

We then denote by

the solution of the following FBSDE

with the optimality condition

We now attempt to extend the solution further. If we suppose that

then we easily get

Appendices

A Proof of Lemma 4.1

with the condition

First we note the fact that

Under assumptions (H1), (H3), (H5) and the estimates (A.2), by standard estimates for SDEs and BSDEs, there exist two constants C1>0 and C2>0 depending only on (L,T), such that

Applying It?o’s lemma on ?pt?Xtand taking expection, we have

By using the Lipschitz-continuity assumption (H5) and the average inequality, we have

From (A.1) and the convex assumption (H4), we have

From the weak monotonicity assumption (H6) and the fact thatwe have

Plugging (A.6)–(A.8) into (A.5), and using the average inequality, we have for any ε>0,

Here, the notation C(T,ε) stands for a constant depending only on T and ε. Plugging (A.3)and (A.4) into above, we have

The constant

depends only on (L,T). If Lm≤δ, then, we choose to get

Plugging (A.9) into (A.3) and (A.4), respectively, we have

From Lemma 3.1 we know that

So we eventually have

B Proof of Lemma 5.2

From Lemma 3.1 we know that

and recall that Bu≤L. From (B.1) and assumptions (H1), (H3) and (H5), we have

Here, the notation C(L,T,Cf) stands for a constant depending only on L, T and Cf, and we have used the following estimates

Similarly, by using Doob’s inequality, Cauchy’s inequality and (B.1), we have

We also have from (B.1) that

From (B.2)–(B.4), we deduce that

From the condition

we have that when (τ ?s) is small enough,

From (B.5), (B.7) and (B.1), we deduce that when (τ ?s) is small enough,

Similar as the above, we have

From (B.6), (B.8) and (B.9), we deduce that

It follows that when (τ ?s) is small enough,

As a result,we get a contraction map for sufficiently small(τ?s)depending only on(L,T,Cv,Cf)as desired.

C Proof of Lemma 5.3

In this section, we give the proof of (5.22) and (5.23), respectively. From the condition(5.16) and Lemma 5.2, we know that v is well-defined.

C.1 Proof of (5.22)

From the optimal conditions (5.21) of u1and u2, we have

From the convexity assumption (H4), we have

From (C.1) and (C.2), we deduce that

C.2 Proof of (5.23)

with the condition

From assumptions (H4) and (H5), we have

Plugging (C.5) into (C.4) and using the average inequality, we have for any ε>0,

where we have used the estimates

By standard estimates for SDEs and BSDEs, we have

where C(L,T)stands for some positive constant depending only on L and T. From(C.6)–(C.8)and (C.6), we deduce that when ε is small enough,

Now we plug (C.7) and (C.9) into (C.8) to get

with the condition

As above, by standard estimates for SDEs and BSDEs, there exist two constants C1> 0 and C2>0 depending only on (L,T), such that

From the weak monotonicity condition (H6), we have

By applying average inequality for?η, we have for any ε>0,

Plugging (C.12) and (C.13) into (C.15) and recall that Lm≤L, we have

The constant

Then, have

Plugging (C.16) into (C.12), we have

Now we plug (C.17) into (C.10). If Lm≤δ, we have

or equivalently,

as desired.