Probabilistic Analysis of Mean-Field Games

Introduction

In a trailblazing contribution, Lasry and Lions proposed a methodology to produce approximate Nash equilibriums for stochastic differential games with symmetric interactions and a large number of players. In their model, the costs to a given player feel the presence and the behavior of the other players through the empirical distribution of their private states. This type of interaction was introduced and studied in statistical physics under the name of mean-field interaction, allowing for the derivation of effective equations in the limit of asymptotically large systems. Using intuition and mathematical results from propagation of chaos, Lasry and Lions propose to assign to each player, independently of what other players may do, a distributed closed loop strategy given by the solution of the limiting problem, arguing that such a resulting game should be in an approximate Nash equilibrium. This streamlined approach is very attractive as large stochastic differential games are notoriously nontractable. They formulated the limiting problem as a system of two highly coupled nonlinear partial differential equations (PDE for short): the first one, of the Hamilton-Jacobi-Bellman type, takes care of the optimization part, while the second one, of Kolmogorov type, guarantees the time consistency of the statistical distributions of the private states of the individual players. The issue of existence and uniqueness of solutions for such a system is a very delicate problem, as the solution of the former equation should propagate backward in time from a terminal condition while the solution of the latter should evolve forward in time from an initial condition. More than the nonlinearities, the conflicting directions of time compound the difficulties.

In a subsequent series of works with PhD students and postdoctoral fellows, Lasry and Lions considered applications to domains as diverse as the management of exhaustible resources like oil, house insulation, and the analysis of pedestrian crowds. Motivated by problems in large communication networks, Caines, Huang and Malhamé introduced, essentially at the same time , a similar strategy which they call the Nash Certainty Equivalence. They also studied practical applications to large populations behavior .

The goal of the present paper is to study the effective Mean-Field Game equations proposed by Lasry and Lions, from a probabilistic point of view. To this end, we recast the challenge as a fixed point problem in a space of flows of probability measures, show that these fixed points do exist and provide approximate Nash equilibriums for large games, and quantify the accuracy of the approximation.

We tackle the limiting stochastic optimization problems using the probabilistic approach of the stochastic maximum principle, thus reducing the problems to the solutions of Forward Backward Stochastic Differential Equations (FBSDEs for short). The search for a fixed flow of probability measures turns the system of forward-backward stochastic differential equations into equations of the McKean-Vlasov type where the distribution of the solution appears in the coefficients. In this way, both the optimization and interaction components of the problem are captured by a single FBSDE, avoiding the twofold reference to Hamilton-Jacobi-Bellman equations on the one hand, and Kolmogorov equations on the other hand. As a by-product of this approach, the stochastic dynamics of the states could be degenerate. We give a general overview of this strategy in Section 2 below. Motivated in part by the works of Lasry, Lions and collaborators, Backward Stochastic Differential Equations (BSDEs) of the mean field type have recently been studied. See for example . However, existence and uniqueness results for BSDEs are much easier to come by than for FBSDEs, and here, we have to develop existence results from scratch.

Our first existence result is proven for bounded coefficients by means of a fixed point argument based on Schauder’s theorem pretty much in the same spirit as Cardaliaguet’s notes . Unfortunately, such a result does not apply to some of the linear-quadratic (LQ) games already studied , and some of the most technical proofs of the papers are devoted to the extension of this existence result to coefficients with linear growth. See Section 3. Our approximation and convergence arguments are based on probabilistic a priori estimates obtained from tailor-made versions of the stochastic maximum principle which we derive in Section 2. The reader is referred to the book of Ma and Yong for background material on adjoint equations, FBSDEs and the stochastic maximum principle approach to stochastic optimization problems. As we rely on this approach, we find it natural to derive the compactness properties needed in our proofs from convexity properties of the coefficients of the game. The reader is also referred to the papers by Hu and Peng and Peng and Wu for general solvability properties of standard FBSDEs within the same framework of stochastic optimization.

The thrust of our analysis is not limited to existence of a solution to a rather general class of McKean-Vlasov FBSDEs, but also to the extension to this non-Markovian set-up of the construction of the FBSDE value function expressing the solution of the backward equation in terms of the solution of the forward dynamics. The existence of this value function is crucial for the formulation and the proofs of the results of the last part of the paper. In Section 4, we indeed prove that the solutions of the fixed point FBSDE (which include a function $\hat{\alpha}$ minimizing the Hamiltonian of the system, three stochastic processes $(X_{t},Y_{t},Z_{t})_{0\leq t\leq T}$ solving the FBSDE, and the FBSDE value function $u$) provide a set of distributed strategies which, when used by the players of a $N$-player game, form an $\epsilon_{N}$-approximate Nash equilibrium, and we quantify the speed at which $\epsilon_{N}$ tends to when $N\to+\infty$. This type of argument has been used for simpler models in or . Here, we use convergence estimates which are part of the standard theory of propagation of chaos (see for example ) and the Lipschitz continuity and linear growth the FBSDE value function $u$ which we prove earlier in the paper.

General Notation and Assumptions

Here, we introduce the notation and the basic tools from stochastic analysis which we use throughout the paper.

where we use the standard notation $(\alpha^{*-i},\alpha^{i})$ for the set of strategies $(\alpha^{*1},\cdots,\alpha^{*N})$ where $\alpha^{*i}$ has been replaced by $\alpha^{i}$.

2. The Mean-Field Problem

In the case of large symmetric games, some form of averaging is expected when the number of players tends to infinity. The Mean-Field Game (MFG) philosophy of Lasry and Lions is to search for approximate Nash equilibriums through the solution of effective equations appearing in the limiting regime $N\to\infty$, and assigning to each player the strategy $\alpha$ provided by the solution of the effective system of equations they derive. In the present context, the implementation of this idea involves the solution of the following fixed point problem which we break down in three steps for pedagogical reasons:

Solve the standard stochastic control problem

3. The Hamiltonian

Here and in the following, whenever $E$ is a separable Banach space and $p$ is an integer greater than $1$, $\mathcal{P}_{p}(E)$ stands for the subspace of $\mathcal{P}(E)$ of probability measures of order $p$, i.e. having a finite moment of order $p$ so that $\mu\in\mathcal{P}_{p}(E)$ if $\mu\in\mathcal{P}(E)$ and

Inequality (9) will be used repeatedly. Moreover, by the implicit function theorem, $\hat{\alpha}$ is Lipschitz-continuous with respect to $(x,y)$, the Lipschitz-constant being controlled by the uniform bound on $b_{2}$ and by the Lipschitz-constant of $\partial_{(x,\alpha)}f$. ∎

4. Stochastic Maximum Principle

Going back to the program (i)–(iii) outlined in Subsection 2.2, the first two steps therein consist in solving a standard minimization problem when the distributions $(\mu_{t})_{0\leq t\leq T}$ are frozen, and one could express the value function of the optimization problem (4) as the solution of the corresponding Hamilton-Jacobi-Bellman (HJB for short) equation. This is the keystone of the analytic approach to the MFG theory, the matching problem (iii) being resolved by coupling the HJB equation with a Kolmogorov equation intended to identify the $(\mu_{t})_{0\leq t\leq T}$ with the marginal distributions of the optimal state of the problem.

for any progressively measurable process $\beta=(\beta_{t})_{0\leq t\leq T}$ satisfying the admissibility condition (2) where $U=(U_{t})_{0\leq t\leq T}$ is the corresponding controlled diffusion process

has a solution $(X_{t},Y_{t},Z_{t})_{0\leq t\leq T}$ such that

if we set $\hat{\alpha}_{t}=\hat{\alpha}(t,X_{t},\mu_{t},Y_{t})$, then for any $\beta=(\beta_{t})_{0\leq t\leq T}$ satisfying (2), it holds

By Lemma 1, $\hat{\alpha}=(\hat{\alpha}_{t})_{0\leq t\leq T}$ satisfies (2), and the standard proof of the stochastic maximum principle, see for example Theorem 6.4.6 in Pham gives

By linearity of $b$ and assumption (A.2) on $b$, the Hessian of $H$ satisfies (8), so that the required convexity assumption is satisfied. The result easily follows. ∎

As the proof shows, the result of Theorem 1 above still holds if the control $\beta=(\beta_{t})_{0\leq t\leq T}$ is merely adapted to a larger filtration as long as the Wiener process $W=(W_{t})_{0\leq t\leq T}$ remains a Brownian motion for this filtration.

so that $X$ and $X^{\prime}$ coincide by the Lipschitz property of the coefficients of the forward equation. As a consequence, $(Y,Z)$ and $(Y^{\prime},Z^{\prime})$ coincide as well.

It should be noticed that in some sense, the bound provided by Theorem 1 is sharp within the realm of convex models as shown for example by the following slight variation on the same theme. We shall use this form repeatedly in the proof of our main result.

The parameter $[\beta,\mu^{\prime}]$ in the cost $J([\beta,\mu^{\prime}];\mu)$ indicates that the flow of measures in the drift of $U^{\prime}$ is $(\mu_{t}^{\prime})_{0\leq t\leq T}$ whereas the flow of measures in the cost functions is $(\mu_{t})_{0\leq t\leq T}$. In fact, we should also indicate that the initial condition $x_{0}^{\prime}$ might be different from $x_{0}$, but we prefer not to do so since there is no risk of confusion in the sequel. Also, when $x_{0}^{\prime}=x_{0}$ and $\mu_{t}^{\prime}=\mu_{t}$ for any $t\in[0,T]$, $J([\beta,\mu^{\prime}];\mu)=J(\beta;\mu)$.

The idea is to go back to the original proof of the stochastic maximum principle and using Itô’s formula, expand

Since the initial conditions $x_{0}$ and $x_{0}^{\prime}$ are possibly different, we get the additional term $\langle x_{0}^{\prime}-x_{0},Y_{0}\rangle$ in the left hand side of (14). Similarly, since the drift of $U^{\prime}$ is driven by $(\mu_{t}^{\prime})_{0\leq t\leq T}$, we get the additional difference of the drifts in order to account for the fact that the drifts are driven by the different flows of probability measures. ∎

The Mean-Field FBSDE

In order to solve the standard stochastic control problem (4) using the Pontryagin maximum principle, we minimize the Hamiltonian $H$ with respect to the control variable $\alpha$, and inject the minimizer $\hat{\alpha}$ into the forward equation of the state as well as the adjoint backward equation. Since the minimizer $\hat{\alpha}$ depends upon both the forward state $X_{t}$ and the adjoint process $Y_{t}$, this creates a strong coupling between the forward and backward equations leading to the FBSDE (12). The MFG matching condition (iii) of Subsection 2.2 then reads: seek a family of probability distributions $(\mu_{t})_{0\leq t\leq T}$ of order 2 such that the process $X$ solving the forward equation of (12) admits $(\mu_{t})_{0\leq t\leq T}$ as flow of marginal distributions.

In a nutshell, the probabilistic approach to the solution of the mean-field game problem results in the solution of a FBSDE of the McKean-Vlasov type

For the sake of convenience, we restate the general FBSDE (16) of McKean-Vlasov type in the special set-up of the present paper. It reads:

where $a^{\dagger}$ denotes the transpose of the matrix $a$.

In addition to (A.1–4), we shall rely on the following assumptions.

(A.5) provides Lipschitz continuity while condition (A.6) controls the smoothness of the running cost $f$ with respect to $\alpha$ uniformly in the other variables. The most unusual assumption is certainly condition (A.7). We refer to it as a weak mean-reverting condition as it looks like a standard mean-reverting condition for recurrent diffusion processes. Moreover, as shown by the proof of Theorem 2, Its role is to control the expectation of the forward equation in (16) and to establish an a priori bound for it. This is of crucial importance in order to make the compactness strategy effective. We use the terminology weak as it is not expected to converge with time.

An interesting example which we should keep in mind is the so-called linear-quadratic model in which $b_{0}$, $f$ and $g$ have the form:

2. Rigorous Definition of the Matching Problem

The proof of Theorem 2 is split into four main steps. The first one consists in making the statement of the matching problem (iii) in (4) rigorous. To this end, we need the following

for some constant $c$ independent of $t_{0}$ and $\delta$. Notice that, by Blumenthal’s Zero-One Law, the random variables $Y_{t_{0}}^{t_{0},x_{0}}$ and $Y_{t_{0}}^{t_{0},x_{0}^{\prime}}$ are deterministic. By (14), we have

where $\hat{J}^{t_{0},x_{0}}=J((\hat{\alpha}_{t}^{t_{0},x_{0}})_{t_{0}\leq t\leq T};\mu)$ and $\hat{\alpha}_{t}^{t_{0},x_{0}}=\hat{\alpha}(t,X_{t}^{t_{0},x_{0}},\mu_{t},Y_{t}^{t_{0},x_{0}})$ (with similar definitions for $\hat{J}^{t_{0},x_{0}^{\prime}}$ and $\hat{\alpha}_{t}^{t_{0},x_{0}^{\prime}}$ by replacing $x_{0}$ by $x_{0}^{\prime}$). Exchanging the roles of $x_{0}$ and $x_{0}^{\prime}$ and adding the resulting inequality with (20), we deduce that

Moreover, by standard SDE estimates first and then by standard BSDE estimates, there exists a constant $c$ (the value of which may vary from line to line), independent of $t_{0}$ and $\delta$, such that

Plugging (21) into the above inequality completes the proof of (19).

Definition 1 captures the essence of the approach of Lasry and Lions who freeze the probability measure at the optimal value when optimizing the cost. This is not the case in the study of the control of McKean-Vlasov dynamics, as investigated in : in this different setting, optimization is also performed with respect to the measure argument. See also and for the linear quadratic case.

3. Existence under Additional Boundedness Conditions

We first prove existence under an extra boundedness assumption.

The system (16) is solvable if, in addition to (A.1–7), we also assume that $\partial_{x}f$ and $\partial_{x}g$ are uniformly bounded, i.e. for some constant $c_{B}>0$

First Step. We first establish several a priori estimates for the solution of (12). The coefficients $\partial_{x}f$ and $\partial_{x}g$ being bounded, the terminal condition in (12) is bounded and the growth of the driver is of the form:

Plugging this bound into the forward part of (12), standard $L^{p}$ estimates for SDEs imply that there exists a constant $c^{\prime}$, only depending upon $c_{B}$, $c_{L}$ and $T$, such that

We consider the restriction of $\Phi$ to the subset ${\mathcal{E}}$ of probability measures of order $4$ whose fourth moment is not greater than $c^{\prime}$, i.e.

${\mathcal{E}}$ is convex and closed for the $1$-Wasserstein distance and $\Phi$ maps ${\mathcal{E}}$ into itself.

Third Step. We finally check that $\Phi$ is continuous on ${\mathcal{E}}$. Given another measure $\mu^{\prime}\in{\mathcal{E}}$, we deduce from (14) in Proposition 1 that:

where $\hat{\alpha}_{t}=\hat{\alpha}(t,X_{t}^{x_{0};\mu},\mu_{t},Y_{t}^{x_{0};\mu})$, for $t\in[0,T]$, with a similar definition for $\hat{\alpha}_{t}^{\prime}$ by replacing $\mu$ by $\mu^{\prime}$. By optimality of $\hat{\alpha}^{\prime}$ for the cost functional $J(\cdot;\mu^{\prime})$, we claim:

We now compare $J(\hat{\alpha};\mu^{\prime})$ with $J(\hat{\alpha};\mu)$ (and similarly $J(\hat{\alpha}^{\prime};\mu^{\prime})$ with $J([\hat{\alpha}^{\prime},\mu^{\prime}];\mu)$). We notice that $J(\hat{\alpha};\mu)$ is the cost associated with the flow of measures $(\mu_{t})_{0\leq t\leq T}$ and the diffusion process $X^{x_{0},\mu}$ whereas $J(\hat{\alpha};\mu)$ is the cost associated with the flow of measures $(\mu_{t}^{\prime})_{0\leq t\leq T}$ and the controlled diffusion process $U$ satisfying

By Gronwall’s lemma, there exists a constant $c$ such that

Since $\mu$ and $\mu^{\prime}$ are in ${\mathcal{E}}$, we deduce from (A.5), (24) and (25) that

with a similar bound for $J([\hat{\alpha}^{\prime},\mu^{\prime}];\mu)-J(\hat{\alpha}^{\prime};\mu^{\prime})$ (the argument is even simpler as the costs are driven by the same processes), so that, from (27) and (23) again, together with Gronwall’s lemma to go back to the controlled SDEs,

4. Approximation Procedure

Examples of functions $f$ and $g$ which are convex in $x$ and such that $\partial_{x}f$ and $\partial_{x}g$ are bounded are rather limited in number and scope. For instance, boundedness of $\partial_{x}f$ and $\partial_{x}g$ fails in the typical case when $f$ and $g$ are quadratic with respect to $x$. In order to overcome this limitation, we propose to approximate the cost functions $f$ and $g$ by two sequences $(f^{n})_{n\geq 1}$ and $(g^{n})_{n\geq 1}$, referred to as approximated cost functions, satisfying (A.1–7) uniformly with respect to $n\geq 1$, and such that, for any $n\geq 1$, equation (16), with $(\partial_{x}f,\partial_{x}g)$ replaced by $(\partial_{x}f^{n},\partial_{x}g^{n})$, has a solution $(X^{n},Y^{n},Z^{n})$. In this framework, Proposition 2 says that such approximated FBSDEs are indeed solvable when $\partial_{x}f^{n}$ and $\partial_{x}g^{n}$ are bounded for any $n\geq 1$. Our approximation procedure relies on the following:

If there exist two sequences $(f^{n})_{n\geq 1}$ and $(g^{n})_{n\geq 1}$ such that

$(i)$ there exist two parameters $c_{L}^{\prime}$ and $\lambda^{\prime}>0$ such that, for any $n\geq 1$, $f^{n}$ and $g^{n}$ satisfy (A.1–7) with respect to $\lambda^{\prime}$ and $c_{L}^{\prime}$;

$(iii)$ for any $n\geq 1$, equation (16), with $(\partial_{x}f,\partial_{x}g)$ replaced by $(\partial_{x}f^{n},\partial_{x}g^{n})$, has a solution which we denote by $(X^{n},Y^{n},Z^{n})$. Then, equation (16) is solvable.

We establish tightness of the processes $(X^{n})_{n\geq 1}$ in order to extract a convergent subsequence. For any $n\geq 1$, we consider the approximated Hamiltonian

Since $X^{n}$ is the diffusion process controlled by $(\hat{\alpha}^{n}_{t})_{0\leq t\leq T}$, we use Theorem 1 to compare its behavior to the behavior of a reference controlled process $U^{n}$ whose dynamics are driven by a specific control $\beta^{n}$. We shall consider two different versions for $U^{n}$ corresponding to the following choices for $\beta^{n}$:

For each of these controls, we compare the cost to the optimal cost by using the version of the stochastic maximum principle which we proved earlier, and subsequently, derive useful information on the optimal control $(\hat{\alpha}_{s}^{n})_{0\leq s\leq T}$.

First Step. We first consider $(i)$ in (29). In this case

from which it easily follows that $\sup_{n\geq 1}\sup_{0\leq s\leq T}\textrm{Var}(U^{n}_{s})<+\infty$.

By (A.5), we deduce that there exists a constant $c$, depending only on $\lambda$, $c_{L}$, $x_{0}$ and $T$, such that (the actual value of $c$ possibly varying from line to line)

From this, the linearity of the dynamics of $X^{n}$ and Gronwall’s inequality, we deduce:

Second Step. We now compare $X^{n}$ to the process controlled by the null control. So we consider case $(ii)$ in (29), and now

By convexity of $f^{n}$ with respect to $\alpha$ (see (A.2)) together with (A.6), we have

for some constant $c$, independent of $n$. Using (A.5) again, we obtain:

the value of $c$ possibly varying from line to line. From (35), Young’s inequality yields

Young’s inequality and the convexity in $x$ of $g^{n}$ and $f^{n}$ from (A.2,4) give:

Using (28) and (36), it is plain to prove that the processes $({X}^{n})_{n\geq 1}$ are tight.

with $\hat{\alpha}_{t}=\hat{\alpha}(t,X_{t},\mu_{t},Y_{t})$, for $t\in[0,T]$. We can now apply the same argument to any $(X^{n}_{t})_{0\leq t\leq T}$, for any $n\geq 1$. We claim

where $J(\cdot;\mu)$ is given by (11) and $J^{n}(\cdot;\mu^{n})$ is defined in a similar way, but with $(f,g)$ and $(\mu_{t})_{0\leq t\leq T}$ replaced by $(f^{n},g^{n})$ and $(\mu_{t}^{n})_{0\leq t\leq T}$; $J([\hat{\alpha}^{n},\mu^{n}];\mu)$ is defined as in (15). With these definitions at hand, we notice that

where $U^{n}$ is the controlled diffusion process:

5. Choice of the Approximating Sequence

In order to complete the proof of Theorem 2, we must specify the choice of the approximating sequence in Lemma 3. Actually, the choice is performed in two steps. We first consider the case when the cost functions $f$ and $g$ are strongly convex in the variables $x$:

Assume that, in addition to (A.1–7), there exists a constant $\gamma>0$ such that the functions $f$ and $g$ satisfy (compare with (8)):

Then, there exist two positive constants $\lambda^{\prime}$ and $c^{\prime}_{L}$, depending only upon $\lambda$, $c_{L}$ and $\gamma$, and two sequences of functions $(f^{n})_{n\geq 1}$ and $(g^{n})_{n\geq 1}$ such that

$(i)$ for any $n\geq 1$, $f^{n}$ and $g^{n}$ satisfy (A.1–7) with respect to the parameters $\lambda^{\prime}$ and $c_{L}^{\prime}$ and $\partial_{x}f^{n}$ and $\partial_{x}g^{n}$ are bounded,

The proof of Lemma 4 is a pure technical exercise in convex analysis, and for this reason, we postpone its proof to an appendix at the end of the paper.

6. Proof of Theorem 2

Equation (16) is solvable when, in addition to (A.1–7), $f$ and $g$ satisfy the convexity condition (42). Indeed, by Lemma 4, there exists an approximating sequence $(f^{n},g^{n})_{n\geq 1}$ satisfying $(i)$ and $(ii)$ in the statement of Lemma 3, and also $(iii)$ by Proposition 2. When $f$ and $g$ satisfy (A.1–7) only, the assumptions of Lemma 3 are satisfied with the following approximating sequence:

Propagation of Chaos and Approximate Nash Equilibriums

While the rationale for the mean-field strategy proposed by Lasry-Lions is clear given the nature of Nash equilibriums (as opposed to other forms of optimization suggesting the optimal control of stochastic dynamics of the McKean-Vlasov type as studied in ), it may not be obvious how the solution of the FBSDE introduced and solved in the previous sections provides approximate Nash equilibrium for large games. In this section, we prove just that. The proof relies on the fact that the FBSDE value function is Lipschitz continuous, standard arguments in the propagation of chaos theory, and the following specific result due to Horowitz et al. (see for example Section 10 in ) which we state as a lemma for future reference:

where $\bar{\mu}^{N}$ denotes the empirical measure of any sample of size $N$ from $\mu$.

where as before, $\hat{\alpha}$ is the minimizer function constructed in Lemma 1. For convenience, we fix a sequence $((W_{t}^{i})_{0\leq t\leq T})_{i\geq 1}$ of independent $m$-dimensional Brownian motions, and for each integer $N$, we consider the solution $(X_{t}^{1},\dots,X_{t}^{N})_{0\leq t\leq T}$ of the system of $N$ stochastic differential equations

with $t\in[0,T]$ and $X_{0}^{i}=x_{0}$. Equation (44) is well posed since $u$ satisfies the regularity property (18) and the minimizer $\hat{\alpha}(t,x,\mu_{t},y)$ was proven, in Lemma 1, to be Lipschitz continuous and at most of linear growth in the variables $x$ and $y$, uniformly in $t\in[0,T]$. The processes $(X^{i})_{1\leq i\leq N}$ give the dynamics of the private states of the $N$ players in the stochastic differential game of interest when the players use the strategies

These strategies are in closed loop form. They are even distributed since at each time $t\in[0,T]$, a player only needs to know the state of his own private state in order to compute the value of the control to apply at that time. By boundedness of $b_{0}$ and by (9) and (18), it holds

For the purpose of comparison, we introduce the notation we use when the players choose a generic set of strategies, say $((\beta_{t}^{i})_{0\leq t\leq T})_{1\leq i\leq N}$. In this case, the dynamics of the private state $U^{i}$ of player $i\in\{1,\cdots,N\}$ are given by:

the cost to the $i$th player. Our goal is to construct approximate Nash equilibriums for the $N$-player game from a solution of (16). We follow the approach used by Bensoussan et al. in the linear-quadratic case. See also .

Using the fact that the strategies $(\bar{\alpha}_{t}^{N,i})_{0\leq t\leq T}$ satisfy the square integrability condition of admissibility, the same argument gives:

for $2\leq i\leq N$, which clearly implies after summation:

For the next step of the proof we introduce the system of decoupled independent and identically distributed states

Using the regularity of the FBSDE value function $u$ and the uniform boundedness of the family $(M_{d+5}(\mu_{t}))_{0\leq t\leq T}$ derived in Theorem 2 together with the estimate recalled in Lemma 5, we can follow Sznitman’s proof (see also Theorem 1.3 of ) and get

(recall that $(X^{1},\dots,X^{N})$ solves (44)), and this implies:

so that, taking expectations on both sides and using (53) and Lemma 5, we get the desired estimate (54). Using the local-Lipschitz regularity of the coefficients $g$ and $f$ together with Cauchy-Schwarz inequality, we get, for each $i\in\{1,\cdots,N\}$,

for some constant $c>0$ which can change from line to line. By (46), we deduce

Now, by the Lipschitz property of the minimizer $\hat{\alpha}$ proven in Lemma 1 and by the Lipschitz property of $u$ in (18), we notice that

Using (53) and (54), this proves that, for any $1\leq i\leq N$,

This suggests that, in order to prove inequality (49) for $i=1$, we could restrict ourselves to compare $\bar{J}^{N,1}(\beta^{1},\bar{\alpha}^{2,N},\dots,\bar{\alpha}^{N,N})$ to $J$. Using the argument which led to (50), (51) and (52), together with the definitions of $U^{j}$ and $X^{j}$ for $j=1,\cdots,N$, we get, for any $t\in[0,T]$:

Therefore, using Gronwall’s inequality, we get:

Putting together (46), (53) and (58), we see that, for any $A>0$, there exists a constant $c_{A}$ depending on $A$ such that

for a constant $c_{A}$ depending upon $A$, and whose value can change from line to line. Now by the triangle inequality for the Wasserstein distance:

which is $O(N^{-1})$ because of (50) and (52). Plugging this inequality into (61), and using (60) to control the second term and Lemma 5 to estimate the third term therein, we conclude that

For the final step of the proof we define $(\bar{U}^{1}_{t})_{0\leq t\leq T}$ as the solution of the SDE

so that, from the definition (47) of $U^{1}$ we get:

Using the Lipschitz property of $b_{0}$, (62) and the boundedness of $b_{1}$ and applying Gronwall’s inequality, we get

so that, going over the computation leading to (56) once more and using (62), (50), (51) and (52):

where $J(\beta^{1})$ stands for the mean-field cost of $\beta^{1}$:

Since $J\leq J(\beta^{1})$ (notice that, even though $\beta^{1}$ is adapted to a larger filtration than the filtration of $W^{1}$, the stochastic maximum principle still applies as pointed out in Remark 1), we get in the end

Using the convexity in $x$ of $g$ around $x=0$ and the convexity of $f$ in $(x,\alpha)$ around $x=0$ and $\alpha=0$, see (8), we get:

The local-Lipschitz assumption with respect to the Wasserstein distance and the definition of the latter imply the existence of a constant $c>0$ such that for any $t\in[0,T]$,

with a similar inequality for $g$. From this, we deduce

By (A.5), we know that $\partial_{x}g$, $\partial_{x}f$ and $\partial_{\alpha}f$ are at most of linear growth in the measure parameter (for the $L^{2}$-norm), so that, for any $\delta>0$, there exists a constant $c_{\delta}$ such that

Estimates (50) and (51) show that one can choose $\delta$ small enough in (66) and $c$ so that

This proves that there exists an integer $N_{0}$ such that, for any integer $N\geq N_{0}$ and constant $\bar{A}>0$, one can choose $A>0$ such that

which provides us with the appropriate tool to choose $A$ and avoid having to consider $(\beta^{1}_{t})_{0\leq t\leq T}$ whose expected square integral is too large. ∎

form an approximate Nash equilibrium of the $N$-player game (47–48) but:

$(i)$ there exists an integer $N_{0}$ such that, for any $N\geq N_{0}$ and $\bar{A}>0$, there exists a constant $A>0$ such that, for any player $i\in\{1,\cdots,N\}$ and any admissible strategy $\beta^{i}=(\beta^{i}_{t})_{0\leq t\leq T}$,

$(ii)$ Moreover, for any $A>0$, there exists a sequence of positive real numbers $(\epsilon_{N})_{N\geq 1}$ converging toward , such that for any admissible strategy $\beta^{1}=(\beta^{1}_{t})_{0\leq t\leq T}$ for the first player

Appendix: Proof of Lemma 4

We focus on the approximation of the running cost $f$ (the case of the terminal cost $g$ is similar) and we ignore the dependence of $f$ upon $t$ to simplify the notation. For any $n\geq 1$, we define $f_{n}$ as the truncated Legendre transform:

Moreover, by strict convexity of $f$ in $x$,

so that $f_{n}$ has finite real values. Clearly, it is also $n$-Lipschitz continuous in $x$.

By (73) and (A.5), $f_{n}(t,x,\mu,\alpha)\geq-c(1+R^{2})$, $c$ depending possibly on $\gamma$, so that optimization in the variable $y$ can be done over points $y^{\star}$ satisfying

In particular, for $n$ large enough (depending on $R$),

the second inequality following from (A.5). By strict convexity of $f$ in $x$, we obtain

so that, by (A.5), $\gamma|z^{\star}|^{2}-c(1+R)|z^{\star}|\leq c(1+R^{2})$, that is

Taking infimum with respect to $z_{1},z_{2}$ and supremum with respect to $y$, we obtain

By expanding $f(\cdot,\mu,\cdot)$ up to the second order, we see that

for some constant $c$. Taking the supremum over $y_{1},y_{2}$, we deduce that

By (A.5) and (77), we can find a constant $c^{\prime}$ (possibly depending on $\gamma$) such that

This proves local Lipschitz-continuity in the measure argument as in (A.5).

In order to prove local Lipschitz-continuity in the variables $x$ and $\alpha$, we use the ${\mathcal{C}}^{1,1}$-property. Indeed, for $x$, $\mu$ and $\alpha$ as in (79), we know that

By (72), for any integer $p\geq 1$, there exists an integer $n_{p}$, such that, for any $n\geq n_{p}$, $f_{n}(0,\mu,0)$ and $f(0,\mu,0)$ coincide for $M_{2}(\mu)\leq p$. In particular, for $n\geq n_{p}$,

instead of $(f_{n})_{n\geq 1}$ itself. Clearly, on any bounded subset, $\hat{f}_{p}$ still coincides with $f$ for $p$ large enough. Moreover, the conclusion of the second step is preserved. In particular, the conclusion of the second step together with (80), (81) and (82) say that (A.5) holds (for a possible new choice of $c_{L}$). From now on, we get rid of the symbol “hat” in $(\hat{f}_{p})_{p\geq 1}$ and keep the notation $(f_{n})_{n\geq 1}$ for $(\hat{f}_{p})_{p\geq 1}$.

For $|x|\leq p/2$, we have $\rho(p^{-1}|x|)=|p^{-1}x|$, so that the right-hand side coincides with $-c_{L}(1+|x|)$. For $|x|\geq p/2$, we have $\rho(p^{-1}|x|)\geq 1/2$ so that

This proves that (A.7) holds with a new constant. $\Box$

References