Mean Field Forward-Backward Stochastic Differential Equations

Introduction

Stochastic differential equations of the McKean - Vlasov type are Itô’s stochastic differential equations where the coefficients depend upon the marginal distribution of the solution. In their partial differential form, they were introduced by Mark Kac’s in his analysis of the Boltzmann equation for the density of particules in kinetic theory of dilute monatomic gases, and a toy model for the Vlasov kinetic equation of plasma (see ).

The purpose of this note is to provide an existence result for the solution of Forward Backward Stochastic Differential Equations (FBSDEs) of the McKean-Vlasov type. Following the wave of interest created by the pathbreaking work of Lasry and Lions on mean field games , simple forms of Backward Stochastic Differential Equations (BSDEs) of McKean Vlasov type have been introduced and called of mean field type. Fully coupled FBSDEs are typically more involved and more difficult to solve than BSDEs. FBSDEs of mean field type occur naturally in the probabilistic analysis of mean field games and the optimal control of dynamics of the McKean Vlasov type as considered in . See also for the particular case of Linear Quadratic (LQ) models. Detailed explanations on how these FBSDEs occur in these contexts and the particular models which were solved are given in Section 3 below.

The existence proofs given in and depend heavily on the fact that the problems at hand are in fact stochastic control problems and FBSDEs are derived from an application of a version of the stochastic maximum principle, and the compactness estimates are derived from the linear nature of the forward dynamics and strong convexity properties of the cost functions of the stochastic optimization problems. The purpose of this note is to provide a general existence result which does not depend upon strong linearity and convexity assumptions. Such an existence result is proven in Section 2. The proof relies on Schauder’s fixed point theorem used in appropriate spaces of functions an measures. A short Section 3 concludes with a short discussion of applications to mean field games and control of McKean-Vlasov dynamics studied in and .

Solvability of Forward-Backward Systems of McKean-Vlasov Type

Notice that if $X$ and $X^{\prime}$ are random variables of order $2$ with values in $E$, then by definition we have

2. Assumptions and Statement of the Main Existence Result

Our goal is to solve fully coupled McKean-Vlasov forward-backward systems of the general form:

in the sense of symmetric matrices, where $I_{d}$ is the $d$-dimensional identity matrix. Here and throughout the paper, we use the exponent † to denote the transpose of a matrix. Moreover, the function $[0,T]\ni t\hookrightarrow\Sigma(t,0,0,\delta_{(0,0)})$ is also assumed to be continuous.

We can now state the main result of the paper.

Under (A1–3), the FBSDE (1) has a solution.

3. Preliminary

Our fixed point argument relies on the following lemma which puts together the existence and uniqueness result contained in Theorem 2.6 of Delarue and the control of the FBSDE value function provided by Corollary 2.8 of :

for some constant $\Gamma$ only depending upon $T$ and $L$. In particular, both $\gamma$ and $\Gamma$ are independent of $\nu^{\prime}$ and $\nu$. Finally, it holds $Y_{s}^{t,x}=u(s,X_{s}^{t,x})$ for any $t\leq s\leq T$.

On the top of (A1-3), let us also assume that $B$ and $F$ are bounded by $L$. Then, there exists a positive constant $\Gamma$, depending on $T$ and $L$ only, such that, for any inputs $(\varphi,\mu)$ and $(\varphi^{\prime},\mu^{\prime})$ as above, the processes $(X,Y,Z)$ and $(X^{\prime},Y^{\prime},Z^{\prime})$ obtained by solving (4) with $(\varphi,\mu)$ and $(\varphi^{\prime},\mu^{\prime})$ respectively, satisfy

For small time $T>0$, this estimate follows immediately from the main estimate Theorem 1.3 p. 218 of and the Lipschitz assumption (A1). We only need to show that one can extend it to arbitrarily large values of $T$. Notice that Lemma 1 gives the existence of the FBSDE values functions $u$ and $u^{\prime}$ such that $Y_{t}=u(t,X_{t})$ and $Y^{\prime}_{t}=u(t,X^{\prime}_{t})$ for all $t\in[0,T]$.

As in Corollary 2.8 of , we choose a regular subdivision $0=T_{0}<T_{1}<\cdots<T_{N-1}<T_{N}=T$ so that the common length of the intervals $[T_{i},T_{i+1}]$ is small enough in order to apply the main estimate Theorem 1.3 p. 218 of . For any $i\in\{0,\cdots,N-1\}$ we have:

We first consider the last interval $[T_{N-1},T_{N}]$ corresponding to the case $i=N-1$. Since $T_{N}=T$ we have $u(T,\cdot)=G(\cdot,\mu_{T})$ and $u^{\prime}(T,\cdot)=G(\cdot,\mu^{\prime}_{T})$ so that using the Lipschitz property of $G$ we get:

We can now plug this estimate into inequality (6) with $i=N-2$ to get:

Plugging this estimate into inequality (6) with $i=N-3$ we get:

Iterating and summing up these estimates we get (as before the value of the constants can change from line to line)

from which we get the desired estimate (5) after noticing that for each $i\geq 1$, we have:

We shall estimate the integral in the right hand side of (5) from the remark:

4. Fixed Point Argument in the Bounded Case

Similarly, since the drift $B$ and the volatility $\Sigma$ are uniformly bounded, the fourth moment of the supremum $\sup_{0\leq t\leq T}|X_{t}|$ is bounded by a constant depending only upon the bounds of $B$ and $\Sigma$. Consequently, we shall choose the input measure $\mu$ in the set:

and on $V_{2}$, the Kantorovitch-Rubinstein norm

We first check the continuity of $\Phi$. Given a sequence $(\varphi^{n},\mu^{n})$ in $E$ converging towards $(\varphi,\mu)\in E$ with respect to the product norm on $V_{1}\times V_{2}$, and given the corresponding solutions $(X^{n},Y^{n},Z^{n})$ and $(X,Y,Z)$ obtained by solving (4) with $(\varphi^{n},\mu^{n})$ and $(\varphi,\mu)$ respectively, we have (compare with (7)): $(i)$ for any $t\in[0,T]$, $W_{2}(\mu_{t},\mu^{n}_{t})\rightarrow 0$ as $n\rightarrow+\infty$ since $(\mu^{n}_{t})_{n\geq 1}$ converges weakly towards $\mu_{t}$ and the moments of order $4$ of the measures $(\mu^{n}_{t})_{n\geq 1}$ are uniformly bounded by $\gamma^{\prime}$; by boundedness of the moments of order 4 again, the integral with respect to $t$ of $W_{2}(\mu_{t},\mu^{n}_{t})$ converges towards ; $(ii)$ by continuity and boundedness of $\varphi$, and by a similar argument, the integral with respect to $t$ of $W_{2}(\varphi(t,\cdot)(\mu_{t}),\varphi(t,\cdot)(\mu_{t}^{n}))^{2}$ converges toward as $n\rightarrow+\infty$; $(iii)$ since the sup-norms of all the $\varphi^{n}$ are not greater than $\gamma$, the tightness of the measures $(\mu^{n})_{n\geq 1}$ together with the uniform convergence of $(\varphi^{n})_{n\geq 1}$ towards $\varphi$ on compact sets can be used to prove that

Similarly, $W_{2}(\mu_{T},\mu_{T}^{n})\rightarrow 0$ as $n$ tends to $+\infty$. From (6) and (5), we obtain

By Lemma 1, we know that all the mappings $(u^{n})_{n\geq 1}$ are Lipschitz continuous with respect to $x$, uniformly with respect to $n$. Therefore

We have completed all the steps needed to get a quick proof of the main result of this subsection.

Assume that, in additition to (A1–3), the coefficients $B$, $\Sigma$, $F$ and $G$ are bounded by $L$. Then equation (1) has a solution.

5. Relaxing the Boundedness Condition.

We now complete the proof of Theorem 1 when the coefficients only satisfy (A1–3). The proof consists in approximating $B$ and $F$ by sequences of bounded coefficients $(B^{n})_{n\geq 1}$ and $(F^{n})_{n\geq 1}$.

We notice first that the processes $(Y^{n})_{n\geq 1}$ are uniformly bounded by a constant that depends upon $L$ only. Indeed, applying Itô’s formula and using the specific growth condition (A2), we get:

Applying Itô’s formula to $|X^{n}|^{2}$, using the growth conditions (A2), the uniform boundedness of $Y^{n}_{t}$, the boundedness of $\Sigma$ and the bound (10), we can use Gronwall’s lemma and get the existence of a finite constant $C$ such that, for any $n\geq 1$,

Using the bound (A2) for $B^{n}$ and the same constant $L$, together with the uniform boundedness of the paths $Y^{n}_{t}$, (10) and (11), it is easy to check that

which shows that the sequence $(\underline{X}^{n_{k}})_{k\geq 1}$ is a Cauchy sequence. We denote by $\underline{X}$ the limit. Since for each $n\geq 1$ we have:

6. Counter-Example to Uniqueness

We close this section with a counter-example showing that uniqueness cannot hold in general under assumptions (A1–3), even in the case $d=m=p=1$. Indeed, let us consider the forward-backward system

with $x_{0}=0$ as initial condition. Therefore, for $|A|\leq R$, $(A\sin(t),A\cos(t))_{0\leq t\leq T}$ is a solution to the deterministic forward-backward system

with $x_{0}=0$ as initial condition. For such a value of $A$, set now:

The reason for the failure of uniqueness can be explained as follows. In the standard framework, as explained in , uniqueness holds because of the smoothing effect of the diffusion operator in the spatial direction. However, in the McKean-Vlasov setting, the smoothing effect of the diffusion operator is ineffective in the direction of the measure variable.

Applications

For the sake of completeness we assume that, in both applications, the drift coefficient $b$ and the volatility matrix $\sigma$ satisfy the following assumptions.

The functions $(b(t,0,\delta_{0},0))_{0\leq t\leq T}$ and $(\sigma(t,0,\delta_{0},0))_{0\leq t\leq T}$ are continuous;

for every $q\geq 1$. See for example for a proof for the McKean-Vlasov equations.

In this subsection, we apply the abstract existence result of Section 2 to the Mean Field Game (MFG) problem described in the introduction. As in , we assume that the volatility function is a constant matrix $\sigma$ (of size $d\times m$), but here we assume in addition that ${\rm det}(\sigma\sigma^{\dagger})>0$. Since the stochastic optimization problem is solved after the flow of measures is frozen, for each fixed $\mu$, the Hamiltonian of the system reads:

The existence of such a function was proven in under specific assumptions on the drift $b$ and the running cost function $f$ used there. Using a standard version of the stochastic maximum principle, and coupling the forward dynamics and the adjoint BSDE by plugging the optimizer (19) lead to the solution, for each frozen flow $\underline{\mu}=(\mu_{t})_{0\leq t\leq T}$ of measures, of a standard FBSDE. Now, if we add the requirement that $\mu_{t}$ should coincide for each $t$ with the marginal distribution of the optimally controlled state, the solution of the MFG stochastic optimization problem reduces to the solution of the FBSDE of McKean - Vlasov type

While the result of the present note provides existence in quite a general set up, also allows for running costs with at most linear growth in $x$ (and provides much more in terms of the identification of approximate Nash equilibriums). However, the drift $b$ needs to be of a very specific affine form, namely $b(t,x,y,\alpha,\mu)=b_{0}(t,\mu)+b_{1}(t)x+b_{2}(t)\alpha$ for some deterministic functions $b_{0}$, $b_{1}$ and $b_{2}$, and the running cost function $f$ has to satisfy a strong convexity assumption, the latter allowing $\sigma$ to be degenerate, providing a more efficient approximation procedure to reduce the result to the bounded case and ensuring the validity of the converse of the stochastic maximum principle.

2. Optimal Control of McKean-Vlasov Stochastic Dynamics

Finally, we explain how the existence result of this paper generalizes the existence result of where a solution of the optimal control of stochastic differential equations of the McKean-Vlasov type is given.

As before, we assume that the volatility function is a constant matrix $\sigma$ such that ${\rm det}(\sigma\sigma^{\dagger})>0$, so the Hamiltonian has the form

The existence of such a function was proven in under specific assumptions on the drift $b$ and the running cost function $f$ used there. However, the major difference with the mean field game problem comes from the form of the adjoint equation which now involves differentiation of the Hamiltonian with respect to the measure parameter. A special form of adjoint equation was introduced, and a new stochastic maximum principle was proven in . Once this new form of adjoint equation is coupled with the forward dynamical equation through the plugged-in optimal control feedback $\hat{\alpha}$ defined in (21), the associated McKean-Vlasov FBSDE takes the form

References