Entropy and the fourth moment phenomenon

Introduction

The study of the classical CLT for sums of independent random elements by entropic methods dates back to Linnik’s seminal paper . Among the many fundamental contributions to this line of research, we cite (see the monograph for more details on the history of the theory). All these influential works revolve around a deep analysis of the effect of analytic convolution on the creation of entropy: in this respect, a particularly powerful tool are the ‘entropy jump inequalities’ proved and exploited e.g. in . As discussed e.g. in , entropy jump inequalities are directly connected with challenging open questions in convex geometry, like for instance the Hyperplane and KLS conjectures. One of the common traits of all the above references is that they develop tools to control the Fisher information and use the aforementioned de Bruijn’s formula to translate the bounds so obtained into bounds on the relative entropy.

One of the main motivations of the present paper is to initiate a systematic information-theoretical analysis of a large class of CLTs that has emerged in recent years in connection with different branches of modern stochastic analysis. These limit theorems typically involve: (a) an underlying infinite dimensional Gaussian field ${\bf G}$ (like for instance a Wiener process), (b) a sequence of rescaled centered random vectors $F_{n}=F_{n}({\bf G})$, $n\geqslant 1$, having the form of some highly non-linear functional of the field ${\bf G}$. For example, each $F_{n}$ may be defined as some collection of polynomial transformations of ${\bf G}$, possibly depending on a parameter that is integrated with respect to a deterministic measure (but much more general forms are possible). Objects of this type naturally appear e.g. in the high-frequency analysis of random fields on homogeneous spaces , fractional processes , Gaussian polymers , or random matrices .

In view of their intricate structure, it is in general not possible to meaningfully represent the vectors $F_{n}$ in terms of some linear transformation of independent (or weakly dependent) vectors, so that the usual analytical techniques based on stochastic independence and convolution (or mixing) cannot be applied. To overcome these difficulties, a recently developed line of research (see for an introduction) has revealed that, by using tools from infinite-dimensional Gaussian analysis (e.g. the so-called Malliavin calculus of variations – see ) and under some regularity assumptions on $F_{n}$, one can control the distance between the distribution of $F_{n}$ and that of some Gaussian target by means of quantities that are no more complex than the fourth moment of $F_{n}$. The regularity assumptions on $F_{n}$ are usually expressed in terms of the projections of each $F_{n}$ on the eigenspaces of the Ornstein-Uhlenbeck semigroup associated with ${\bf G}$ .

The estimate (1.1) is obtained by combining the Malliavin calculus of variations with the Stein’s method for normal approximations . Stein’s method can be roughly described as a collection of analytical techniques, allowing one to measure the distance between random elements by controlling the regularity of the solutions to some specific ordinary (in dimension 1) or partial (in higher dimensions) differential equations. The needed estimates are often expressed in terms of the same Stein factors that lie at the core of the present paper (see Section 2.3 for definitions). It is important to notice that the strength of these techniques significantly breaks down when dealing with normal approximations in dimension strictly greater than $1$.

For instance, in view of the structure of the associated PDEs, for the time being there is no way to directly use Stein’s method in order to deduce bounds in the multidimensional total variation distance. (This fact is demonstrated e.g. in references , where multidimensional bounds are obtained for distances involving smooth test functions, as well as in , containing a quantitative version of the classical multidimensional CLT, with bounds on distances involving test functions that are indicators of convex sets: all these papers use some variations of the multidimensional Stein’s method, but none of them achieves bounds in the total variation distance.)

where $\|\cdot\|$ stands for the Euclidean norm, and assume that $\Delta_{n}\to 0$, as $n\to\infty$. Then,

where $O(1)$ stands for a bounded numerical sequence, depending on $d,q_{1},...,q_{d}$ and on the sequence $\{F_{n}\}$.

As in the one-dimensional case, one has always that $\Delta_{n}>0$ for $f_{n}$ as in the previous statement. The quantity of the left-hand-side of (1.2) equals of course the relative entropy of $f_{n}$. In view of the Csiszar-Kullback-Pinsker inequality (see ), according to which

relation (1.2) then translates in a bound on the square of the total variation distance between $f_{n}$ and $\phi_{d}$, where the dependence in $\Delta_{n}$ hinges on the order of the chaoses only via a multiplicative constant. This bound agrees up to a logarithmic factor with the estimates in smoother distances established in (see also ), where it is proved that there exists a constant $K_{0}=K_{0}(d,q_{1},...,q_{d})$ such that

where ${\bf W}_{1}$ stands for the usual Wasserstein distance of order 1. Relation (1.2) also drastically improves the bounds that can be deduced from , yielding that, as $n\to\infty$,

where $\alpha_{d}$ is any strictly positive number verifying ${\alpha_{d}<\frac{1}{1+(d+1)(3+4d(q_{d}-1))}}$, and the symbol $O(1)$ stands again for some bounded numerical sequence. The estimate (1.2) seems to be largely outside the scope of any other available technique. Our results will also show that convergence in relative entropy is a necessary and sufficient condition for CLTs involving random vectors whose components live in a fixed Wiener chaos. As in , an important tool for establishing our main results is the Carbery-Wright inequality , providing estimates on the small ball probabilities associated with polynomial transformations of Gaussian vectors. Observe also that, via the Talagrand’s transport inequality , our bounds trivially provide estimates on the 2-Wasserstein distance ${\bf W}_{2}(f_{n},\phi_{d})$ between $f_{n}$ and $\phi_{d}$, for every $d\geqslant 1$. Notice once again that, in Theorem 1.1 and its generalisations, no additional regularity (a part from the fact of being elements of a fixed Wiener chaos) is required from the components of the vector $F_{n}$. One should contrast this situation with the recent work by Hu, Lu and Nualart , where the authors achieve fourth moment bounds on the supremum norm of the difference $f_{n}-\phi_{d}$, under very strong additional conditions expressed in terms of the finiteness of negative moments of Malliavin matrices.

We stress that, although our principal motivation comes from asymptotic problems on a Gaussian space, the methods developed in Section 2 are general. In fact, at the heart of the present work lie the powerful equivalences (2.45)– (2.46) (which can be considered as a new form of so-called Stein identities) that are valid under very weak assumptions on the target density; it is also easy to uncover a wide variety of extensions and generalizations so that we expect that our tools can be adapted to deal with a much wider class of multidimensional distributions.

The connection between Stein identities and information theory has already been noted in the literature (although only in dimension 1). For instance, explicit applications are known in the context of Poisson and compound Poisson approximations , and recently several promising identities have been discovered for some discrete as well as continuous distributions . However, with the exception of , the existing literature seems to be silent about any connection between entropic CLTs and Stein’s identities for normal approximations. To the best of our knowledge, together with (which however focusses on bounds of a completely different nature) the present paper contains the first relevant study of the relations between the two topics.

In order to simplify the discussion, we shall sometimes use the shorthand notation

To enhance the readability of the text, the next Subsection 1.2 contains an intuitive description of our method in dimension one.

2 Illustration of the method in dimension one

Recall also that, in view of the Pinsker-Csiszar-Kullback inequality, one has that

for every smooth test function $g$. Specifying $g(x)=x$ implies, in particular, that $E[\tau_{F}(F)]=E[F^{2}]=1$. It is easily seen that, under standard regularity assumptions, a version of $\tau_{F}$ is given by $\tau_{F}(x)=(f(x))^{-1}\int_{x}^{\infty}zf(z)dz$, for $x$ in the support of $f$ (in particular, the Stein factor of $Z$ is 1). The relevance of the factor $\tau_{F}$ in comparing $F$ with $Z$ is actually revealed by the following Stein’s bound , which is one of the staples of Stein’s method:

providing a formal meaning to the intuitive fact that the distributions of $F$ and $Z$ are close whenever $\tau_{F}$ is close to $\tau_{Z}$, that is, whenever $\tau_{F}$ is close to 1. To motivate the reader, we shall now present a simple illustration of how the estimate (1.7) applies to the usual CLT.

Let $\{F_{i}:i\geqslant 1\}$ be a sequence of i.i.d. copies of $F$, set $S_{n}=n^{-1/2}\sum_{i=1}^{n}F_{i}$ and assume that $E[\tau_{F}(F)^{2}]<+\infty$ (a simple sufficient condition for this to hold is e.g. that $f$ has compact support, and $f$ is bounded from below inside its support). Then, using e.g. [51, Lemma 2],

Since (by definition) $E\left[\tau_{S_{n}}(S_{n})\right]=E\left[\tau_{F}(F_{i})\right]=1$ for all $i=1,\ldots,n$ we get

In particular, writing $f_{n}$ for the density of $S_{n}$, we deduce from (1.7) that

We shall demonstrate in Section 3 and Section 4 that the quantity $E[|1-\tau_{F}(F)|]$ (as well as its multidimensional generalisations) can be explicitly controlled whenever $F$ is a smooth functional of a Gaussian field. In view of these observations, the following question is therefore natural: can one bound $D(F\|Z)$ by an expression analogous to the right-hand-side of (1.7)?

for every smooth test function $g$. We also write, for $t\in[0,1)$,

for the Fisher information of $F_{t}$, and we observe that

where $J_{st}(F_{t})$ is the so-called standardised Fisher information of $F_{t}$ (note that $J_{st}(F_{0})=J_{st}(Z)=0$). With this notation in mind, de Bruijn’s formula (in an integral and rescaled version due to Barron ) reads

(see Lemma 2.3 below for a multidimensional statement).

Using the standard relation $J_{st}(F_{t})\leqslant tJ_{st}(F)+(1-t)J_{st}(Z)=tJ_{st}(F)$ (see e.g. [21, Lemma 1.21]), we deduce the upper bound

a result which is often proved by using entropy power inequalities (see also Shimizu ). Formula (1.13) is a quantitative counterpart to the intuitive fact that the distributions of $F$ and $Z$ are close, whenever $J_{st}(F)$ is close to zero. Using (1.5) we further deduce that closeness between the Fisher informations of $F$ and $Z$ (i.e. $J_{st}(F)\approx 0$) or between the entropies of $F$ and $Z$ (i.e. $D(F||Z)\approx 0$) both imply closeness in terms of the total variation distance, and hence in terms of many more probability metrics. This observation lies at the heart of the approach from where a fine analysis of the behavior of $\rho_{F}(F)$ over convolutions (through projection inequalities in the spirit of (1.8)) is used to provide explicit bounds on the Fisher information distance which in turn are transformed, by means of de Bruijn’s identity (1.12), into bounds on the relative entropy. We will see in Section 4 that the bound (1.13) is too crude to be of use in the applications we are interested in.

Let the previous notation prevail. We have

Proof. Using $\rho_{Z}(Z)=-Z$ we see that, for any function $g\in C^{1}_{c}$, one has

and the desired conclusion follows from de Bruijn’s identity (1.12).

To properly control the integral on the right-hand-side of (1.14), we need to deal with the fact that the mapping $t\mapsto\frac{t}{1-t}$ is not integrable in $t=1$, so that we cannot directly apply the estimate E\big{[}E[Z(1-\tau_{F}(F))|F_{t}]^{2}\big{]}\leqslant{\rm Var}(\tau_{F}(F)) to deduce the desired bound. Intuitively, one has to exploit the fact that the mapping $t\mapsto E[Z(1-\tau_{F}(F))|F_{t}]$ satisfies $E[Z(1-\tau_{F}(F))|F_{1}]=0$, thus in principle compensating for the singularity at $t\approx 1$.

As we will see below, one can make this heuristic precise provided there exist three constants $c,\delta,\eta>0$ such that

Under the assumptions appearing in condition (1.16), the following strategy can indeed be implemented in order to deduce a satisfactory bound. First split the integral in two parts: for every $0<\varepsilon\leqslant 1$,

the last inequality being a consequence of $\int_{0}^{1-\varepsilon}\frac{t\,dt}{1-t}=\int_{\varepsilon}^{1}\frac{(1-u)du}{u}\leqslant\int_{\varepsilon}^{1}\frac{du}{u}=-\log\varepsilon$. To deal with the second term in (1.17), let us observe that, by using in particular the Hölder inequality and the convexity of the function $x\mapsto|x|^{\eta+2}$, one deduces from (1.16) that

By virtue of (1.17) and (1.18), the term $D(F\|Z)$ is eventually amenable to analysis, and one obtains:

Assuming finally that $E[(1-\tau_{F}(F))^{2}]\leqslant 1$ (recall that, in the applications we are interested in, such a quantity is meant to be close to 0) we can optimize over $\varepsilon$ and choose $\varepsilon=E[(1-\tau_{F}(F))^{2}]^{\frac{\eta+1}{\delta\eta}}$, which leads to

Clearly, combining (1.19) with (1.5), one also obtains an estimate in total variation which agrees with (1.7) up to the square root of a logarithmic factor.

The problem is now how to identify sufficient conditions on the law of $F$ for (1.16) to hold; we shall address this issue by means of two auxiliary results. We start with a useful technical lemma, that has been suggested to us by Guillaume Poly.

where the supremum is taken over all $g\in C^{1}_{c}$ such that $\|g\|_{\infty}\leq 1$.

Proof. Since $|\mbox{sign}(E[X|Y])|=1$ we have, by using e.g. Lusin’s Theorem,

To see the reversed inequality, observe that, for any $g$ bounded by 1,

Then (1.16) holds, with $\delta=\frac{1}{2}\wedge\alpha$ and $c=4(\kappa+1)$.

Proof. Take $g\in C^{1}_{c}$ such that $\|g\|_{\infty}\leqslant 1$. Then, by independence of $Z$ and $F$,

so that, since $\|g\|_{\infty}\leqslant 1$ and $E|F|\leqslant\sqrt{E[F^{2}]}=1$,

Inequality (1.16) now follows by applying Lemma 1.5.

As anticipated, in Section 4 (see Lemma 4.4 for a precise statement) we will describe a wide class of distributions satisfying (1.21). The previous discussion yields finally the following statement, answering the original question of providing a bound on $D(F\|Z)$ that is comparable with the estimate (1.7).

then, provided $\Delta:=E[(1-\tau_{F}(F))^{2}]\leqslant 1$,

3 Plan

The rest of the paper is organised as follows. In Section 2 we will prove that Theorem 1.7 can be generalised to a fully multidimensional setting. Section 3 contains some general results related to (infinite-dimensional) Gaussian stochastic analysis. Finally, in Section 4 we shall apply our estimates in order to deduce general bounds of the type appearing in Theorem 1.1.

Entropy bounds via de Bruijn’s identity and Stein matrices

In Section 2.1 and Section 2.2 we discuss some preliminary notions related to the theory of information (definitions, notations and main properties). Section 2.3 contains the proof of a new integral formula, allowing one to represent the relative entropy of a given random vector in terms of a Stein matrix. The reader is referred to the monograph , as well as to [1, Chapter 10], for any unexplained definition and result concerning information theory.

As discussed above, we are interested in estimating the distance between the law of $F$ and the law of a $d$-dimensional centered Gaussian vector $Z=(Z_{1},...,Z_{d})\sim\mathscr{N}_{d}(0,C)$, where $C>0$ is the associated covariance matrix. Our measure of the discrepancy between the distributions of $F$ and $Z$ is the relative entropy (often called Kullback-Leibler divergence or information entropy)

where $\phi=\phi_{d}(\cdot\,;C)$ is the density of $Z$. It is easy to compute the Gaussian entropy $\mbox{Ent}(Z)={1}/{2}\log\left((2\pi e)^{d}|C|\right)$ (where $|C|$ is the determinant of $C$), from which we deduce the following alternative expression for the relative entropy

where ‘tr’ stands for the usual trace operator. If $Z$ and $F$ have the same covariance matrix then the relative entropy is simply the entropy gap between $F$ and $Z$ so that, in particular, one infers from (2.26) that $Z$ has maximal entropy among all absolutely continuous random vectors with covariance matrix $C$.

We stress that the relative entropy $D$ does not define a bona fide probability distance (for absence of a triangle inequality, as well as for lack of symmetry): however, one can easily translate estimates on the relative entropy in terms of the total variation distance, using the already recalled Pinsker-Csiszar-Kullback inequality (1.3). In the next subsection, we show how one can represent the quantity $D(F||Z)$ as the integral of the standardized Fisher information of some adequate interpolation between $F$ and $Z$.

2 Fisher information and de Bruijn’s identity

(with components $J(F_{t})_{ij}=E\left[\rho_{t,i}(F_{t})\rho_{t,j}(F_{t})\right]$ for $1\leq i,j\leq d$), and is customarily called the Fisher information matrix of $F_{t}$. Focussing on the case $t=0$, one sees immediately that the Gaussian vector $F_{0}=Z\sim\mathscr{N}_{d}(0,C)$ has linear score function $\rho_{0}({\bf x})=\rho_{Z}({\bf x})=-C^{-1}{\bf x}$ and Fisher information $J(F_{0})=J(Z)=C^{-1}$.

Fix $t\in[0,1)$. Using formula (2.28) one deduces that a version of $\rho_{t}(F_{t})$ is given by the conditional expectation $-(1-t)^{-1/2}E[C^{-1}Z|F_{t}]$, from which we infer that the matrix $J(F_{t})$ is well-defined and its entries are all finite.

For $t\in[0,1)$, we define the standardized Fisher information matrix of $F_{t}$ as

where $I_{d}$ is the $d\times d$ identity matrix, and the last equality holds because $E\left[\rho_{t}(F_{t})F_{t}\right]=-I_{d}$. Note that the positive semidefinite matrix $\Gamma_{t}^{-1}J_{st}(F_{t})=J(F_{t})-\Gamma_{t}^{-1}$ is the difference between the Fisher information matrix of $F_{t}$ and that of a Gaussian vector having distribution $\mathscr{N}_{d}(0,\Gamma_{t})$. Observe that

is completely characterized (up to sets of $P$-measure 0) by the equation

holding for every smooth test function $g$.

Of course the above information theoretic quantities are not defined only for Gaussian mixtures of the form $F_{t}$ but more generally for any random vector satisfying the relevant assumptions (which are necessarily verified by the $F_{t}$). In particular, if $F$ has covariance matrix $B$ and differentiable density $f$ then, letting $\rho_{F}(\mathbf{x}):=\nabla\log f({\mathbf{x}})$ be the score function for $F$, the standardized Fisher information of $F$ is

The following fundamental result is known as the (multidimensional) de Bruijn’s identity: it shows that the relative entropy $D(F||Z)$ can be represented in terms of the integral of the mapping $t\mapsto{\rm tr}(C\Gamma_{t}^{-1}J_{st}(F_{t}))$ with respect to the measure $dt/2t$ on $(0,1]$. It is one of the staples of the entire paper. We refer the reader e.g. to for proofs in the case $d=1$. Our multidimensional statement is a rescaling of [23, Theorem 2.3] (some more details are given in the proof). See also .

Let the above notation and assumptions prevail. Then,

Proof. In [23, Theorem 2.3] it is proved that

(note that the definition of standardized Fisher information used in is different from ours). The conclusion is obtained by using the change of variables $t=(1+\tau)^{-1}$, as well as the fact that

which follows from the scale-invariance of standardized Fisher information mentionned in Remark 2.2.

Assume that $C_{n}$, $n\geqslant 1$, is a sequence of $d\times d$ nonsingular covariance matrices such that $C_{n;i,j}\to B_{i,j}$ for every $i,j=1,...,d$, as $n\to\infty$. Then, the second and third summands of (2.35) (with $C_{n}$ replacing $C$) converge to 0 as $n\to\infty$.

For future reference, we will now rewrite formula (2.35) for some specific choices of $d$, $F$, $B$ and $C$.

Assume $F\sim\mathscr{N}_{d}(0,B)$. Then, $J_{st}(F_{t})=0$ (null matrix) for every $t\in[0,1)$, and formula (2.35) becomes

Assume that $d=1$ and that $F$ and $Z$ have variances $b,c>0$, respectively. Defining $\gamma_{t}=tb+(1-t)c$, relation (2.35) becomes

Relation (2.37) in the case $b=c$ $(=\gamma_{t})$ corresponds to the integral formula (1.12) proved by Barron in [8, Lemma 1].

In the special case where $B=C=I_{d}$, one has that

of which (1.12) is a particular case ($d=1$).

In the univariate setting, the general variance case ($E\left[F^{2}\right]=\sigma^{2}$) follows trivially from the standardized one ($E\left[F^{2}\right]=1$) through scaling; the same cannot be said in the multivariate setting since the appealing form (2.39) cannot be directly achieved for $d\geq 2$ when the covariance matrices are not the identity because here the dependence structure of $F$ needs to be taken into account. In Lemma 2.6 we provide an estimate allowing one to deal with this difficulty in the case $B=C$, for every $d$. The proof is based on the following elementary fact: if $A,B$ are two $d\times d$ symmetric matrices, and if $A$ is semi-positive definite, then

where $\lambda_{\min}(B)$ and $\lambda_{\max}(B)$ stand, respectively, for the maximum and minimum eigenvalue of $B$. Observe that $\lambda_{\max}(B)=\|B\|_{op}$, the operator norm of $B$.

Fix $d\geqslant 1$, and assume that $B=C$. Then, $C\Gamma_{t}^{-1}=I_{d}$, and one has the following estimates

Proof. Write ${\rm tr}\left(J_{st}(F_{t})\right)={\rm tr}\left(C^{-1}J_{st}(F_{t})C\right)$ and apply (2.40) first to $A=C^{-1}J_{st}(F_{t})$ and $B=C$, and then to $A=J_{st}(F_{t})C$ and $B=C^{-1}$.

In the next section, we prove a new representation of the quantity $\rho_{t}(F_{t})+C^{-1}F_{t}$ in terms of Stein matrices: this connection will provide the ideal framework in order to deal with the normal approximation of general random vectors.

3 Stein matrices and a key lemma

The centered $d$-dimensional vectors $F,Z$ are defined as in the previous section (in particular, they are stochastically independent).

The entries of the random matrix $\tau_{F}(F)$ are called the Stein factors of $F$.

Selecting $g(F)=F_{j}$ in (2.43), one deduces that, if $\tau_{F}$ is a Stein matrix for $F$, then $E[\tau_{F}(F)]=B$. More to this point, if $F\sim\mathscr{N}_{d}(0,B)$, then the covariance matrix $B$ is itself a Stein matrix for $F$. This last relation is known as the Stein’s identity for the multivariate Gaussian distribution.

Assume that $d=1$ and that $F$ has density $f$ and variance $b>0$. Then, under some standard regularity assumptions, it is easy to see that $\tau_{F}(x)=b\int_{x}^{\infty}yf(y)dy/f(x)$ is a Stein factor for $F$.

Let the above notation and framework prevail, and assume that $\tau_{F}$ is a Stein matrix for $F$ such that $\tau_{F}^{i,j}(F)\in L^{1}(\Omega)$ for every $i,j=1,...,d$. Then, for every $t\in[0,1)$, the mapping

is a version of the score $\rho_{t}$ of $F_{t}$. Also, the mapping

is a version of the function $\rho^{\star}_{t}$ defined in formula (2.32).

Proof. Remember that $-C^{-1}Z$ is the score of $Z$, and denote by ${\bf x}\mapsto A_{t}({\bf x})$ the mapping defined in (2.45). Removing the conditional expectation and exploiting the independence of $F$ and $Z$, we infer that, for every smooth test function $g$,

To simplify the forthcoming discussion, we shall use the shorthand notation: $\widetilde{Z}=(\widetilde{Z}_{1},...,\widetilde{Z}_{d}):=C^{-1}Z\sim\mathscr{N}_{d}(0,C^{-1})$, $\widetilde{F}=(\widetilde{F}_{1},...,\widetilde{F}_{d}):=C^{-1}F$, and $\widetilde{\tau}_{F}=\{\widetilde{\tau}^{i,j}_{F}:i,j=1,...,d\}:=C^{-1}\tau_{F}$. The following statement is the main achievement of the section, and is obtained by combining Lemma 2.9 with formulae (2.38) and (2.41)–(2.42), in the case where $C=B$.

Let the above notation and assumptions prevail, assume that $B=C$, and introduce the notation

The next subsection focusses on general bounds based on the estimates (2.47)–(2.51).

4 A general bound

The following statement provides the announced multidimensional generalisation of Theorem 1.7. In particular, the main estimate (2.55) provides an explicit quantitative counterpart to the heuristic fact that, if there exists a Stein matrix $\tau_{F}$ such that $\|\tau_{F}-C\|_{H.S.}$ is small (with $\|\cdot\|_{H.S.}$ denoting the usual Hilbert-Schmidt norm), then the distribution of $F$ and $Z\sim\mathscr{N}_{d}(0,C)$ must be close. By virtue of Theorem 2.10, the proximity of the two distributions is expressed in terms of the relative entropy $D(F\|Z)$.

Proof. Take $g\in C^{1}_{c}$ such that $\|g\|_{\infty}\leqslant 1$. Then, by independence of $\widetilde{Z}$ and $F$ and using (2.44), one has, for any $j=1,\ldots,d$,

As a result, due to Lemma 1.5 and since $E|F_{j}|\leqslant\sqrt{E[F_{j}^{2}]}\leqslant\sqrt{\max_{j}C(j,j)}$, one obtains

Now, using among others the Hölder inequality and the convexity of the function $x\mapsto|x|^{\eta+2}$, we have that

with $C_{d,\eta,\tau}$ given by (2.56). At this stage, we shall use the key identity (2.50). To properly control the right-hand-side of (2.48), we split the integral in two parts: for every $0<\varepsilon\leqslant\frac{1}{2}$,

the second inequality being a consequence of (2.57) as well as $\int_{0}^{1-\varepsilon}\frac{t\,dt}{1-t}=\int_{\varepsilon}^{1}\frac{(1-u)du}{u}\leqslant\int_{\varepsilon}^{1}\frac{du}{u}=-\log\varepsilon$. Since (2.54) holds true, one can optimize over $\varepsilon$ and choose $\varepsilon=\Delta^{\frac{\eta+1}{\alpha\eta}}$, which leads to the desired estimate (2.55).

5 A general roadmap for proving quantitative entropic CLTs

In Section 4, we will apply the content of Theorem 2.11 to deduce entropic fourth moment theorems on a Gaussian space. As demonstrated in the discussion to follow, this achievement is just one possible application of a general strategy, leading to effective entropic estimates by means of Theorem 2.11.

Description of the strategy. Fix $d\geqslant 1$, and consider a sequence $F_{n}=(F_{1,n},...,F_{d,n})$, $n\geqslant 1$, of centered random vectors with covariance $C_{n}>0$ and such that $F_{n}$ has a density for every $n$. Let $Z_{n}\sim\mathscr{N}_{d}(0,C_{n})$, $n\geqslant 1$, be a sequence of Gaussian vectors, and assume that $C_{n}\to C>0$, as $n\to\infty$. Then, in order to show that $D(F_{n}\|Z_{n})\to 0$ one has to accomplish the following steps:

Write explicitly the Stein matrix $\tau_{F_{n}}$ for every $n$. This task can be realised either by exploiting the explicit form of the density of $F_{n}$, or by applying integration by parts formulae, whenever they are available. This latter case often leads to a representation of the Stein matrix by means of a conditional expectation. As an explicit example and as shown in Section 3.3 below, Stein matrices on a Gaussian space can be easily expressed in terms of Malliavin operators.

Check that, for some $\eta>0$, the quantity E\big{[}|\tau_{F_{n}}^{j,k}(F_{n})|^{\eta+2}\big{]} is bounded by a finite constant independent of $n$. This step typically requires one to show that the sequence $\{\tau_{F_{n}}^{j,k}(F_{n}):n\geqslant 1\}$ is bounded in $L^{2}(\Omega)$, and moreover that the elements of this sequence verify some adequate hypercontractivity property. We will see in Section 4 that, when considering random variables living inside a finite sum of Wiener chaoses, these properties are implied by the explicit representation of Stein matrices in terms of Malliavin operators, as well as by the fundamental inequality stated in Proposition 3.4.

Prove a total variation bound such as (2.53) for every $F_{n}$, with a constant $\kappa$ independent of $n$. This is arguably the most delicate step, as it consists in an explicit estimate of the total variation distance between the law of $F_{n}$ and that of $\sqrt{t}F_{n}+\sqrt{1-t}\,{\bf x}$, when $t$ is close to one. It is an interesting remark that, for every $i=1,...,d$ (as $t\to 1$), $E[(F_{i,n}-(\sqrt{t}F_{i,n}+\sqrt{1-t}\,x_{i}))^{2}]=O(1-t)$, so that an estimate such as (2.53) basically requires one to explicitly relate the total variation and mean-square distances between $F_{n}$ and $\sqrt{t}F_{n}+\sqrt{1-t}\,{\bf x}$. We will see in Section 4 that, for random variables living inside a fixed sum of Wiener chaoses, adequate bounds of this type can be deduced by applying the Carbery-Wright inequalities , that we shall combine with the approach developed by Nourdin, Nualart and Poly in .

Show that $E\|\tau_{F_{n}}-C_{n}\|^{2}\to 0$. This is realised by using the explicit representation of the Stein matrices $\tau_{F_{n}}$.

Once steps (a)–(d) are accomplished, an upper bound on the speed of convergence to zero of the sequence $D(F_{n}\|Z_{n})$ can be deduced from (2.55), whereas the total variation distance between the laws of $F_{n}$ and $Z$ follows from the inequality

Note that the quantity $D(Z\|Z_{n})$ can be assessed by using (2.36).

Before proceeding to the application of the above roadmap on Gaussian space we now first provide, in the forthcoming Section 3, the necessary preliminary results about Gaussian stochastic analysis.

Gaussian spaces and variational calculus

As announced, we shall now focus on random variables that can be written as functionals of a countable collection of independent and identically distributed Gaussian $\mathscr{N}(0,1)$ random variables, that we shall denote by

Note that our description of ${\bf G}$ is equivalent to saying that ${\bf G}$ is a Gaussian sequence such that $E[G_{i}]=0$ for every $i$ and $E[G_{i}G_{j}]={\bf 1}_{\{i=j\}}$. We will write $L^{2}(\sigma({\bf G})):=L^{2}(P,\sigma({\bf G}))$ to indicate the class of square-integrable (real-valued) random variables that are measurable with respect to the $\sigma$-field generated by ${\bf G}$.

The reader is referred e.g. to for any unexplained definition or result appearing in the subsequent subsections.

We will now briefly introduce the notion of Wiener chaos.

The sequence of Hermite polynomials $\{H_{m}:m\geqslant 0\}$ is defined as follows: $H_{0}=1$, and, for $m\geqslant 1$,

A multi-index $\alpha=\{\alpha_{i}:i\geqslant 1\}$ is a sequence of nonnegative integers such that $\alpha_{i}\neq 0$ only for a finite number of indices $i$. We use the symbol $\Lambda$ in order to indicate the collection of all multi-indices, and use the notation $|\alpha|=\sum_{i\geqslant 1}\alpha_{i}$, for every $\alpha\in\Lambda$.

It is easily seen that two random variables belonging to Wiener chaoses of different orders are orthogonal in $L^{2}(\sigma({\bf G}))$. Moreover, since linear combinations of polynomials are dense in $L^{2}(\sigma({\bf G}))$, one has that $L^{2}(\sigma({\bf G}))=\bigoplus_{q\geqslant 0}C_{q}$, that is, any square-integrable functional of ${\bf G}$ can be written as an infinite sum, converging in $L^{2}$ and such that the $q$th summand is an element of $C_{q}$. This orthogonal decomposition of $L^{2}(\sigma({\bf G}))$ is customarily called the Wiener-Itô chaotic decomposition of $L^{2}(\sigma({\bf G}))$.

where $\Phi(\alpha)$ is given in (3.59). Another classical result (see e.g. ) is that, for every $q\geqslant 1$, the mapping $I_{q}:\mathfrak{H}^{\odot q}\rightarrow C_{q}$ (as defined in (3.60)) is onto, and defines an isomorphism between $C_{q}$ and the Hilbert space $\mathfrak{H}^{\odot q}$, endowed with the modified norm $\sqrt{q!}\|\cdot\|_{\mathfrak{H}^{\otimes q}}$. This means that, for every $h,h^{\prime}\in\mathfrak{H}^{\odot q}$, $E[I_{q}(h)I_{q}(h^{\prime})]=q!\langle h,h^{\prime}\rangle_{\mathfrak{H}^{\otimes q}}.$

Finally, we observe that one can reexpress the Wiener-Itô chaotic decomposition of $L^{2}(\sigma({\bf G}))$ as follows: every $F\in L^{2}(\sigma({\bf G}))$ admits a unique decomposition of the type

where the series converges in $L^{2}({\bf G})$, the symmetric kernels $h_{q}\in\EuFrak{H}^{\odot q}$, $q\geqslant 1$, are uniquely determined by $F$, and $I_{0}(h_{0}):=E[F]$. This also implies that

2 The language of Malliavin calculus: chaoses as eigenspaces

We let the previous notation and assumptions prevail: in particular, we shall fix for the rest of the section a real separable Hilbert space $\EuFrak{H}$, and represent the elements of the $q$th Wiener chaos of ${\bf G}$ in the form (3.60). In addition to the previously introduced notation, $L^{2}(\EuFrak{H}):=L^{2}(\sigma({\bf G});\EuFrak{H})$ indicates the space of all $\EuFrak{H}$-valued random elements $u$, that are measurable with respect to $\sigma({\bf G})$ and verify the relation E\big{[}\|u\|_{\EuFrak{H}}^{2}\big{]}<\infty. Note that, as it is customary, $\EuFrak{H}$ is endowed with the Borel $\sigma$-field associated with the distance on $\EuFrak{H}$ given by $(h_{1},h_{2})\mapsto\|h_{1}-h_{2}\|_{\EuFrak{H}}$.

Let $\mathscr{S}$ be the set of all smooth cylindrical random variables of the form

In what follows, we denote by $\delta$ the adjoint of the operator $D$, also called the divergence operator. A random element $u\in L^{2}(\EuFrak{H})$ belongs to the domain of $\delta$, written ${\rm Dom}\,\delta$, if and only if it satisfies

for some constant $c_{u}$ depending only on $u$. If $u\in{\rm Dom}\,\delta$, then the random variable $\delta(u)$ is defined by the duality relationship (customarily called “integration by parts formula”):

Let ${\bf G}^{\prime}$ be an independent copy of ${\bf G}$, and denote by $E^{\prime}$ the mathematical expectation with respect to ${\bf G}^{\prime}$. For every $t\geqslant 0$ the operator $P_{t}:L^{2}(\sigma({\bf G}))\to L^{2}(\sigma({\bf G}))$ is defined as follows: for every $F({\bf G})\in L^{2}(\sigma({\bf G}))$,

in such a way that $P_{0}F({\bf G})=F({\bf G})$ and $P_{\infty}F({\bf G})=E[F({\bf G})]$. The collection $\{P_{t}:t\geqslant 0\}$ verifies the semigroup property $P_{t}P_{s}=P_{t+s}$ and is called the Ornstein-Uhlenbeck semigroup associated with ${\bf G}$.

The properties of the semigroup $\{P_{t}:t\geqslant 0\}$ that are relevant for our study are gathered together in the next statement.

For every $t>0$, the eigenspaces of the operator $P_{t}$ coincide with the Wiener chaoses $C_{q}$, $q=0,1,...$, the eigenvalue of $C_{q}$ being given by the positive constant $e^{-qt}$.

The infinitesimal generator of $\{P_{t}:t\geqslant 0\}$, denoted by $L$, acts on square-integrable random variables as follows: a random variable $F$ with the form (3.61) is in the domain of $L$, written ${\rm Dom}\,L$, if and only if $\sum_{q\geqslant 1}qI_{q}(h_{q})$ is convergent in $L^{2}(\sigma({\bf G}))$, and in this case

In particular, each Wiener chaos $C_{q}$ is an eigenspace of $L$, with eigenvalue equal to $-q$.

In view of the previous statement, it is immediate to describe the pseudo-inverse of $L$, denoted by $L^{-1}$, as follows: for every mean zero random variable $F=\sum_{q\geqslant 1}I_{q}(h_{q})$ of $L^{2}(\sigma({\bf G}))$, one has that

For future reference, we record the following estimate involving random variables living in a finite sum of Wiener chaoses: it is a direct consequence of the hypercontractivity of the Ornstein-Uhlenbeck semigroup – see e.g. in [36, Theorem 2.7.2 and Theorem 2.8.12].

Let $q\geqslant 1$ and $1\leqslant s<t<\infty$. Then, there exists a finite constant $c(s,t,q)<\infty$ such that, for every $F\in\bigoplus^{q}_{k=0}C_{k}$,

In particular, all $L^{p}$ norms, $p\geqslant 1$, are equivalent on a finite sum of Wiener chaoses.

Since we will systematically work on a fixed sum of Wiener chaoses, we will not need to specify the explicit value of the constant $c(s,t,q)$. See again , and the references therein, for more details.

which is a well-defined element of $L^{2}(\EuFrak{H})$.

3 The role of Malliavin and Stein matrices

The following statement is taken from , and provides a simple necessary and sufficient condition for a random vector living in a finite sum of Wiener chaoses to have a density.

(which is a direct consequence of $\log u\leqslant u-1$ for any $u>0$), one has that

To conclude, we present a result providing an explicit representation for Stein matrices associated with random vectors in the domain of $D$.

Taking conditional expectations yields the desired conclusion.

The next section contains the statements and proofs of our main bounds on a Gaussian space.

Entropic fourth moment bounds on a Gaussian space

Our main result is the following entropic central limit theorem for sequences of chaotic random variables.

Let $d\geqslant 1$ and $q_{1},\ldots,q_{d}\geqslant 1$ be fixed integers. Consider vectors

where $O(1)$ indicates a bounded numerical sequence depending on $d,q_{1},...,q_{d}$, as well as on the sequence $\{F_{n}\}$.

One immediate consequence of the previous statement is the following characterisation of entropic CLTs on a finite sum of Wiener chaoses.

Let the sequence $\{F_{n}\}$ be as in the statement of Theorem 4.1, and assume that $C_{n}\to C>0$. Then, the following three assertions are equivalent, as $n\to\infty$:

$F_{n}$ converges in distribution to $Z\sim\mathscr{N}_{d}(0,C)$;

The proofs of the previous results are based on two technical lemmas that are the object of the next section.

2 Estimates based on the Carbery-Wright inequalities

where $X_{1},\dots,X_{n}$ are independent random variables with common distribution $\mathscr{N}(0,1)$.

Fix $d,q_{1},\ldots,q_{d}\geqslant 1$, and let $F=(F_{1},\ldots,F_{d})$ be a random vector such that $F_{i}=I_{q_{i}}(h_{i})$ with $h_{i}\in\EuFrak{H}^{\odot q_{i}}$. Let $\Gamma=\Gamma(F)$ denote the Malliavin matrix of $F$, and assume that $E[\det\Gamma]>0$ (which is equivalent to assuming that $F$ has a density by Theorem 3.6). Set $N=2d(q-1)$ with $q=\max_{1\leqslant i\leqslant d}q_{i}$. Then, there exists a universal constant $c>0$ such that

Proof. Let $\{e_{i}:i\geqslant 1\}$ be an orthonormal basis of $\mathfrak{H}$. Since $\det\Gamma$ is a polynomial of degree $d$ in the entries of $\Gamma$ and because each entry of $\Gamma$ belongs to $\bigoplus_{k=0}^{2q-2}C_{k}$ by the product formula for multiple integrals (see, e.g., [36, Chapter 2]), we have, by iterating the product formula, that $\det\Gamma\in\bigoplus_{k=0}^{N}C_{k}$. Thus, there exists a sequence $\{Q_{n},n\geqslant 1\}$ of real-valued polynomials of degree at most $N$ such that the random variables $Q_{n}(I_{1}(e_{1}),\ldots,I_{1}(e_{n}))$ converge in $L^{2}$ and almost surely to $\det\Gamma$ as $n$ tends to infinity (see [40, proof of Theorem 3.1] for an explicit construction). Assume now that $E[\det\Gamma]>0$. Then, for $n$ sufficiently large, $E[|Q_{n}(I_{1}(e_{1}),\ldots,I_{1}(e_{n}))|]>0$. We deduce from the estimate (4.67) the existence of a universal constant $c>0$ such that, for any $n\geqslant 1$,

from which (4.68) follows by letting $n$ tend to infinity.

In order to estimate the two last terms in (4.71), we decompose the expectation into two parts using the identity

Step 3. For all $\varepsilon,\lambda>0$ and using (4.68),

Choosing $\lambda=\varepsilon^{\frac{N}{N+1}}\beta^{\frac{1}{N+1}}$ yields

with ${\rm Adj}$ the usual adjugate matrix operator.

From the hypercontractivity property together with the equality

one immediately deduces the existence of $c_{q,d,\|C\|_{H.S.}}>0$ such that

Substituting this estimate into (4.74) and assuming that $M\geqslant 1$, yields

Choosing $R=\alpha^{\frac{1}{d+1}}(M+\|{\bf x}\|_{1})^{\frac{d}{d+1}}$ and assuming $\alpha\leqslant 1$, we obtain

It is worthwhile noting that the inequality (4.2) is valid for any $t\in[\frac{1}{2},1]$, in particular for $t=1$.

Step 6. From (4.71), (LABEL:7) and (4.2) we obtain

By plugging this inequality into (4.70) we thus obtain that, for every $M\geqslant 1$, $\varepsilon>0$ and $0<\alpha\leqslant 1$:

Choosing $M=\varepsilon^{-\frac{1}{N+1}}$, $\varepsilon=\alpha^{\frac{N+1}{(2N+4)(d+1)}}$ and $\alpha=(1-t)^{\frac{(2N+4)(d+1)}{2((2N+4)(d+1)+1)}}$, one obtains the desired conclusion (4.69).

3 Proof of Theorem 4.1

In the proof of [41, Theorem 4.3], the following two facts have been shown:

Using Proposition 3.7, one infers immediately that

defines a Stein’s matrix for $F_{n}$, which moreover satisfies the relation

Now let $\Gamma_{n}$ denote the Malliavin matrix of $F_{n}$. Thanks to [43, Lemma 6], we know that, for any $i,j=1,\ldots,d$,

Since $\langle DF_{i,n},DF_{j,n}\rangle_{\EuFrak{H}}$ lives in a finite sum of chaoses (see e.g. [36, Chapter 5]) and is bounded in $L^{2}(\sigma({\bf G}))$, we can again apply the hypercontractive estimate (3.64) to deduce that $\langle DF_{i,n},DF_{j,n}\rangle_{\EuFrak{H}}$ is actually bounded in $L^{p}(\sigma({\bf G}))$ for every $p\geqslant 1$, so that the convergence in (4.79) is in the sense of any of the spaces $L^{p}(\sigma({\bf G}))$. As a consequence, $E[\det\Gamma_{n}]\to\det C\prod_{i=1}^{d}q_{i}=:\gamma>0,$ and there exists $n_{0}$ large enough so that

This means that relation (2.53) is satisfied uniformly on $n$. Concerning (2.52), again by hypercontractivity and using the representation (4.77), one has that, for all $\eta>0$,

Finally, since $\Delta_{n}\to 0$ and because (4.78) holds true, the condition (2.54) is satisfied for $n$ large enough. The proof of (4.66) is concluded by applying Theorem 2.11.

4 Proof of Corollary 4.2

In view of Theorem 4.1, one has only to prove that (b) implies (a). This is an immediate consequence of the fact that the covariance $C_{n}$ converges to $C$, and that the sequence $\{F_{n}\}$ lives in a finite sum of Wiener chaoses. Indeed, by virtue of (3.64) one has that

yielding in particular that, if $F_{n}$ converges in distribution to $Z$, then $E\|F_{n}\|^{4}\to E\|Z\|^{4}$ or, equivalently, $\Delta_{n}\to 0$. The proof is concluded.

Acknowledgement. We heartily thank Guillaume Poly for several crucial inputs that helped us achieving the proof of Theorem 4.1. We are grateful to Larry Goldstein and Oliver Johnson for useful discussions. Ivan Nourdin is partially supported by the ANR grant ANR-10-BLAN-0121.

References