Stein's method, logarithmic Sobolev and transport inequalities

Introduction

is the relative entropy of $d\nu=hd\gamma$ with respect to $\gamma$ and

where ${\rm Hess}(\varphi)$ stands for the Hessian of $\varphi$, whereas ${\langle\cdot,\cdot\rangle}_{\rm HS}$ and ${\|\cdot\|}_{\rm HS}$ denote the usual Hilbert-Schmidt scalar product and norm, respectively. Note that while Stein kernels appear implicitly in the literature about Stein’s method (see the original monograph [Ste, Lecture VI] of C. Stein, as well as [C2, C3, G-S1, G-S2]…), they gained momentum in recent years, specially in connection with probabilistic approximations involving random variables living on a Gaussian (Wiener) space (see the recent monograph [N-P2] for an overview of this emerging area). The terminology ‘kernel’ with respect to ‘factor’ seems the most appropriate to avoid confusion with related but different existing notions.

becomes relevant as a measure of the proximity of $\nu$ and $\gamma$. This quantity is actually at the root of the Stein method [C-G-S, N-P2]. For example, in dimension one, the classical Stein bound expresses that the total variation distance ${\rm TV}(\nu,\gamma)$ between a probability measure $\nu$ and the standard Gaussian distribution $\gamma$ is bounded from above as

justifying therefore the interest in the Stein discrepancy (see also [C-P-U]). It is actually a main challenge addressed in [N-P-S1] and this work to investigate the multidimensional setting in which inequalities such as (1.2) are no more available.

With the Stein discrepancy ${\rm S}(\nu\,|\,\gamma)$, we emphasize here the inequality, for every probability $d\nu=hd\gamma$,

as a new improved form of the logarithmic Sobolev inequality (1.1). In addition, this inequality (1.3) transforms bounds on the Stein discrepancy into entropic bounds, hence allowing for entropic approximations (under finiteness of the Fisher information). Indeed as is classical, the relative entropy ${\rm H}(\nu\,|\,\gamma)$ is another measure of the proximity between two probabilities $\nu$ and $\gamma$ (note that ${\rm H}(\nu\,|\,\gamma)\geq 0$ and ${\rm H}(\nu\,|\,\gamma)=0$ if and only if $\nu=\gamma$), which is moreover stronger than the total variation distance by the Pinsker-Csizsár-Kullback inequality

The proof of (1.3) is achieved by the classical interpolation scheme along the Ornstein-Uhlenbeck semigroup ${(P_{t})}_{t\geq 0}$ towards the logarithmic Sobolev inequality, but modified for time $t$ away from by a further integration by parts involving the Stein kernel. Indeed, while the exponential decay ${\rm I}_{\gamma}(P_{t}h)\leq e^{-2t}\,{\rm I}_{\gamma}(h)$ of the Fisher information classically produces the logarithmic Sobolev inequality (1.1), the argument is supplemented by a different control of ${\rm I}_{\gamma}(P_{t}h)$ by the Stein discrepancy for $t>0$.

We call the inequality (1.3) HSI, connecting entropy H, Stein discrepancy S and Fisher information I, by analogy with the celebrated Otto-Villani HWI inequality [O-V] relating entropy H, (quadratic) Wasserstein distance W (${\rm W}_{2}$) and Fisher information I. We actually provide in Section 3 a comparison between the HWI and HSI inequalities (suggesting even an HWSI inequality). Moreover, based on the approach developed in [O-V], we prove that

an inequality that improves upon the celebrated Talagrand quadratic transportation cost inequality [T]

(since $\arccos(e^{-r})\leq\sqrt{2r}$ for every $r\geq 0$). We shall refer to (1.4) as the ‘WSH inequality’. Note also that ${\rm W}_{2}(\nu,\gamma)\,\leq\,{\rm S}(\nu\,|\,\gamma)$ so that, as entropy, the Stein discrepancy is a stronger measurement than the Wasserstein metric ${\rm W}_{2}$.

The new HSI inequality put forward in this work has a number of significant applications to exponential convergence to equilibrium and concentration inequalities. For example, the standard exponential decay of entropy ${\rm H}(\nu^{t}\,|\,\gamma)\leq e^{-2t}\,{\rm H}(\nu^{0}\,|\,\gamma)$ along the flow $d\nu^{t}=P_{t}hd\gamma$, $t\geq 0$ ($\nu^{0}=\nu$, $\nu^{\infty}=\gamma$), which characterizes the logarithmic Sobolev inequality (1.1) may be strengthened under finiteness of the Stein discrepancy ${\rm S}={\rm S}(\nu\,|\,\gamma)={\rm S}(\nu^{0}\,|\,\gamma)$ into

for all $0\leq r\leq r_{n}$ where $r_{n}\to\infty$ according to the growth of ${\rm S}_{p}(\nu\,|\,\gamma)$ as $p\to\infty$.

As alluded to above, the HSI inequality (1.3) is designed to yield entropic central limit theorems for sequences of probability measures of the form $d\nu_{n}=h_{n}d\gamma$, $n\geq 1$, such that $s_{n}={\rm S}(\nu_{n}\,|\,\gamma)\to 0$ and

of the relative entropy of the distribution $\nu_{F}$ of a vector $F=(F_{1},\ldots,F_{d})$ on $(E,\mu,\Gamma)$ with respect to $\gamma$ by the Stein discrepancy ${\rm S}(\nu_{F}\,|\,\gamma)$, where $C_{F},\widetilde{C}_{F}>0$ depend on integrability properties of $F$, the carré du champ operators $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$, and the inverse of the determinant of the matrix ${(\Gamma(F_{i},F_{j}))}_{1\leq i,j\leq d}$. In particular, ${\rm H}(\nu_{F}\,|\,\gamma)\to~{}0$ as ${\rm S}(\nu_{F}\,|\,\gamma)\to 0$ providing therefore entropic convergence under the Stein discrepancy. The general results obtained here cover not only normal approximation but also gamma approximation.

The inequality (1.7) thus transfers bounds on the Stein discrepancy to entropic bounds. The issue of controlling the Stein discrepancy ${\rm S}(\nu_{F}\,|\,\gamma)$ itself (in terms of moment conditions for example) is not addressed here, and has been the subject of numerous recent studies around the so-called Nualart-Peccati fourth moment theorem (cf. [N-P2]). This investigation is in particular well adapted to functionals $F=(F_{1},\ldots,F_{d})$ whose coordinates are eigenfunctions of the underlying Markov generator. See [A-C-P, A-M-P, L3] for several results in this direction and [N-P2, Chapters 5-6] for a detailed discussion of estimates on ${\rm S}(\nu_{F}\,|\,\gamma)$ that are available for random vectors $F$ living on the Wiener space.

The structure of the paper thus consists of two main parts, the first one devoted to the new HSI and WSH inequalities, the second one to an investigation of entropic bounds via the Stein discrepancy. Section 2 is devoted to the proof and discussions of the HSI inequality in the Gaussian case, with a first sample of illustrations and applications to convergence to equilibrium and measure concentration. In Section 3, we investigate connections between the Stein discrepancy, Wasserstein distances and transportation cost inequalities, in particular the HWI inequality, and establish the WSH inequality. Extensions of the HSI inequality to more general distributions arising as invariant probability measures of second order differential operators are addressed in Section 4. The second part consists of Section 5 which develops a general methodology (in the context of Markov Triples) to reach entropic bounds on densities of families of functionals under conditions which do not necessarily involve the Fisher information.

Logarithmic Sobolev inequality and Stein discrepancy

Observe from (2.1) that, without loss of generality, one may and will assume in the sequel that $\tau_{\nu}^{ij}(x)=\tau_{\nu}^{ji}(x)$ $\nu$-a.e., $i,j=1,\ldots,d$. Also, by choosing $\varphi=x_{i}$, $i=1,\ldots,d$, in (2.1) one sees that, if $\nu$ admits a Stein kernel, then $\nu$ is necessarily centered. Moreover, by selecting $\varphi=x_{i}x_{j}$, $i,j=1,\ldots,d$, and since $\tau^{ij}_{\nu}=\tau^{ji}_{\nu}$,

(and in particular $\nu$ has finite second moments).

In dimension $d\geq 2$, a Stein kernel $\tau_{\nu}$ may not be unique – see [N-P-S2, Appendix A].

It is important to notice that, in dimension $d\geq 2$, the definition (2.1) of Stein kernel is actually weaker than the one used in [N-P-S1, N-P-S2]. Indeed, in those references a Stein kernel $\tau_{\nu}$ is required to satisfy the stronger ‘vector’ (as opposed to the trace identity (2.1)) relation

Definition (2.1) is directly inspired by the Gaussian integration by parts formula according to which

so that the proximity of $\tau_{\nu}$ with the identity matrix ${\rm Id}$ indicates that $\nu$ should be close to $\gamma$. In particular, it should be clear that the notion of Stein kernel in the sense of (2.1) is motivated by normal approximation. Section 4 will introduce analogous definitions adapted to the target measure in the context of the generator approach to Stein’s method. Whenever a Stein kernel exists, we consider to this task the quantity, called Stein discrepancy of $\nu$ with respect to $\gamma$ in the introduction,

(Note that ${\rm S}(\nu\,|\,\gamma)$ may be infinite if one of the $\tau_{\nu}^{ij}$’s is not in ${\rm L}^{2}(\nu)$.) Whenever ${{\rm S}(\nu\,|\,\gamma)=0}$, then $\nu=\gamma$ since $\tau_{\nu}$ is the identity matrix (see e.g. [N-P2, Lemma 4.1.3]). Observe also that if $C$ denotes the covariance matrix of $\nu$, then

where ${\rm Var}_{\nu}$ indicates the variance under the probability measure $\nu$.

2 The Gaussian HSI inequality

The following result emphasizes the Gaussian HSI inequality connecting entropy ${\rm H}$, Stein discrepancy ${\rm S}$ and Fisher information ${\rm I}$. In the statement, we use the conventions $0\log(1+\frac{s}{0})=0$ and $\infty\log(1+\frac{s}{\infty})=s$ for every $s\in[0,\infty]$, and $r\log(1+\frac{\infty}{r})=\infty$ for every $r\in(0,\infty)$.

from which ${\rm I}(\nu\,|\,\gamma)=0$, and therefore $\nu=\gamma$, which is in contrast with the assumption ${\rm S}(\nu\,|\,\gamma)>0$. As a consequence, we infer that ${\rm S}(\nu\,|\,\gamma)=0$, from which it follows that $\nu=\gamma$.

Let $\gamma_{C}$ be as above (with $C$ non-singular), and let $d\nu=hd\gamma_{C}$ be centered with smooth probability density $h$ with respect to $\gamma_{C}$. Assume that $\nu$ admits a Stein kernel $\tau_{\nu}$ in the sense of (2.1). Then,

where $C^{-\frac{1}{2}}$ denotes the unique symmetric non-singular matrix such that ${(C^{-\frac{1}{2}})}^{2}=C^{-1}$.

Corollary 2.3 is easily deduced from Theorem 2.2 and details are left to the reader. The argument simply uses that if $M$ is the unique non-singular symmetric matrix such that $C=M^{2}$, then ${\rm H}(\nu\,|\,\gamma_{C})={\rm H}(\nu^{0}\,|\,\gamma)$ where $d\nu^{0}(x)=h(Mx)d\gamma(x)$.

3 Proof of the Gaussian HSI inequality

According to our conventions, if either ${\rm S}(\nu\,|\,\gamma)$ or ${\rm I}(\nu\,|\,\gamma)$ is infinite, then (2.6) coincides with the logarithmic Sobolev inequality (1.1). On the other hand, if ${\rm S}(\nu\,|\,\gamma)$ or ${\rm I}(\nu\,|\,\gamma)$ equals zero, then $\nu=\gamma$, and therefore ${\rm H}(\nu\,|\,\gamma)=0$. It follows that, in order to prove (2.6), we can assume without loss of generality that ${\rm S}(\nu\,|\,\gamma)$ and ${\rm I}(\nu\,|\,\gamma)$ are both non-zero and finite.

owing to a standard integration by parts of the Gaussian density.

The generator $\cal L$ is a diffusion and satisfies the integration by parts formula

(These expressions should actually be considered for $h+\varepsilon$ as $\varepsilon\to 0$.) Using $P_{t}h$ instead of $h$ in the previous relations and writing $v_{t}=\log P_{t}h$, one deduces from the symmetry of $P_{t}$ that

Recall finally that if $d\nu^{t}=P_{t}hd\gamma$, $t\geq 0$ (with $\nu^{0}=\nu$ and $\nu^{t}\to\gamma$), the classical de Bruijn’s formula (see e.g. [B-G-L, Proposition 5.2.2]) indicates that

Theorem 2.2 will follow from the next Proposition 2.4. In this proposition, (i) corresponds to the integral version of (2.13) whereas (ii) describes the well-known exponential decay of the Fisher information along the Ornstein-Uhlenbeck semigroup. This decay actually yields the logarithmic Sobolev inequality (1.1), see [B-G-L, Section 5.7]. The new third point (iii) is a reformulation of [N-P-S1, Theorem 2.1] for which we provide a self-contained proof. It describes an alternate bound on the Fisher information along the semigroup in terms of the Stein discrepancy for values of $t>0$ away from . It is the combination of (ii) and (iii) which will produce the HSI inequality. Point $(iv)$ will be needed in the forthcoming proof of the WSH inequality (1.4), as well as in the proof of Proposition 3.1 providing a direct bound of the Wasserstein distance ${\rm W}_{2}$ by the Stein discrepancy.

Under the above notation and assumptions, denote by $\tau_{\nu}$ a Stein kernel of $d\nu=hd\gamma$. For every $t>0$, recall $d\nu^{t}=P_{t}h\,d\gamma$, and write $v_{t}=\log P_{t}h$. Then,

(Exponential decay of Fisher information) For every $t\geq 0$,

(Exponential decay of Stein discrepancy) For every $t\geq 0$,

In view of the preceding discussion, only the proofs of (iii) and $(iv)$ need to be detailed. Throughout the various analytical arguments below, it may be assumed that the density $h$ is regular enough, the final conclusions being then reached by approximation arguments as e.g. in [O-V, B-G-L]. Starting with (iii), use (2.12) and the definition (2.1) of $\tau_{\nu}$ to write, for any $t>0$,

Now, for all $i,j=1,\ldots,d$, by (2.8) and (2.9),

which is (2.16). To deduce the estimate (2.17), it suffices to apply (twice) the Cauchy-Schwarz inequality to the right-hand side of (2.16) in such a way that, by integrating out the $y$ variable,

by symmetry of $P_{t}$, the proof of (2.17) is complete.

By the integral representation of $P_{t}$,

Use now the definition of $\tau_{\nu}$ in the $x$ variable and integration by parts in the $y$ variable to get that

As a consequence, a Stein kernel for $\nu^{t}$ is

By the Cauchy-Schwarz inequality along $P_{t}$,

that is the announced result (iv). Proposition 2.4 is established.

For every $t>0$, it is easily checked that the mapping $x\mapsto\tau_{\nu^{t}}(x)$ appearing in (2.20) admits the probabilistic representation

We are now in a position to prove Theorem 2.2.

Proof of Theorem 2.2. As announced, on the basis of the interpolation (2.14), we apply (2.15) and (2.17) respectively to bound the Fisher information ${\rm I}_{\gamma}(P_{t}h)$ for $t$ around and away from . We thus get, for every $u>0$,

Optimizing in $u$ (set $1-e^{-2u}=r\in(0,1)$) concludes the proof. ∎

It is worth mentioning that a slight modification of the proof of (iii) in Proposition 2.4 leads to the improved form of the exponential decay (2.15) of the Fisher information

As for the classical logarithmic Sobolev inequality, the inequality (2.22) may be integrated along de Bruijin’s formula (2.13) towards the better, although less tractable, HSI inequality

(understood in the limit as ${\rm S}^{2}={\rm I}$), where ${\rm H}={\rm H}(\nu\,|\,\gamma)$, ${\rm S}={\rm S}(\nu\,|\,\gamma)$ and ${\rm I}={\rm I}(\nu\,|\,\gamma)$.

Together with the de Bruijn identity (2.13), the classical logarithmic Sobolev inequality (1.1) ensures the exponential decay in $t\geq 0$ of the relative entropy

along the Ornstein-Uhlenbeck semigroup (cf. e.g. [B-G-L, Theorem 5.2.1]). The new HSI produces a reinforcement of this exponential convergence to equilibrium under finiteness of the Stein discrepancy.

Let $\nu$ with Stein discrepancy ${\rm S}(\nu\,|\,\gamma)={\rm S}$. For any $t\geq 0$,

Together with (2.18) and since r\mapsto r\log\big{(}1+\frac{s}{r}\big{)} is increasing for any fixed $s$, the HSI inequality applied to $\nu^{t}$ implies that

Set $U(t)=\frac{e^{4t}}{{\rm S}^{2}}\,{\rm H}(\nu^{t}\,|\,\gamma)$, $t\geq 0$, so that by (2.13), $U^{\prime}=4U-\frac{e^{4t}}{{\rm S}^{2}}\,{\rm I}(\nu^{t}\,|\,\gamma)$. The latter inequality therefore rewrites as

Since $e^{r}-1-r\geq\frac{r^{2}}{2}$ for $r\geq 0$, this inequality may be relaxed into ${-2U+2U^{2}\,\leq\,-U^{\prime}}$. Setting $V(t)=e^{-2t}U(t)$, $t\geq 0$, it follows that $2e^{2t}V^{2}(t)\leq-V^{\prime}(t)$ so that, after integration,

By definition of $V$, this inequality amounts to the conclusion of Corollary 2.7 and the proof is complete. ∎

4 Stein discrepancy and concentration inequalities

Equivalently (up to numerical constants) in terms of moment growth,

Hence ${\rm S}_{2}(\nu\,|\,\gamma)={\rm S}(\nu\,|\,\gamma)$ is the Stein discrepancy as defined earlier. Recall ${\|\cdot\|}_{\rm op}$ the operator norm on the $d\times d$ matrices.

Before turning to the proof of this result, let us comment on its measure concentration content. One first important aspect is that the constant $C$ is dimension free. When $\nu=\gamma$, (2.28) exactly fits the Gaussian case (2.27). In general, the moment growth in $p$ describes various concentration regimes of $\nu$ (cf. [L2, Section 1.3], [B-L-M, Chapter 14]) according to the growth of the $p$-Stein discrepancy ${\rm S}_{p}(\nu\,|\,\gamma)$.

In view of the elementary estimate ${\|\tau_{\nu}\|}_{\rm op}\leq 1+{\|\tau_{\nu}-{\rm Id}\|}_{\rm HS}$, the conclusion (2.28) immediately yields the moment growth (1.6) emphasized in the introduction

Note that there is already an interest to write this bound for $p=2$,

Together with E. Milman’s Lipschitz characterization of Poincaré inequalities for log-concave measures [M], it shows that the Stein discrepancy ${\rm S}(\nu\,|\,\gamma)$ with respect to the standard Gaussian measure is another control of the spectral properties in this class of measures.

Similar inequalities hold for arbitrary covariances by suitably adapting the Stein kernel as in Corollary 2.3.

By the triangle inequality, the latter is bounded from above by

which produces a first bound of interest. If it is assumed in addition that the covariance matrix of $X$ is the identity, we may use classical inequalities for sums of independent centered random vectors (in Euclidean space) to the family $\tau_{\nu}(X_{k})-{\rm Id}$, $k=1,\ldots,n$. Hence, by for example Rosenthal’s inequality (see e.g. [B-L-M, M-J-C-F-T]), for $p\geq 2$,

for some numerical $C>0$. Note that the bound is optimal both for $\nu=\gamma$ and as $n\to\infty$ describing the standard Gaussian concentration (2.27). By Markov’s inequality, optimizing in $p\geq 2$, one deduces that for some numerical $C^{\prime}>0$,

for all $0\leq r\leq r_{n}$ where $r_{n}\to\infty$ according to the growth of ${\rm S}_{p}$ as $p\to\infty$. For example, if ${\rm S}_{p}=O(p^{\alpha})$ for some $\alpha>0$ (see below for such illustrations), then

(for some possibly different numerical $C>0$) for every $p\leq n^{\frac{1}{2\alpha+2}}$. By Markov’s inequality in this range of $p$,

and with $p\sim\frac{r^{2}}{4C^{2}}$, the claims follows with $r_{n}$ of the order of $n^{\frac{1}{4\alpha+4}}$.

For concreteness, we describe two classes of one-dimensional distributions such that the associated Stein kernel has finite moments of all orders. Denote by $X$ a centered real-valued random variable with law $\nu$ and Stein kernel $\tau_{\nu}$. Recall from (2.2) of Remark 2.1, that if $\nu$ has density $\rho$ with respect to the Lebesgue measure, a version of $\tau_{\nu}$ is given by $\tau_{\nu}(x)=\rho(x)^{-1}\int_{x}^{\infty}y\rho(y)dy$ for $x$ inside the support of $\rho$.

Assume that the support of $\rho$ coincides with an open interval of the type $(a,b)$, with $-\infty\leq a<b\leq+\infty$. Say then that the law $\nu$ of $X$ is a (centered) member of the Pearson family of continuous distributions if the density $\rho$ satisfies the differential equation

for some real numbers $a_{0},a_{1},b_{0},b_{1},b_{2}$. We refer the reader e.g. to [DZ, Sec. 5.1] for an introduction to the Pearson family. It is a well-known fact that there are basically five families of distributions satisfying (2.30): the centered normal distributions, centered gamma and beta distributions, and distributions that are obtained by centering densities of the type $\rho(x)=Cx^{-\alpha}e^{-\beta/x}$ or $\rho(x)=$ ${C(1+x)^{-\alpha}\exp(\beta\arctan(x))}$ ($C$ being a suitable normalizing constant). According to [Ste, Theorem 1, p. 65], if $\tau_{\nu}$ satisfies

then $\tau_{\nu}(x)=\alpha x^{2}+\beta x+\gamma$, $x\in(a,b)$ (with $\alpha,\,\beta,\,\gamma$ real constants) if and only if $\nu$ is a member of the Pearson family in the sense that $\rho$ satisfies (2.30) for every $x\in(a,b)$ with $a_{0}=\beta$, $a_{1}=2\alpha+1$, $b_{0}=\gamma$, $b_{1}=\beta$ and $b_{2}=\alpha$. It follows that if $\nu$ is centered member of the Pearson family such that (2.31) is satisfied and $X$ has finite moments of all orders, so has $\tau_{\nu}(X)$. This includes the case of Gaussian, gamma and beta distribution for example.

In concrete instances, such as Wiener chaos for example, the latter expression may be easily controled so to yield concentration properties of the underlying distribution of $F$. For example, in the setting of the recent [A-C-P], it may be shown by hypercontractive means that for the Hermite, Laguerre or Jacobi (or mixed ones) chaos structures, for any $p\geq 2$,

According to the respective growth in $p$ of $C_{p,\lambda}$, concentration properties on $F$ may be achieved.

Using that $|\nabla u|\leq 1$ (since $u$ is $1$-Lipschitz) and furthermore

it easily follows as in the previous section that for every $t$,

where $\alpha=2q$ and $(2q-1)\beta=2q$, and

where $\alpha^{\prime}=q$ and $(2q-2)\beta^{\prime}=2q$. Therefore (2.32) implies that, for every $t$,

Integrating this differential inequality yields that

where ${\widetilde{C}}(t)=\int_{t}^{\infty}C(s)ds$, $t\geq 0$. It follows that $\phi(0)\leq e^{{\widetilde{C}}(0)}\int_{0}^{\infty}D(s)ds$ and therefore

Since ${\widetilde{C}}(0)$ is bounded above by $Cq$ for some numerical $C>0$, the announced claim follows. The proof of Theorem 2.8 is therefore complete. ∎

5 On the rate of convergence in the entropic central limit theorem

On the other hand, as in the previous paragraph,

where $\alpha(a)=\sum_{i=1}^{n}a_{i}^{4}$.

As a consequence therefore of the HSI inequality of Theorem 2.2,

This result has to be compared with the works [A-B-B-N] and [B-J] (cf. [J]) which produce the bound

under the hypothesis that $\nu$ satisfies a Poincaré inequality with constant $c>0$.

For the classical average $T_{n}=\frac{1}{\sqrt{n}}\sum_{k=1}^{n}X_{k}$, (2.34) yields a rate $O(\frac{1}{n})$ in the entropic central limit theorem while (2.33) only produces $O(\frac{\log n}{n})$, however at a cheap expense and under potentially different conditions as described in Remark 2.9. For this classical average, the recent works [B-C-G1, B-C-G2] actually provide a complete picture with rate $O(\frac{1}{n})$ under a fourth-moment condition on $X$ based on local central limit theorems and Edgeworth expansions. General sums $T=\sum_{k=1}^{n}a_{k}X_{k}$ are studied in [B-C-G3] as a particular case of sums of independent non-identically distributed random variables. Vector-valued random variables may be considered similarly.

Transport distances and Stein discrepancy

We shall subdivide the analysis into two parts. In Section 3.1, we deal with the special case of the quadratic Wasserstein distance ${\rm W}_{2}$, for which we use the definition (2.1) of a Stein kernel. In Section 3.2, we deal with general Wasserstein distances ${\rm W}_{p}$ possibly of order $p\neq 2$, for which it seems necessary to use the stronger definition (2.3) adopted in [N-P-S1, N-P-S2].

Note that (3.2) is actually the central argument in the Otto-Villani theorem [O-V] asserting that a logarithmic Sobolev inequality implies a Talagrand transport inequality. Here, by making use of (3.2) and then (2.17) we get that

The general case is obtained by a simple regularization procedure which is best presented in probabilistic terms. Fix $\varepsilon>0$ and introduce the auxiliary random variable $F_{\varepsilon}=e^{-\varepsilon}F+\sqrt{1-e^{-2\varepsilon}}Z$ where $F$ and $Z$ are independent with respective laws $\nu$ and $\gamma$. It is immediately checked that: (a) the distribution of $F_{\varepsilon}$, denoted by $\nu^{\varepsilon}$, admits a smooth density $h_{\varepsilon}$ with respect to $\gamma$ (of course, this density coincides with $P_{\varepsilon}h$ whenever the distribution of $F$ admits a density $h$ with respect to $\gamma$ as in the first part of the proof); (b) a Stein kernel for $\nu^{\varepsilon}$ is given by

(consistent with (2.21)); (c) ${\rm S}(\nu^{\varepsilon}\,|\,\gamma)\leq e^{-2\varepsilon}\,{\rm S}(\nu\,|\,\gamma)$; (d) as $\varepsilon\to 0$, $F_{\varepsilon}$ converges to $F$ in ${\rm L}^{2}$, so that, in particular, ${\rm W}_{2}(\nu^{\varepsilon},\gamma)\to{\rm W}_{2}(\nu,\gamma)$. One therefore infers that

The inequality (3.1) may of course be compared to the Talagrand quadratic transportation cost inequality [T, V, B-G-L]

As announced in the introduction, one can actually further refine (3.1) in order to deduce an improvement of (3.3) in the form of a WSH inequality. The refinement relies on the HSI inequality itself.

Proof. For any $t\geq 0$, recall $d\nu^{t}=P_{t}hd\gamma$ (in particular, $\nu^{0}=\nu$ and $\nu^{t}\to\gamma$ as $t\to\infty$). The HSI inequality (2.6) applied to $\nu^{t}$ yields that

Now, ${\rm S}^{2}(\nu^{t}\,|\,\gamma)\leq{\rm S}^{2}(\nu\,|\,\gamma)$ by (2.18) and r\mapsto r\log\big{(}1+\frac{s}{r}\big{)} is increasing for any fixed $s$ from which it follows that

By exponentiating both sides, this inequality is equivalent to

Combining with (3.2) and recalling (2.13) leads to

The desired conclusion is achieved by integrating between $t=0$ and $t=\infty$. The proof of Theorem 3.2 is complete. ∎

Proposition 3.1 and Theorem 3.2 raise a number of observations.

Since $\arccos(e^{-r})\leq\sqrt{2r}$ for every $r\geq 0$, the WSH inequality thus represents an improvement upon the Talagrand inequality (3.3). Moreover, as for the HSI inequality, the WSH inequality produces the case of equality in (3.3) since $\arccos(e^{-r})\leq\sqrt{2r}$ is an equality only at $r=0$.

The Talagrand inequality may combined with the HSI inequality of Theorem 2.2 to yield the bound

(HWI inequality). As described in the introduction, a fundamental estimate connecting entropy ${\rm H}$, Wassertein distance ${\rm W}_{2}$ and Fisher information ${\rm I}$ is the so-called HWI inequality of Otto and Villani [O-V] stating that, for all $d\nu=hd\gamma$ with density $h$ with respect to $\gamma$,

(see, e.g. [V, pp. 529-542] or [B-G-L, Section 9.3.1] for a general discussion). Recall that the HWI inequality (3.5) improves upon both the logarithmic Sobolev inequality (1.1) and the Talagrand inequality (3.3). It is natural to look for a more general inequality, involving all four quantities ${\rm H}$, ${\rm W}_{2}$, ${\rm I}$ and the Stein discrepancy ${\rm S}$, and improving both the HSI and HWI inequalities. One strategy towards this task would be to follow again the heat flow approach of the proof of Theorem 2.2 and write, for $0<u\leq t$,

Here, we used (2.15) and (2.17), as well as the known reverse Talagrand inequality along the semigroup given by

(cf. e.g. [B-G-L, p. 446]). Setting $\alpha=1-e^{-2u}\leq 1-e^{-2t}=\beta$, the preceding estimate yields

However, elementary computations show that, unless the rather unnatural inequality $2{\rm W}_{2}(\nu,\gamma)\leq{\rm S}(\nu\,|\,\gamma)$ is verified, the minimum in the above expression is attained at a point $(\alpha,\beta)$ such that either $\alpha=\beta$ (and in this case one recovers HWI) or $\beta=1$ (yielding HSI). Hence, at this stage, it seems difficult to outperform both HWI and HSI estimates with a single ‘HWSI’ inequality. In the subsequent point (d), we provide an elementary explicit example in which the HSI estimate perform better than the HWI inequality.

In this item, we thus compare the HWI and HSI inequalities on a specific example in dimension $d=1$. For every $n\geq 1$, consider the probability measure $d\nu_{n}(x)=\rho_{n}(x)dx$ with density

where ${(a_{n})}_{n\geq 1}$ is such that $a_{n}\in$ for every $n\geq 1$, a_{n}=o\big{(}\frac{1}{\log n}\big{)} and ${n^{2/3}a_{n}\to\infty}$. A direct computation easily shows that ${\rm H}(\nu_{n}\,|\,\gamma)\to 0$. Also, since

one may show after simple (but a bit lengthy) computations that

We next examine the Stein discrepancy ${\rm S}(\nu_{n}\,|\,\gamma)$ and Wassertein distance ${\rm W}_{2}(\nu_{n},\gamma)$. Since a Stein kernel $\tau_{n}$ of $\nu_{n}$ is given by

Concerning the Wasserstein distance, from the inequality (3.1), we deduce that ${\rm W}_{2}(\nu_{n},\gamma)\leq\sqrt{a_{n}}$. On the other hand, by the Lipschitz characterization of ${\rm W_{1}}$ (specializing to the Lipschitz function $x\mapsto|\cos(x)|$), cf. e.g. [V, Remark 6.5]),

Now, the right-hand side of this inequality multiplied by $\frac{1}{a_{n}}$ is equal to

which, by dominated convergence, converges to a non-zero limit. As a consequence, there exists $c>0$ such that, for $n$ large enough, ${\rm W}_{2}(\nu_{n},\gamma)\geq c\,a_{n}$.

Summarizing the conclusions, the quantity

is bigger than a sequence of the order of $na_{n}^{3/2}=(n^{2/3}a_{n})^{3/2}$, which (by construction) diverges to infinity as $n\to\infty$. This fact implies that, in this specific case, the bound in the HWI inequality diverges to infinity, whereas ${\rm H}(\nu_{n}\,|\,\gamma)\to 0$. On the other hand, the HSI bound converges to zero, since

2 General Wasserstein distances under a stronger notion of Stein kernel

(where $\delta_{ij}=1$ if $i=j$ and if not), possibly infinite if $\tau_{\nu}^{ij}\notin{\rm L}^{p}(\nu)$. In particular, ${\|\tau_{\nu}-{\rm Id}\|}_{2,\nu}={\rm S}(\nu\,|\,\gamma)$.

Let $p\in[2,\infty)$. If $\nu$ has finite moments of order $p$, then (with the same $C_{p}$ as in (i))

In particular, for $p=2$ we recover (3.1).

Owing to an approximation argument analogous to the one rehearsed at end of the proof of Proposition 3.1, it is sufficient to consider the case $d\nu=h\,d\gamma$ where $h$ is a smooth density. Write as before $v_{t}=\log P_{t}h$ and $d\nu^{t}=P_{t}hd\gamma$. By virtue of [N-P-S1, Lemma 2.9], under thus the strengthened assumption (2.3), a version of $\nabla v_{t}$, $t>0$, is given by

where, as in Remark 2.5, $F$ and $Z$ are independent with respective law $\nu$ and $\gamma$, and $F_{t}=e^{-t}F+\sqrt{1-e^{-2t}}Z$. Moreover, one can straightforwardly modify the proof of [O-V, Lemma 2] (cf. also [V, Theorem 24.2(iv)]) in order to obtain the general estimate

yielding (i). On the other hand, if $p\geq 2$, then

which immediately yields (ii). The proof of Proposition 3.4 is complete. ∎

Specializing (3.6) to the case $p=1$ yields the estimate

which improves previous dimensional bounds obtained by an application of the multidimensional Stein method (cf. the proof of [N-P2, Theorem 6.1.1]). It is important to note that, apart from the results obtained in the present paper, there is no other version of Stein’s method allowing one to deal with Wasserstein distances of order $p>1$. Observe that coupling results from [C3] (that are based on completely different methods) may be used to deduce analogous estimates in the case when $d=1$ and the Stein kernel $\tau_{\nu}$ is bounded.

HSI inequalities for further distributions

The operator $\mathcal{L}$ satisfies the chain rule formula and defines a diffusion operator. We assume that $\mathcal{L}$ is the generator of a symmetric Markov semigroup ${(P_{t})}_{t\geq 0}$, where the symmetry is with respect to an invariant probability measure $\mu$.

A central object of interest in this context is the carré du champ operator $\Gamma$ defined from the generator $\mathcal{L}$ by

for all $(f,g)\in\mathcal{A}\times\mathcal{A}$. Note that $\Gamma$ is bilinear and symmetric and $\Gamma(f,f)\geq 0$. Moreover, the integration by parts property for $\mathcal{L}$ with respect to the invariant measure $\mu$ is expressed by the fact that, for functions $f,g\in\mathcal{A}$,

The structure $(E,\mu,\Gamma)$ then defines a Markov Triple in the sense of [B-G-L] to which we refer for the necessary background.

The requested semigroup analysis toward HSI inequalities will actually involve in addition the iterated gradient operators $\Gamma_{n}$, $n\geq 1$, defined inductively for $(f,g)\in\mathcal{A}\times\mathcal{A}$ via the relations $\Gamma_{0}(f,g)=fg$ and

In particular $\Gamma_{1}=\Gamma$ and the operators $\Gamma_{n}$, $n\geq 1$, are similarly symmetric and bilinear. In what follows, we shall often adopt the shorthand notation $\Gamma_{n}(f)$ instead of $\Gamma_{n}(f,f)$. The $\Gamma_{2}$ operator is part of the famous Bakry-Émery criterion for logarithmic Sobolev inequalities [B-E], [B-G-L, Section 5.7]. As a new feature of the analysis here, the iterated gradient $\Gamma_{3}$ will turn essential towards a suitable analogue of (iii) in Proposition 2.4.

Given thus the preceding Markov Triple $(E,\mu,\Gamma)$ associated to the second order differential operator $\mathcal{L}$ of (4.1), let $d\nu=hd\mu$ where $h$ is a smooth probability density with respect to $\mu$. As in the Gaussian case, the relative entropy of $\nu$ with respect to $\mu$ is the quantity

Similarly, the Fisher information of $\nu$ (or $h$) with respect to $\mu$ is defined as

The (integrated) de Bruijn’s identity (cf. Proposition 5.2.2 in [B-G-L]) reads as in (i) of Proposition 2.4,

is a Stein kernel for the probability $\nu$ on $E$ with respect to the generator of $\cal L$ of (4.1), where $b={(b_{i}(x))}_{1\leq i\leq d}$ is part of the definition of $\cal L$. For the Ornstein-Uhlenbeck operator ${\cal L}=\Delta-x\cdot\nabla$, the definition corresponds to (2.1). Since $\int_{E}\mathcal{L}f\,d\mu=0$, observe that $a$ is a Stein kernel for $\mu$. The main result in this section is an HSI inequality that relates ${\rm H}(\nu\,|\,\mu)$, ${\rm I}(\nu\,|\,\mu)$ and the Stein discrepancy of $\nu$ with respect to $\mu$

that we regard, as in the Gaussian case of Section 2, as a measure of the distance between $\nu$ and $\mu$ (since $\tau_{\mu}=a$). Note that choosing $a=C$ in (4.4), with $C$ non-singular, yields the quantity arising in Corollary 2.3. It should also be mentioned that the Stein discrepancy (4.4) is somewhat in contrast with the bounds one customarily obtains when applying Stein’s method (see e.g. [N-P1] for the specific example of the one-dimensional Gamma distribution, or [R] for a general reference), which typically involve quantities of the type $\int_{E}\|\tau_{\nu}-a\|^{2}_{\rm HS}\,d\nu$. The appearance of the inverse matrices $a^{-\frac{1}{2}}$ seems to be inextricably connected with the fact that we deal with information-theoretical functionals.

The following general statement collects the necessary assumptions on the iterated gradients $\Gamma$, $\Gamma_{2}$ and $\Gamma_{3}$ to achieve the expected HSI inequality by the semigroup interpolation scheme. The next paragraphs will provide illustrations in various concrete instances of interest. In Theorem 4.1 below, (i) amounts to the Bakry-Émery $\Gamma_{2}$ criterion to ensure the logarithmic Sobolev inequality (cf. [B-G-L, Section 5.7]) while condition (ii) linking the $\Gamma_{2}$ and $\Gamma_{3}$ operators will provide (together with (iii)) the suitable semigroup bound for the time control of ${\rm I}(P_{t}h)$ away from . Recall $\Psi(r)=1+\log r$ if $r\geq 1$ and $\Psi(r)=r$ if $0\leq r\leq 1$.

In the preceding context, let $d\nu=hd\mu$ where $h$ is a smooth density with Stein kernel $\tau_{\nu}$ with respect to $\mu$. Assume that there exists $\rho,\kappa,\sigma>0$ such that, for any $f\in\mathcal{A}$,

$\Gamma_{3}(f)\geq\kappa\,\Gamma_{2}(f)$;

$\Gamma_{2}(f)\geq\sigma\,\|a^{\frac{1}{2}}\,{\rm Hess}(f)\,a^{\frac{1}{2}}\|_{\rm HS}^{2}$ (with $a$ as in (4.1)).

Note that in the Ornstein-Uhlenbeck example, $\rho=\kappa=\sigma=1$ from which we recover the HSI inequality (2.6), however in a slightly weaker formulation.

It is therefore a classical fact (see e.g. [B-G-L, (5.7.4)]) that (i) ensures the exponential decay of the Fisher information along the semigroup

for every $t\geq 0$ (and then yields a logarithmic Sobolev inequality for $\mu$.) Now, fix $t>0$ and let $f\in\mathcal{A}$. The $\Gamma$-calculus as developed in [B-G-L], but at the level of the $\Gamma_{2}$ and $\Gamma_{3}$ operators, yields on $[0,t]$ (by the very definition of $\Gamma_{3}$ from $\Gamma_{2}$),

By (ii), the latter is non-negative so that the map $s\mapsto P_{s}(\Gamma_{2}(P_{t-s}f))e^{-2\kappa s}$ is increasing on $[0,t]$, and thus

Together with (iii), it then follows that

We shall apply (4.6) to $v_{t}=\log P_{t}h$ (with $h$ regular enough). First, by symmetry of $\mu$ with respect to ${(P_{t})}_{t\geq 0}$,

where the last step follows from (4.6). Since

Finally, using (4.5) for small $t$ and (4.8) for large $t$, one deduces that, for every $u>0$,

Now, using that $1-r^{\rho}\leq\max(1,\frac{\rho}{\kappa})(1-r^{\kappa})$ for $r\in(0,1)$, a simple (non-optimal) optimization yields the desired conclusion. The proof of Theorem 4.1 is complete. ∎

It should be pointed out that, on the basis of (4.8), transport inequalities as studied in Section 3 may be investigated similarly in the preceding general context, and with similar illustrations as developed below. For example, as an analogue of (3.1),

In order not to expand too much the exposition, we leave the details to the reader.

The next paragraphs present various illustrations of Theorem 4.1.

2 Multivariate gamma distribution

After some easy but cumbersome calculations, it may be checked that, along suitable smooth functions $f$,

Note that (recall $x_{i},x_{j},x_{k}\geq 0$)

Since $p_{i}\geq\frac{3}{2}$, it follows at once that $\Gamma_{3}(f)\geq\frac{1}{2}\,\Gamma_{2}(f)$. Analogous computations lead to

As a consequence, Theorem 4.1 applies with $\rho=\kappa=\sigma=\frac{1}{2}$ to yield the following result (the numerical constants there are not sharp). The restrictions $p_{i}\geq\frac{3}{2}$, ${i=1,\ldots,d}$, are probably not optimal. For example, it is not difficult to see from the preceding computations that in the one-dimensional case $d=1$, it is actually enough to assume that $p\geq\frac{1}{2}$.

3 One-dimensional uniform distribution on [−1,+1]11[-1,+1]

1[-1,+1] In this section, we examine the case of the one-dimensional Jacobi operator of parameters $\alpha=\beta=1$, that is,

whose associated invariant measure $\mu$ is uniform distribution on $[-1,+1]$. The general family of parameters with the beta distributions as invariant measures (cf. [B-G-L, Section 2.7.4]) may be considered similarly, at the expense however of tedious computations, as well as multivariate (product) versions. For simplicity, we only detail this case to better illustrate the conclusion.

Easy calculations lead to, for a smooth function $f$ on $[-1,+1]$,

so that $\Gamma_{3}(f)\geq\Gamma_{2}(f)$. Also,

Hence, Theorem 4.1 applies with $\rho=\kappa=1$ and $\sigma=\frac{1}{2}$ (note that $a(x)=1-x^{2}$) to yield the following conclusion. Again, the numerical constants are not sharp.

Let $\mu$ be uniform probability measure on $[-1,+1]$. Then, for any $d\nu=hd\mu$ where $h$ is a smooth probability density,

4 Families of log-concave distributions

We consider here a diffusion operator on the line of the type

Assume that there exists $c>0$ such that, uniformly, $u^{\prime\prime}\geq c$,

Then $\Gamma_{2}(f)\geq c\,\Gamma(f)$, $\Gamma_{2}(f)\geq{f^{\prime\prime}}^{2}$ and $\Gamma_{3}(f)\geq 3c\,\Gamma_{2}(f)$ for every $f$. Hence, Theorem 4.1 applies with $\rho=c$, $\kappa=3c$ and $\sigma=1$.

Recall that in this context, the only condition $u^{\prime\prime}\geq c>0$ ensures the logarithmic Sobolev inequality for $\mu$ [B-G-L, Corollary 5.7.2]. It is not difficult to find (simple) examples outside the Gaussian model (corresponding to $c=\frac{1}{3}$) such that conditions (4.9) and (4.10) are fulfilled. For example, if $u(x)=\frac{x^{2}}{2}+\varepsilon x^{4}$, it is easily seen that these hold for $c=\frac{1}{4}$ and $\varepsilon=\frac{1}{12}$ (for instance). In the Gaussian case, the estimate obtained in this proposition is somewhat worse than the HSI inequality of Theorem 2.2. At the expenses of more involved conditions (4.9) and (4.10), multidimensional versions may be considered similarly.

Entropy bounds on laws of functionals

Referring as before to [B-G-L] for a complete account, we thus deal with a Markov Triple $(E,\mu,\Gamma)$ on a probability space $(E,{\cal E},\mu)$, with Markov semigroup ${(P_{t})}_{t\geq 0}$ with symmetric and invariant probability measure $\mu$, infinitesimal generator $\rm L$, associated carré du champ operator $\Gamma$ and underlying algebra of (smooth) functions $\cal A$. Integration by parts expresses that

The second order differential operators of Section 4 provide instances of this general framework. Gaussian and Wiener spaces with associated Ornstein-Uhlenbeck semigroup and generator are a prototypical example for the illustrations. Note in particular that Wiener chaoses as investigated in [N-P-S1] are eigenfunctions of the Ornstein-Uhlenbeck generator. Eigenfunctions of the underlying operator $\rm L$ are actually of special interest in the context of the Stein method as illustrated in Section 5.1.

The first statement shows that, whenever the vector $F$ is composed of eigenfunctions of ${\rm L}$, a Stein kernel $\tau_{\nu_{F}}$ of $\nu_{F}$ with respect to $\gamma$ as defined in (2.1) can be expressed in terms of the carré du champ operator $\Gamma$.

Let $F=(F_{1},\ldots,F_{d})$ on $(E,{\cal E},\mu)$ such that, for every $i=1,\ldots,d$, the random variable $F_{i}$ is an eigenfunction of $-{\rm L}$, with eigenvalue $\lambda_{i}>0$. Assume moreover that $\Gamma(F_{i},F_{j})\in{\rm L}^{1}(\mu)$ for every $i,j=1,\ldots,d$. Then, the matrix-valued map $\tau_{\nu_{F}}$ defined as

is a Stein kernel for $\nu_{F}$, that is, it satisfies (2.1). (The right-hand side of (5.2) indicates a version of the conditional expectation of $\Gamma(F_{i},F_{j})$ with respect to $F$ under the probability measure $\mu$.)

The proof is concluded by taking conditional expectations. ∎

As a consequence, together with (2.5) and Jensen’s inequality,

where $C$ denotes the covariance matrix of $\nu_{F}$, providing therefore a tractable way to control the Stein discrepancy in this case. In addition, combining with the HSI inequality of Theorem 2.2 immediately yields the following statement.

Under the assumptions and notation of Proposition 5.1,

In particular, if $d=1$ and $C=1$, ${\rm H}(\nu_{F}\,|\,\gamma)\to 0$ whenever ${\rm Var}(\Gamma(F))\to 0$ (cf. [N-P2, L3]).

A typical example of a Markov Triple for which the quantity ${\rm V}^{2}$ appearing in the above bound can be estimated explicitly corresponds to the case where $(E,\mathcal{E},\mu)$ is a probability space supporting an isonormal Gaussian process $X=\{X(h):h\in\mathfrak{H}\}$ over some real separable Hilbert space $\mathfrak{H}$, and ${\rm L}$ is the generator of the associated Ornstein-Uhlenbeck semigroup. In this case, $\Gamma(F,G)={\langle DF,DG\rangle}_{\mathfrak{H}}$ for smooth functionals $F$ and $G$, where $D$ stands for the Malliavin derivative operator, and the eigenspaces of $-{\rm L}$ are the so-called Wiener chaoses $\{C_{k}:k\geq 0\}$ of $X$. For $k=0,1,2,\ldots$, the eigenvalue of $C_{k}$ is given by $k$. A detailed discussion about how to bound a quantity such as ${\rm V}^{2}$ in the case of random vectors with components inside a Wiener chaos can be found in [N-P2, Chapter 6]. In particular, if $d=1$ and $F$ belongs to $C_{k}$, then ${\rm V}^{2}$ can be controlled by the second and fourth moments of $F$ as

In particular, such an estimate provides a proof of the famous ‘fourth moment theorem’ for chaotic random variables, cf. [N-P2, Theorem 5.2.7].

While eigenfunctions appear as functionals of particular interest for the control of the Stein discrepancy itself, the $\Gamma$-calculus actually provides a formal description of Stein kernels of a given functional $F$ on $(E,\mu,\Gamma)$ (in dimension one for simplicity) as the conditional expectation with respect to $F$ of $\Gamma(F,{\rm L}^{-1}F)$ (where ${\rm L}^{-1}F=\int_{0}^{\infty}P_{t}Fdt$). This observation further expands on the preceding example, allowing for a rather general analysis.

2 Bounds on the Fisher information

When dealing with the upper-bound (5.4), the Fisher information ${\rm I}(\nu_{F}\,|\,\gamma)={\rm I}_{\gamma}(h)$ of the density $h$ of the law $\nu_{F}$ of $F$ cannot always be explicitly deduced from the data concerning the random vector $F$. The task of this paragraph is therefore to deduce some useful bounds on ${\rm I}(\nu_{F}\,|\,\gamma)$ in terms of $F$ and its gradients.

Let ${\widetilde{\Gamma}}$ be the symmetric matrix with entries $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$. Applying the latter to $w=w_{ij}$, symmetric in $i,j$, yields

Applied to $\phi=v=\log h$, by the Cauchy-Schwarz inequality and (4.2),

The consequences of the previous computations are gathered together in the next statement, where we point out a set of sufficient conditions on $F$ and its gradients $\Gamma(F_{i},F_{j})$ ensuring that the random variable $U$ is indeed square-integrable.

Let $F=(F_{1},\ldots,F_{d})$ be a vector of elements of $\cal A$ on $(E,\mu,\Gamma)$. Assume that all the $F_{i}$, ${\rm L}F_{i}$, $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$, and $\frac{1}{{\rm det}({\widetilde{\Gamma}})}$ are in ${\rm L}^{p}(\mu)$ for every $p\geq 1$. Then, $\int_{E}|U|^{2}d\mu<\infty$ and

The condition on $\frac{1}{{\rm det}({\widetilde{\Gamma}})}$ in Proposition 5.5 has some similarity with basic assumptions in Malliavin calculus (cf. [N, N-P2]).

3 Fisher information growth and normal approximation

One evident drawback of Proposition 5.5 of the previous paragraph is that, since the quantity $|U|$ is singular as the determinant of ${\widetilde{\Gamma}}$ is close to , one is forced to assume that $\frac{1}{{\rm det}({\widetilde{\Gamma}})}$ is in all ${\rm L}^{p}(\mu)$ spaces (or at least for some $p$ large enough depending on $d$). This assumption is in general too strong, and very difficult to check in concrete situations. The idea developed in this section (which generalizes the approach initiated in [N-P-S1]) is that, under weaker moment assumptions, while the Fisher information ${\rm I}_{\gamma}(h)$ might be infinite, it is nevertheless possible to control the growth as $t\to 0$ of ${\rm I}_{\gamma}(P_{t}h)$. Together with the control in terms of the Stein discrepancy for large time achieved in Section 2, one may then reach entropic bounds which can be handled in concrete examples (such as those of random vectors whose components belong to some Wiener chaos).

As before, let $F=(F_{1},\ldots,F_{d})$ be a general vector of centered and square-integrable random variables (in the algebra $\cal A$ or some natural extension), with distribution $d\nu_{F}=hd\gamma$. As a crucial assumption, $\nu_{F}$ has a Stein kernel $\tau_{\nu_{F}}$ with respect to $\gamma$ as defined in (2.1) (see also Proposition 5.1 and Remark (5.4)). Recall the matrix ${\widetilde{\Gamma}}$ with entries $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$. Also, in what follows we use the convention that, if $\widetilde{\Gamma}$ is singular, then the matrix ${\rm det}({\widetilde{\Gamma}})\,{\widetilde{\Gamma}}^{-1}$ must be understood as the transpose of usual adjugate matrix operator of $\widetilde{\Gamma}$ (both quantities being of course equal for non-singular matrices).

Choose $W=\frac{{\rm det}({\widetilde{\Gamma}})\,{\widetilde{\Gamma}}^{-1}}{{\rm det}({\widetilde{\Gamma}})+\varepsilon}$ in (5.5), so that

Apply now the preceding to $\phi=P_{t}v_{t}$, $v_{t}=\log P_{t}h$, $t>0$. Since $\nabla P_{t}v_{t}(F)=e^{-t}P_{t}(\nabla v_{t})$ and

by the Cauchy-Schwarz inequality, assuming for simplicity that $0<\varepsilon\leq 1$,

On the other hand, using the same semigroup computations as in Section 2,

Collecting the preceding bounds and recalling from (4.7) that

yields that, for $t>0$ and $0<\varepsilon\leq 1$,

Let $F=(F_{1},\ldots,F_{d})$ be a vector of centered elements of $\cal A$ on $(E,\mu,\Gamma)$. Assume that all the $F_{i}$, ${\rm L}F_{i}$, $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$, are in ${\rm L}^{p}(\mu)$ for every $p\geq 1$, and that

for some $\alpha>0$. Then, $A_{F}<\infty$ (as defined in (5.7)) and

where $\kappa=\frac{2+\alpha}{2(4+3\alpha)}\,(<\frac{1}{4}\,)$. In particular, under the assumptions on $F$, ${\rm H}(\nu_{F}\,|\,\gamma)\to 0$ as ${\rm S}(\nu_{F}\,|\,\gamma)\to 0$.

First of all, we have that the parameter $A_{F}$ is finite, since the expressions ${\rm det}({\widetilde{\Gamma}}){\widetilde{\Gamma}}^{-1}{\rm L}F$, $V_{1}$ and $V_{2}$ only involve products of $F_{i}$, ${\rm L}F_{i}$ and $\Gamma(F_{i},F_{j})$, $i,j=1,\ldots,d$.