Bounds on the constant in the mean central limit theorem

Introduction

The classical central limit theorem allows the approximation of the distribution of sums of “comparable” independent real-valued random variables by the normal. As this theorem is an asymptotic, it provides no information as to whether the resulting approximation is useful. For that purpose, one may turn to the Berry–Esseen theorem, the most classical version giving supremum norm bounds between the distribution function of the normalized sum and that of the standard normal. Various authors have also considered Berry–Esseen-type bounds using other metrics and, in particular, bounds in $L^{p}$. The case $p=1$, where the value

is used to measure the distance between distribution functions $F$ and $G$, is of some particular interest and results using this metric are known as mean central limit theorems (see, e.g., dedecker , Erickson , HoChen and Ibragimov ; the latter three of these works consider nonindependent summand variables). One motivation for studying $L^{1}$-bounds is that, combined with one of type $L^{\infty}$, bounds on $L^{p}$-distance for all $p\in(1,\infty)$ may be obtained by the inequality

For $\sigma\in(0,\infty)$, let $\mathcal{F}_{\sigma}$ be the collection of distributions with mean zero, variance $\sigma^{2}$ and finite absolute third moment. We prove the following Berry–Esseen-type result for the mean central limit theorem.

In particular, when $X_{1},\ldots,X_{n}$ are identically distributed with distribution $G\in\mathcal{F}_{\sigma}$,

For the case where all variables are identically distributed as $X$ having distribution $G$, letting

the second part of Theorem 1.1 yields the upper bound $c_{1}\leq 1$. Regarding lower bounds, we also prove

Clearly, the elements of the sequence $\{c_{m}\}_{m\geq 1}$ are nonnegative and decreasing in $m$, and so have a limit, say $c_{\infty}$. Regarding limiting behavior, Esseen esseen showed that

for an explicit constant $A(G)$ depending only on $G$. Zolotarev Zolotarev provides the representation

where $\omega=|EX^{3}|/(3\sigma^{2})$ and $h$ is the span of the distribution $G$ in the case where $G$ is lattice, and is zero otherwise. Zolotarev obtains

showing that $c_{\infty}=1/2$, hence giving the asymptotic $L^{1}$-Berry–Esseen constant value.

for all absolutely continuous functions $f$ for which these expectations exist. The zero bias transformation, mapping the distribution of $X$ to that of $X^{*}$, was motivated by the Stein characterization of the normal distribution Stein81 , which states that $Z$ is normal with mean zero and variance $\sigma^{2}$ if and only if

for all absolutely continuous functions $f$ for which these expectations exist. Hence, the mean zero normal with variance $\sigma^{2}$ is the unique fixed point of the zero bias transformation. How closeness to normality may be measured by the closeness of a distribution to its zero bias transform, and related applications, are the topics of res , zsm and L1bounds .

As shown in L1bounds and Goldstein-Reinert , for a random variable $X$ with $EX=0$ and $\operatorname{Var}(X)=\sigma^{2}$, the distribution of $X^{*}$ is absolutely continuous with density and distribution functions given, respectively, by

Theorem 1.1 results by showing that the functional

is bounded by 1 for all $X$ with distribution $G\in\mathcal{F}_{\sigma}$. As in (3), one may write out a more “explicit” form for $B(G)$ using (6) and expressions for the moments on which $B(G)$ depends, but such expressions appear to be of little value for the purposes of proving Theorem 1.1. In turn, the proof here employs convexity properties of $B(G)$ which depend on the behavior of the zero bias transformation on mixtures. We also note that the functional $B(G)$ has a different character than $A(G)$; for instance, $A(G)$ is zero for all nonlattice distributions with vanishing third moment, whereas $B(G)$ is zero only for mean zero normal distributions.

by replacing $\sigma_{i}^{2}$ by $\sigma_{i}^{2}/\sigma^{2}$ and $\|G_{i}^{*}-G_{i}\|_{1}$ by $\|G_{i}^{*}-G_{i}\|_{1}/\sigma$ in equation (16) of Theorem 2.1 of L1bounds , we obtain the following.

Under the hypotheses of Theorem 1.1, we have

For $\mathcal{F}$ a collection of nontrivial mean zero distributions with finite absolute third moments, we let

Clearly, Theorem 1.1 follows immediately from Proposition 1.1 and the following result.

The equality to 1 in Lemma 1.1 improves the upper bound of 3 shown in L1bounds . Although our interest here is in best universal constants, we note that Proposition 1.1 shows that $B(G)$ is a distribution-specific $L^{1}$-Berry–Esseen constant, in that

when $X_{1},\ldots,X_{n}$ are identically distributed according to $G\in\mathcal{F}_{\sigma}$. For instance, $B(G)=1/3$ when $G$ is a mean zero uniform distribution and $B(G)=1$ when $G$ is a mean zero two-point distribution; see Corollary 2.1 of L1bounds , and Lemmas 1.2 and 1.3 below.

We close this section with two preliminaries. The first collects some facts shown in L1bounds and the second demonstrates that to prove Lemma 1.1, it suffices to consider the class of random variables $\mathcal{F}_{1}$. Then, following Hoeffding Hoeffding (see also Utev ), in Section 2, we use a continuity property of $B(G)$ to show that its supremum over $\mathcal{F}_{1}$ is attained on finitely supported distributions. Exploiting a convexity-type property of the zero bias transformation on mixtures over distributions having equal variances, we reduce the calculation further to the calculation of the supremum over $D_{3}$, the collection of all mean zero distributions with variance 1 and supported on at most three points. As three-point distributions are, in general, a mixture of two two-point distributions with unequal variances, an additional argument is given in Section 3, where a coupling of an $X$ with distribution $G\in D_{3}$ to a variable $X^{*}$ having the $X$ zero bias distribution is constructed, using the optimal $L^{1}$-couplings on the component two-point distributions of which $G$ is the mixture, in order to obtain $B(G)\leq 1$ for all $G\in D_{3}$. The lower bound (2) on $c_{1}$ is calculated in Section 4.

The following simple formula will be of some use. For $a\geq 0,b>0$ and $l>0$, we have

Let $G$ be the distribution of a nontrivial mean zero random variable $X$ supported on the two points $x<y$. Then $X^{*}$ is uniformly distributed on $[x,y]$,

Being nontrivial, $G$ has positive variance and, from (6), we see that the density $g^{*}$ of $G^{*}$ at $u$, which is proportional to $E[X\mathbf{1}(X>u)]$, is zero outside $[x,y]$ and constant within it, so $G^{*}(w)=(w-x)/(y-x)$ for $w\in[x,y]$. That $G$ has mean zero implies that the support points $x$ and $y$ satisfy $x<0<y$ and that $G$ gives positive probabilities $y/(y-x)$ and $-x/(y-x)$ to $x$ and $y$, respectively. The moment identities are immediate.

Making the change of variable $u=w-x$ and applying (9) with $a=y/(y-x),b=-x/(y-x)$ and $l=y-x$ yields

Let $G\in\mathcal{F}_{\sigma}$ for some $\sigma\in(0,\infty)$, let $X$ have distribution $G$ and, for $a\not=0$, let $G_{a}$ denote the distribution of $aX$. Then $B(G_{a})=B(G)$ and, in particular,

That $aX^{*}$ has the same distribution as $(aX)^{*}$ follows from (4). The identities $\sigma_{aX}^{2}=a^{2}\sigma_{X}^{2}$, $E|aX|^{3}=|a|^{3}E|X^{3}|$ and (8) now imply the first claim. The second claim now follows from

Reduction to three-point distributions

is measurable. When $\mu$ is a probability measure on $(S,\Sigma)$, the set function given by

is a probability measure, called the $\mu$ mixture of $\{m_{s}\}_{s\in S}$. With some slight abuse of notation, we let $E_{\mu}$ and $E_{s}$ denote expectations with respect to $m_{\mu}$ and $m_{s}$, and let $X_{\mu}$ and $X_{s}$ be random variables with distributions $m_{\mu}$ and $m_{s}$, respectively. For instance, for all functions $f$ which are integrable with respect to $\mu$, we have

In particular, if $\{m_{s}\}_{s\in S}$ is a collection of mean zero distributions with variances $\sigma_{s}^{2}=EX_{s}^{2}$ and absolute third moments $\gamma_{s}=E|X_{s}^{3}|$, the mixture distribution $m_{\mu}$ has variance $\sigma_{\mu}^{2}$ and third absolute moment $\gamma_{\mu}$ given by

where both may be infinite. Note that $\sigma_{\mu}^{2}<\infty$ implies that $\sigma_{s}^{2}<\infty$ $\mu$-almost surely and, therefore, that $m_{s}^{*}$, the $m_{s}$ zero bias distribution, exists $\mu$-almost surely.

Theorem 2.1 shows that the zero bias distribution of a mixture is a mixture of zero bias distributions, the mixing measure of which is the original measure weighted by the variance and rescaled. Define (arbitrarily, see Remark 2.1) the zero bias distribution of $\delta_{0}$, a point mass at zero, to be $\delta_{0}$. Write $X=_{d}Y$ when $X$ and $Y$ have the same distribution.

In particular, $\nu=\mu$ if and only if $\sigma_{s}^{2}$ is a constant $\mu$ a.s.

The distribution $m_{\mu}^{*}$ exists as $m_{\mu}$ has mean zero and finite nonzero variance. Let $X_{\mu}^{*}$ have the $m_{\mu}$ zero bias distribution and let $Y$ have distribution $m_{\mu}^{*}$. For any absolutely continuous function $f$ for which the expectations below exist, we have

Since $Ef^{\prime}(X_{\mu}^{*})=Ef^{\prime}(Y)$ for all such $f$, we conclude that $X_{\mu}^{*}=_{d}Y$.

If $m_{s}=\delta_{0}$ for any $s\in S$, then $\sigma_{s}^{2}=0$ and, therefore,

Hence, the mixture $X_{\mu}^{*}$ gives zero weight to all corresponding $m_{s}^{*}$, showing that $(\delta_{0})^{*}$ may be defined arbitrarily.

$X$ and $Y$ having distributions $F$ and $G$, respectively. With a slight abuse of notation, we may write $B(X)$ in place of $B(G)$ when $X$ has distribution $G$.

If $X_{\mu}$ is the $\mu$ mixture of a collection $\{X_{s},s\in S\}$ of mean zero, variance 1 random variables satisfying $E|X_{\mu}^{3}|<\infty$, then

If $\mathcal{C}$ is a collection of mean zero, variance 1 random variables with finite absolute third moments and $\mathcal{D}\subset\mathcal{C}$ such that every distribution in $\mathcal{C}$ can be represented as a mixture of distributions in $\mathcal{D}$, then

Since the variances $\sigma_{s}^{2}$ of $X_{s}$ are constant, the distribution $X_{\mu}^{*}$ is the $\mu$ mixture of $\{X_{s}^{*},s\in S\}$, by Theorem 2.1. Hence, applying (2), we have

Noting that $\operatorname{Var}(X_{\mu})=\int_{S}EX_{s}^{2}\,d\mu=1$ and applying (2), we find that

Regarding (13), clearly, $B(\mathcal{D})\leq B(\mathcal{C})$ and the reverse inequality follows from (12).

Note that no bound of the type provided by Theorem 2.2 holds, in general, when taking mixtures of variables that have unequal variances. In particular, if $X_{s}\sim\mathcal{N}(0,\sigma_{s}^{2})$ and $\sigma_{s}^{2}$ is not constant in $s$, then $X_{\mu}$ is a mixture of normals with unequal variances, which is not normal. Hence, in this case, $B(X_{\mu})>0$, whereas $B(X_{s})=0$ for all $s$.

To apply Theorem 2.2 to reduce the computation of $B(\mathcal{F}_{1})$ to finitely supported distributions, we apply the following continuity property of the zero bias transformation; see Lemma 5.2 in lin . We write $X_{n}\Rightarrow X$ for the convergence of $X_{n}$ to $X$ in distribution.

Let $X$ and $X_{n},n=1,2,\ldots,$ be mean zero random variables with finite, nonzero variances. If

If $U$ is uniform on $$, then$F^{-1}(U)$has distribution function$F$. If$X_{n}$and$X$have distribution functions$F_{n}$and$F$, respectively, and$X_{n}\Rightarrow X$, then$F_{n}^{-1}(U)\rightarrow F^{-1}(U)$a.s. (see, e.g., Theorem 2.1 of durrett , Chapter 2). For distribution functions$F$and$G$, we have

where the infimum is over all joint distributions on $X,Y$ which have marginals $F$ and $G$, respectively, and the variables $F^{-1}(U)$ and $G^{-1}(U)$ achieve the minimal $L^{1}$-coupling, that is,

With the use of Lemma 2.1, we are able to prove the following continuity property of the functional $B(X)$.

By Lemma 2.1, we have $X_{n}^{*}\Rightarrow X^{*}$. Let $U$ be a uniformly distributed variable and set

By (4) with $f(x)=x^{2}\mbox{sgn}(x)$, we find, for $Y$, for example, that

Hence, as $n\rightarrow\infty$, we have $EY_{n}^{2}=EX_{n}^{2}\rightarrow EX^{2}=EY^{2}$ and

Combining (2) with the convergence of the variances and the absolute third moments, as provided by (18), the proof is complete.

Lemmas 2.3 and 2.4 borrow much from Theorem 2.1 of Hoeffding , the latter lemma indeed being implicit. However, the results of Hoeffding cannot be applied directly as $B(G)$ is not expressed as the expectation of $K(X)$ for some $K$ when $\mathcal{L}(X)=G$. For $m\geq 2$, let $D_{m}$ denote the collection of all mean zero, variance 1 distributions which are supported on at most $m$ points.

Letting $\mathcal{M}$ be the collection of distributions in $\mathcal{F}_{1}$ which have compact support, we first show that

we have $X_{n}\Rightarrow X$, by Slutsky’s theorem, so, in view of (2), the hypotheses of Lemma 2.2 are satisfied, yielding

Since $|X|\leq M$ a.s., each $Y_{n}$ is supported on finitely many points and uniformly bounded. Clearly, $Y_{n}\rightarrow X$ a.s. and (2) holds by the bounded convergence theorem. Now, defining $X_{n}$ by (22), the hypotheses of Lemma 2.2 are satisfied, yielding

showing $B(\mathcal{M})\leq B(\bigcup_{m\geq 3}D_{m})$. Combining this inequality with (20) yields $B(\mathcal{F}_{1})\leq B(\bigcup_{m\geq 3}D_{m})$ and therefore the lemma, the reverse inequality being obvious.

Every distribution in $\bigcup_{m\geq 3}D_{m}$ can be expressed as a finite mixture of $D_{3}$ distributions.

The lemma is trivially true for $m=3$, so consider $m>3$ and assume that the lemma holds for all integers from $3$ to $m-1$.

The distribution of any $X\in D_{m}$ is determined by the supporting values $a_{1}<\cdots<a_{m}$ and a vector of probabilities $\mathbf{p}=(p_{1},\ldots,p_{m})^{\prime}$. If any of the components of $\mathbf{p}$ are zero, then $X\in D_{k}$ for $k<m$ and the induction would be finished, so we assume that all components of $\mathbf{p}$ are strictly positive. As $X\in D_{m}$, the vector $\mathbf{p}$ must satisfy

Since $\mathbf{v}\not=0$ and the equation specified by the first row of $A$ is $\sum_{i}v_{i}=0$, the vector $\mathbf{v}$ contains both positive and negative numbers. Since the vector $\mathbf{p}$ has strictly positive components, the numbers $t_{1}$ and $t_{2}$ given by

by (23), so that $\mathbf{p}_{1}$ and $\mathbf{p}_{2}$ are probability vectors since their components are nonnegative and sum to one. Additionally, the corresponding distributions have mean zero and variance 1, and in each of these two vectors, at least one component has been set to zero. Hence, we may express the $m$-point probability vector $\mathbf{p}$ as the mixture

of probability vectors on at most $m-1$ support points, thus showing $X$ to be the mixture of two distributions in $D_{m-1}$, completing the induction.

The following theorem is an immediate consequence of Theorem 2.2 and Lemmas 2.3 and 2.4.

Hence, we now restrict our attention to $D_{3}$.

Lemma 1.3 and Theorem 2.3 imply that $B(\mathcal{F}_{\sigma})=B(\mathcal{F}_{1})=B(D_{3})$. Hence, Lemma 1.1 follows from Theorem 3.1 below, which shows that $B(D_{3})=1$. We prove Theorem 3.1 with the help of the following result.

Let $x<y<0<z$, and let $m_{1}$ and $m_{0}$ be the unique mean zero distributions with support $\{x,z\}$ and $\{y,z\}$, respectively, that is,

Let $F_{1},F_{0}$ and $F_{1}^{*}$ denote the distribution functions of $m_{1},m_{0}$ and $m_{1}^{*}$, respectively. By Lemma 1.2, $m_{1}^{*}$ is uniform over $[x,z]$. There are two cases, depending on the relative magnitudes of $F_{1}^{*}(y)=(y-x)/(z-x)$ and $F_{0}(y)=z/(z-y)$.

Letting $J_{1}=[x,y)$ and $J_{2}=[y,z]$, we have

Since $F_{1}^{*}(w)\geq 0=F_{0}(w)$ for all $w\in J_{1}$,

Recalling that $F_{1}^{*}(y)\leq F_{0}(y)$, applying (9) with $a=\frac{z}{z-y}-\frac{y-x}{z-x}$, $b=-\frac{y}{z-y}$ and $l=z-y$, after the change of variable $u=w-y$, yields

Now, subtracting from (3) and simplifying by noting that the terms inside the parentheses in the numerators of these two expressions are equal, we find that

The denominator in (3) is positive, as is $-y$ and $y-x$. For the remaining term, (25) yields

Hence, (3) is positive, thus proving (24) when $F_{1}^{*}(y)\leq F_{0}(y)$.

When $F_{1}^{*}(y)>F_{0}(y)$, we have $F_{1}^{*}(w)\geq F_{0}(w)$ for all $w\in[x,z)$ as $F_{0}(w)$ is zero in $[x,y)$ and equals $F_{0}(y)$ in $[y,z)$, and $F_{1}^{*}(w)$ is increasing over $[y,z)$. Hence,

Now, since $(x+z)(x-z)=x^{2}-z^{2}\leq z^{2}+x^{2}$ and $z-x>0$, using Lemma 1.2, we obtain

thus proving inequality (24) when $F_{1}^{*}(y)>F_{0}(y)$ and, therefore, proving the lemma.

Lemma 1.2 shows that $B(X)=1$ if $X$ is supported on two points, so $B(D_{3})\geq 1$ and it only remains to consider $X$ positively supported on three points. We first prove that

$EX=0$ implies that $x<0<z$. After proving (3), we treat the remaining case, where $y=0$, by a continuity argument.

Let $X$ be supported on $x<y<z$ with $y\not=0$. Lemma 1.3 with $a=-1$ implies that $B(-X)=B(X)$, so we may assume, without loss of generality, that $x<y<0<z$. Let $m_{1}$ and $m_{0}$ be the unique mean zero distributions supported on $\{x,z\}$ and $\{y,z\}$, respectively, and let $\mathcal{L}(X_{1})=m_{1}$ and $\mathcal{L}(X_{0})=m_{0}$. As, in general, every mean zero distribution having no atom at zero can be represented as a mixture of mean zero two-point distributions (as in the Skorokhod representation, see durrett ), letting

we have $\mathcal{L}(X)=\mathcal{L}(X_{\alpha})$ for some $\alpha\in$; in fact, for the given $X$, one may verify that $P(X=x)/P(X_{1}=x)\in(0,1)$ and that (32) holds when $\alpha$ assumes this value. Therefore, to prove (3), it suffices to show that

and, by (32), the variance of $X_{\alpha}$ is given by

Applying Theorem 2.1 with $S=\{0,1\}$ and $\mu$ being the probability measure putting mass $\alpha$ and $1-\alpha$ on the points $1$ and , respectively, in view of (34) and (3), $m_{\alpha}^{*}$, the $X_{\alpha}$ zero bias distribution, is given by the mixture

Let $F_{1},F_{0},F_{1}^{*}$ and $F_{0}^{*}$ denote the distribution functions of $m_{1},m_{0},m_{1}^{*}$ and $m_{0}^{*}$, respectively. Let $U$ be a standard uniform variable and, with the inverse functions below given by (15), set

Then $Y_{i}=_{d}X_{i}$, $Y_{i}^{*}=_{d}X_{i}^{*}$ for $i\in\{1,2\}$ and, by (17), all pairs of the variables $Y_{1},Y_{0},Y_{1}^{*},Y_{0}^{*}$ achieve the $L^{1}$-distance between their respective distributions. Now, let $(Y_{\alpha},Y_{\alpha}^{*})$ be defined on the same space with joint distribution given by the mixture

Then $(Y_{\alpha},Y_{\alpha}^{*})$ has marginals $Y_{\alpha}=_{d}X_{\alpha}$ and $Y_{\alpha}^{*}=_{d}Y_{\alpha}^{*}$, hence, by (16),

Lemma 1.2 shows that $G(X_{i})=1$, that is, $E|X_{i}^{3}|=2EX_{i}^{2}\|m_{i}^{*}-m_{i}\|_{1}$ for $i=1,2$, so (32) yields

and, by (34), (3) and (36), we now find that

Lemma 3.1 shows that the right-hand side and, therefore, the left-hand side of (3) are bounded by (3), that is, that $B(X_{\alpha})=2EX_{\alpha}^{2}\|m_{\alpha}^{*}-m_{\alpha}\|_{1}/E|X_{\alpha}^{3}|\leq 1$, completing the proof of (33) and hence of (3).

Lower bound

By (1), with $m=1$ and $\mathcal{L}(X)=G\in\mathcal{F}_{1}$,

Motivated by Theorem 2.3, that two-point distributions achieve the suprema of $B(G)$, for $p\in(0,1)$, let

where $\xi$ is a Bernoulli variable with $P(\xi=1)=p=1-P(\xi=0)$. The distribution function $G_{p}$ of $X$ is given by

and, therefore, the $L^{1}$-distance between $G_{p}$ and the standard normal is given by

As $G_{p}\in\mathcal{F}_{1}$ for all $p\in(0,1)$ and $E|X^{3}|=(p^{2}+q^{2})/\sqrt{pq}$, letting

inequality (39) gives $c_{1}\geq\psi(p)$ for all $p\in(0,1)$ and $\psi(1/2)$ yields (2).

Remarks

This article was submitted on November 18th, 2008. In the article Tyu , submitted on June 8th, 2009, Ilya Tyurin independently proved Theorem 1.1, also by applying the zero bias method. The current article was posted on arXiv on June 28, 2009; article Tyu was posted on December 3rd, 2009. In Tyu , Theorem 1.1 is used to prove the upper bound $0.4785$ on the $L^{\infty}$-Berry–Esseen constant.

Acknowledgment

The author would like to sincerely thank Sergey Utev for helpful suggestions.

References