Stein's method and the Laplace distribution

Background and Introduction

If $W\sim\textrm{Laplace}(0,b)$, then its moments are given by

This distribution was introduced by P.S. Laplace in 1774, four years prior to his proposal of the “second law of errors,” now known as the normal distribution. Though nowhere near as ubiquitous as its younger sibling, the Laplace distribution appears in numerous applications, including image and speech compression, options pricing, and modeling sizes of sand particles, diamonds, and beans. For more properties and applications of the Laplace distribution, the reader is referred to the text .

Our interest in the Laplace distribution was sparked by the fact that if $X_{1},X_{2},...$ is a sequence of random variables (satisfying certain technical assumptions) and $N_{p}\sim\textrm{Geometric}(p)$ is independent of the $X_{i}$’s, then the sum $p^{\frac{1}{2}}\sum_{i=1}^{N_{p}}X_{i}$ converges weakly to the Laplace distribution as $p\rightarrow 0$ . Such geometric sums arise in a variety of settings , and the general setup (distributional convergence of sums of random variables) is exactly the type of problem for which one expects Stein’s method computations to yield useful results. Indeed, Erol Peköz and Adrian Röllin have applied Stein’s method arguments to generalize a theorem due to Rényi concerning the convergence of sums of a random number of positive random variables to the exponential distribution . By an analogous line of reasoning, we are able to carry out a similar program for convergence of random sums of certain mean zero random variables to the Laplace distribution.

We begin in Section 2 by introducing a Stein operator which we show completely characterizes the mean zero Laplace distribution. Specifically, we prove

Let $W\sim\textnormal{Laplace}(0,b)$ and define the operator $\mathcal{A}$ by

Finally, in Section 4 we apply these tools to the study of random sums of mean zero random variables. As a special case, we show

Characterizing the Laplace Distribution

Our first order of business is to establish a characterizing operator for the Laplace distribution. As is typical in Stein’s method constructions, we split the proof of Theorem 1.1 into two parts. We begin with

Applying Fubini’s theorem twice shows that

Setting $h(y)=g(-y)$, it follows from the previous calculation that

Note that since the density of a $\textrm{Laplace}(0,b)$ random variable is given by $f_{W}(w)=\frac{1}{2b}e^{-\frac{\left|w\right|}{b}}$, the density method suggests the following characterizing equation for the Laplace distribution:

and indeed one can verify that if $W\sim\textrm{Laplace}(0,b)$, then

for all absolutely continuous $g$ for which these expectations exist. Thus if $g^{\prime}$ is such a function as well, setting $G(w)=\textrm{sgn}(w)\left(g(w)-g(0)\right)$ gives

so the general form of the equation in Theorem 1.1 can be ascertained by iterating the density method.

Now, in order to establish the second part of Theorem 1.1, we will show that any $X$ satisfying the hypotheses has

where $d_{BL}$ denotes the bounded Lipschitz distance given by

In keeping with the general strategy laid out in the introduction, we consider the initial value problem

For $h\in\mathcal{H}_{BL}$, $W\sim\textnormal{Laplace}(0,b)$, $\widetilde{h}(x)=h(x)-Wh,$ a bounded, twice-differentiable solution to the initial value problem

This solution satisfies $\left\|g_{h}\right\|_{\infty}\leq 2$, $\left\|g_{h}^{\prime}\right\|_{\infty}\leq\frac{2}{b}$, $\left\|g_{h}^{\prime\prime}\right\|_{\infty}\leq\frac{4}{b^{2}}$ and $\left\|g_{h}^{\prime\prime\prime}\right\|_{\infty}\leq\frac{b+2}{b^{3}}.$

The general solution to the homogeneous equation $g^{\prime\prime}(x)-b^{-2}g(x)=0$ is given by $g_{0}(x)=C_{1}e^{\frac{x}{b}}+C_{2}e^{-\frac{x}{b}}$, so, since the associated Wronskian is nonzero, the variation of parameters method suggests that a solution to the inhomogeneous equation $g^{\prime\prime}(x)-b^{-2}g(x)=-b^{-2}\widetilde{h}(x)$ is given by

To see that the initial condition is satisfied, we observe that

Moreover, since $\left\|h\right\|_{\infty}\leq 1$,

With the preceding result in hand, we can finish of the proof of Theorem 1.1 via

for every twice-differentiable function $g$ with $\left\|g\right\|_{\infty},\left\|g^{\prime}\right\|_{\infty},\left\|g^{\prime\prime}\right\|_{\infty}<\infty$, then $X\sim\textnormal{Laplace}(0,b)$.

Let $W\sim\textrm{Laplace}(0,b)$ and, for $h\in\mathcal{H}_{BL}$, let $g_{h}$ be as in Lemma 2.2. Because $g_{h}(0)=0$ and $g_{h},g_{h}^{\prime},g_{h}^{\prime\prime}$ are bounded, it follows from the above assumptions that

Taking the supremum over $h\in\mathcal{H}_{BL}$ shows that $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(W)\right)=0$. ∎

Before moving on, we observe that the reason we are working with the bounded Lipschitz distance is that the bounds on $g_{h}$ and its derivatives depended on both $h$ and $h^{\prime}$ having finite sup norm. As $d_{BL}$ is not especially common (at least explicitly) in the Stein’s method literature, we conclude this section with a proposition relating it to the more familiar Kolmogorov distance

If $Z$ is an absolutely continuous random variable whose density, $f_{Z}$, is uniformly bounded by a constant $C<\infty$, then for any random variable $X$,

We first note that the inequality holds trivially if $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)=0$ as $d_{BL}$ and $d_{K}$ are metrics. Also, since $d_{K}(P,Q)\leq 1$ for all probability measures $P$ and $Q$, $\frac{C+2}{2}\geq 1$, and $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)\geq 1$ implies $\sqrt{d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)}\geq 1$, we have

whenever $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)\geq 1$. Thus it suffices to consider the case where $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)\in(0,1)$.

Since $d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)\in(0,1)$, if we take $\varepsilon=\sqrt{d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)}\in(0,1)$, then $\varepsilon h_{x,\varepsilon}\in\mathcal{H}_{BL}$ and thus

When $C\geq 1$, we can take $\varepsilon=\sqrt{\frac{1}{C}d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)}$ in the above argument to obtain an improved bound of

To the best of the authors’ knowledge, the above proposition is original, though the proof follows the same basic line of reasoning as the well-known bound on the Kolmogorov distance by the Wasserstein distance (see Proposition 1.2 in ). It seems that the primary reason for using the Wasserstein metric, $d_{W}$, is that it enables one to work with smoother test functions while still implying convergence in the more natural Kolmogorov distance. Proposition 2.4 shows that $d_{BL}$ also upper-bounds $d_{K}$ while enjoying all of the resulting smoothness of Wasserstein test functions and with additional boundedness properties to boot. Moreover, the Wasserstein distance is not always well-defined (e.g. if one of the distributions does not have a first moment), whereas $d_{BL}$ always exists. Finally, $d_{BL}$ is a fairly natural measure of distance since it metrizes weak convergence . However, we always have $d_{W}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)\geq d_{BL}\left(\mathscr{L}(X),\mathscr{L}(Z)\right)$, and it is possible for a sequence to converge in $d_{BL}$ but not in $d_{W}$ or $d_{K}$. Furthermore, the bounded Lipschitz metric does not scale as nicely as the Kolmogorov or Wasserstein distances when the associated random variables are multiplied by a positive constant. For the remainder of this paper, we will state our results in terms of $d_{BL}$ with the corresponding Kolmogorov bound being implicit therein, though one should note that, as with the Wasserstein bound on $d_{K}$, Kolmogorov bounds obtained in this fashion are not necessarily optimal, often giving the root of the true rate.

The Centered Equilibrium Transformation

Our next task is to use the characterization in Theorem 1.1 to obtain bounds on the error terms resulting from approximation by the Laplace distribution. To this end, we introduce the following definition.

For any nondegenerate random variable $X$ with mean zero and finite variance, we say that the random variable $X^{L}$ has the centered equilibrium distribution with respect to $X$ if

for all twice-differentiable functions $f$ such that $f$, $f^{\prime}$, and $f^{\prime\prime}$ are bounded. We call the map $X\mapsto X^{L}$ the centered equilibrium transformation.

Suppose that $X$ has mean zero and variance $\sigma^{2}\in(0,\infty)$. Let $X^{z}$ have the zero bias distribution with respect to $X$ and let $B\sim\textnormal{Beta}(2,1)$ be independent of $X^{z}$. Then $X^{L}:=BX^{z}$ satisfies

for all twice-differentiable $f$ with $\left\|f\right\|_{\infty},\left\|f^{\prime}\right\|_{\infty},\left\|f^{\prime\prime}\right\|_{\infty}<\infty$.

Applying the fundamental theorem of calculus, Fubini’s theorem, the definition of $X^{z}$, and the fact that $B$ has density $p(x)=2x1_{}(x)$ gives

The assumptions ensure that all of the functions are integrable. ∎

In an earlier version of this paper, we established the existence of the centered equilibrium distribution by showing that for certain random variables $X$, $X^{L}$ can be obtained by iterating the $X-P$ bias transformation from with $P(x)=\textrm{sgn}(x)$. Though there may be some merit to such a strategy and it provides another example of how results for higher order Stein operators may be obtained by iterating more traditional techniques, in our case it required the rather artificial assumption that the variates in the domain of the transformation have median zero. Those interested in the iterated $X-P$ bias approach are referred to the article by Christian Döbler, which contains the essential technical details of our original argument.

Lemma 2.3 shows that, up to scaling, the mean zero Laplace distribution is the unique fixed point of the centered equilibrium transformation. Thus one expects that if a random variable is close to its centered equilibrium transform, then its distribution is close to the Laplace. Theorem 1.2 formalizes this intuition.

For the example in Section 4, we will also need the following complementary result.

Of course, neither $f$, $f^{\prime}$, nor $f^{\prime\prime}$ is bounded, so we must proceed by approximation. To this end, define

Fatou’s lemma shows that $f^{\prime\prime}(Y^{L})$ is integrable since

so another application of dominated convergence gives

Random Sums

Suppose that $X_{1},X_{2},...$ are i.i.d., symmetric, and nondegenerate random variables with finite variance $\sigma^{2}$, and let $N_{p}\sim\textrm{Geometric}(p)$ be independent of the $X_{i}^{\prime}s$. If

then there exists $\gamma>0$ such that $a_{p}=p^{\frac{1}{2}}\gamma+o(p^{\frac{1}{2}})$ and $X$ has the Laplace distribution with mean and variance $\sigma^{2}\gamma^{2}$.

A recent theorem due to Alexis Toda gives the following Lindeberg-type conditions for the existence of the distributional limit in Theorem 4.1.

Then as $p\rightarrow 0$, the sum $p^{\frac{1}{2}}\sum_{i=1}^{N_{p}}X_{i}$ converges weakly to the Laplace distribution with mean and variance $\sigma^{2}$.

The original statement of Toda’s theorem is slightly more general, allowing for convergence to a possibly asymmetric Laplace distribution.

Now, taking $V=\mu^{-\frac{1}{2}}\sum_{i=1}^{N}X_{i}$, we claim that $V^{L}=\mu^{-\frac{1}{2}}\left(\sum_{i=1}^{M-1}X_{i}+X_{M}^{L}\right)$ has the centered equilibrium distribution with respect to $V$. (Throughout, $X_{m}^{L}$ is taken to be independent of $M$, $N$, and $X_{k}$ for $k\neq m$.) To see that this is so, let $f$ be any function satisfying the assumptions in Definition 3.1. Then, using the notation

letting $\nu_{m}$ denote the distribution of

Finally, since the $X_{i}^{\prime}s$ are independent with mean zero, the Cauchy-Schwarz inequality gives

We conclude our discussion with a proof of Theorem 1.3, which gives sufficient conditions for weak convergence in the setting of Theorem 4.1. Though it requires that the $X_{i}^{\prime}s$ have uniformly bounded third absolute moments, the condition of symmetry is dropped and the identical distribution assumption is reduced to the requirement that the $X_{i}^{\prime}s$ have common variance. This result is not quite as general as Theorem 4.2, but it does provide bounds on the error terms.

In the language of Theorem 4.3, we have $\mu=\frac{1}{p}$, $\sigma=b\sqrt{2\mu}$, and

Acknowledgements

The authors would like to thank Larry Goldstein for suggesting the use of Stein’s method to get convergence rates in the general setting of Theorem 4.2 and for many helpful comments throughout the preparation of this paper. Thanks also to Alex Rozinov for several illuminating conversations concerning the material in Section 4 and to the anonymous referees whose careful notes were of great help to us.

References