Tracy-Widom law for the extreme eigenvalues of sample correlation matrices

Introduction

Suppose we have a $p$-dimensional distribution with mean $\mu$ and covariance matrix $\Sigma$. In recent three or four decades, in many research areas, including signal processing, network security, image processing, genetics, stock marketing and other economic problems, people are interested in the case where $p$ is quite large or proportional to the sample size. Naturally, one may ask how to test the independence among the $p$ components of the population. From the principal component analysis point of view, the independence test statistic is usually the maximum eigenvalue of the sample covariance matrices. Under the additional normality assumption, Johnstone derived the asymptotic distribution of the largest eigenvalue of the sample covariance matrices to study the test $H_{0}:\Sigma=I$ assuming $\mu=0$.

However, sample covariance matrices are not scale-invariant. So if $\mu=0$, Johnstone proposes to perform principal component analysis (PCA) by the maximum eigenvalue of the matrix $W=YY^{T}$, where

Here $\mathbf{x}_{i}=(x_{i1},\cdots,x_{in})^{T}$ contains $n$ observations for the $i$-th component of the population, $i=1,\cdots,p$, and $||\cdot||$ represents the vector norm.

Performing PCA on $W$ amounts to PCA on the sample correlations of the original data if $\mu=0$. So for simplicity, we call $W$ the sample correlation matrix in this paper. From now on, the eigenvalues of $W$ will be denoted by

Then the empirical distribution (ESD) of $W$ is defined by

The asymptotic property of $F_{p}$ was studied in and . For the almost sure convergence of $\lambda_{1}$ and $\lambda_{p}$, see .

In this paper, we will study the fluctuations of the extreme eigenvalues $\lambda_{1},\lambda_{p}$ of $W$ for a general population, including multivariate normal one. The basic assumption on the distribution of our population throughout the paper is $\bf{Condition~{}C_{1}}$. We assume $x_{ij}$ are independent symmetric distributed random variables with variance $1$. And for any $i$, we assume $x_{i1},\cdots,x_{in}$ to be i.i.d. Furthermore, we request the distributions of the $x_{ij}^{\prime}$s have sub-exponential tails, i.e., there exist positive constants $C,C^{\prime}$ such that for all $1\leq i\leq p,1\leq j\leq n$ one has

for all $t\geq C^{\prime}$. And we also assume $p/n\rightarrow y$ as $p,n=n(p)\rightarrow\infty$, where $0<y<1$.

We use $C,C_{0},C_{1},C_{2},C^{\prime},O(1)$ to denote some positive constants independent of $p$, which may differ from line to line. And we use $C_{\alpha}$ to denote some positive constants depending on the parameter $\alpha$. The notation $||\cdot||_{op},||\cdot||_{F}$ represent the operator norm and Frobenius norm of a matrix respectively. And $||\cdot||$ represents a Euclidean norm of a vector.

The sample correlation matrix $W$ is invariant under the scaling on the elements $x_{ij}$, so the assumption $Var(x_{ij})=1$ is not necessary indeed. We specify it to be $1$ here just for convenience. Owing to the exponential tails, we can always truncate the variables so that $|x_{ij}|\leq K$ with some $K\geq\log^{O(1)}n$.

A special sample correlation matrix model is the Bernoulli case, i.e. $x_{ij}$ takes value of $1$ or $-1$ with equal probability. Notice that if $x_{ij}$ are Bernoulli, we always have for all $1\leq i\leq p$

As a consequence, the sample correlation matrix with Bernoulli elements coincides with its corresponding sample covariance matrix for which the limiting distribution of the extreme eigenvalues are well known under some moment assumptions. One can refer to ,, , and . We only summarize their results for the special Bernoulli case as the following theorem.

(Bernoulli case) For the matrix $W$ in (1.4), if $x_{ij}$ are $\pm 1$ Bernoulli variables, we have

as $p,n\rightarrow\infty$ with $p/n\rightarrow y\in(0,1)$.

Here $TW_{1}$ is the famous Tracy-Widom distribution of type $1$, which was firstly raised by Tracy and Widom in for the Gaussian orthogonal ensemble. The distribution function $F_{1}(t)$ of $TW_{1}$ admits the representation

where $q$ statisfies the Painlev$\acute{e}$ $II$ equation

The main purpose of this paper is to generalize Theorem 1.1 to the population satisfying the basic condition $\mathbf{C}_{1}$. Our main results are the following two theorems.

Let $W$ be a sample correlation matrix satisfying the basic condition $\mathbf{C}_{1}$. We have

For technical reasons, it is convenient to work with the continuous random variables $x_{ij}$. As a result, the events such as eigenvalue collision will only occur with probability zero (see Lemma 3.5). Because none of our bounds depends on how continuous the $x_{ij}$ are, one can recover the discrete case from the continuous one by a standard limiting argument by using Weyl’s inequality (see Lemma 2.2), especially for the Bernoulli case.

If the population is normal, then we can derive the Tracy-Widom law for both the largest and smallest eigenvalues of the matrix $\mathcal{R}=RR^{T}$, where

Here $\bar{x}_{i}=n^{-1}\sum_{j=1}^{n}x_{ij}$ and $\mathbf{x}_{i}-\bar{x}_{i}$ means each element $x_{ij}$ of $\mathbf{x}_{i}$ will be subtracted by $\bar{x}_{i}$, $i=1,\cdots,p$. We denote the ordered eigenvalues of $\mathcal{R}$ by $0\leq\lambda_{1}(\mathcal{R})\leq\cdots\leq\lambda_{p}(\mathcal{R})$ below. Actually $\mathcal{R}$ is the sample correlation matrix when the population mean is unknown.

For the sample correlation matrix $\mathcal{R}$ with i.i.d $N(0,1)$ elements, if $p/n\rightarrow y\in(0,1)$, we have

The main strategy is to prove a so-called “Green function comparison theorem”, which was raised by Erdös, Yau and Yin in for generalized Wigner matrices. We will provide a “Green function comparison theorem” to the sample correlation matrices obeying the assumption $\mathbf{C}_{1}$ in Section 4, see Theorem 4.3. Then by the comparison theorem, we can compare the general distributed case with the Bernoulli case to get Theorem 1.2. And as an application, we can also get Theorem 1.3.

Our article is organized as follows. In Section 2, we state some basic tools, which can be also found in the series work , , and . And we provide some main technical lemmas and theorems in Section 3. The most important one is the so-called delocalization property of singular vectors, which will be shown as an obstacle to establish the Green function comparison theorem in the sample correlation matrices case. And in Section 4, we provide a Green function comparison theorem to prove the edge universality for sample correlation matrices satisfying the assumption $\mathbf{C}_{1}$. In Section 5, we state the proofs for our main results: Theorem 1.2 and Theorem 1.3.

Basic Tools

In this section, we state some basic tools from linear algebra and probability theory. Firstly, we denote the ordered singular values of $Y$ by

then we have $\sigma_{i}=\lambda_{i}^{1/2}$. If we further denote the unit right singular vector of $Y$ corresponding $\sigma_{i}$ by $u_{i}$ and the left one by $v_{i}$, we have

Below we shall state some tools for eigenvalues, singular values and singular vectors without proof.

(Cauchy’s interlacing law). Let $1\leq p\leq n$

(i) If $A_{n}$ is an $n\times n$ Hermitian matrix, and $A_{n-1}$ is an $n-1\times n-1$ minor, then $\lambda_{i}(A_{n})\leq\lambda_{i}(A_{n-1})\leq\lambda_{i+1}(A_{n})$ for all $1\leq i<n$.

(ii) If $A_{n,p}$ is a $p\times n$ matrix, and $A_{n,p-1}$ is a $p-1\times n$ minor, then $\sigma_{i}(A_{n,p})\leq\sigma_{i}(A_{n,p-1})\leq\sigma_{i+1}(A_{n,p})$ for all $1\leq i<p$.

(iii) If $p<n$, $A_{n,p}$ is a $p\times n$ matrix, and $A_{n-1,p}$ is a $p\times n-1$ minor, then $\sigma_{i-1}(A_{n,p})\leq\sigma_{i}(A_{n-1,p})\leq\sigma_{i}(A_{n,p})$ for all $1\leq i\leq p$, with the understanding that $\sigma_{0}(A_{n,p})=0$. (For $p=n$, one can consider its transpose and use (ii) instead.)

$\bullet$ If $M,N$ are $n\times n$ Hermitian matrices, then $||\lambda_{i}(M)-\lambda_{i}(N)||\leq||M-N||_{op}$ for all $1\leq i\leq n$.

$\bullet$ If $M,N$ are $p\times n$ matrices, then $||\sigma_{i}(M)-\sigma_{i}(N)||\leq||M-N||_{op}$ for all $1\leq i\leq p$.

The following lemma is on the components of a singular vector, which can be found in .

Further, we need a frequently used tool in the Random Matrix Theory: the Stieltjes transform of ESD $F_{p}(x)$, which is defined by

Here we denote $G_{jk}$ as the $(j,k)$ entry of $G(z)$. As is well known, the convergence of a tight probability measure sequence is equivalent to the convergence of its Stieltjes transform sequence towards the corresponding transform of the limiting measure. So corresponding to the convergence of $F_{p}(x)$ towards $F_{MP,y}(x)$, the famous Marc̆enko-Pastur law $F_{MP,y}(x)$ whose density function is given by

where $a=(1-\sqrt{y})^{2},b=(1+\sqrt{y})^{2}$, $s_{p}(z)$ almost surely converges to the Stieltjes transform $s(z)$ of $F_{MP,y}(x)$. Here

where the square root is defined as the analytic extension of the positive square root of the positive numbers. Moreover, $s(z)$ satisfies the equation

If we denote the $k$-th row of $Y$ by $\mathbf{y}_{k}^{T}$ and the remaining $(p-1)\times n$ matrix after deleting $\mathbf{y}_{k}^{T}$ by $Y^{(k)}$, one has

The formula of $G_{kk}$ is analogous. By (2.3), we have the following lemma on the decomposition of $s_{p}(z)$:

The last main tool we need comes from the probability theory, which is a concentration inequality for projections of random vectors. The details of the proof can also be found in .

Main Technical Results

In this section, we provide our main technical results: the local MP law for sample correlation matrices, and the delocalization property for the singular vectors. Both results will be proved under much weaker assumption than $\mathbf{C}_{1}$. We form them into the following two theorems.

(Local MP law). Assume that $p/n\rightarrow y$ with $0<y<1$. And $\{x_{ij}:1\leq i\leq p,1\leq j\leq n\}$ is a collection of independent (but not necessary identically distributed) random variables with mean zero and variance 1. If $|x_{ij}|\leq K$ almost surely for some $K=o(p^{1/C_{0}}\delta^{2}log^{-1}p)$ with some $0<\delta<1/2$ and some large constant $C_{0}$ for all $i,j$, one has with overwhelming probability that the number of eigenvalues $N_{I}$ for any interval $I\subset[a/2,2b]$ with $|I|\geq\frac{K^{2}\log^{7}p}{\delta^{9}p}$ obeys

The topic of the limiting spectrum distribution on short scales was firstly raised by Erdős, Schlein and Yau in for Wigner matrices. Such type of results are shown to be quite necessary for the proof of the famous universality conjectures in the Random Matrix Theory, for example, see and .

A strong type of the local MP law has been established for more general matrix models in a very recent paper of Pillai and Yin, see Theorem 1.5, . In fact, from Theorem 1.5 of , one can get a more precise bound than that in (3.1) if we replace $\rho_{MP,y}(x)$ by the nonasymptotic MP law $\rho_{W}(x)$ defined in Section 4. Moreover, Pillai and Yin’s strong local MP law also provides some crucial estimates on individual elements of the Green function $G$, which will be used to establish our Green function comparison theorem in Section 4.

Note that a little weaker delocalization property for the left singular vector $v_{i}$ can also be found in Theorem 1.2 $(iv)$ of Pillai and Yin .

holds with overwhelming probability. So below we always assume $\lambda_{i}\in(a/2,2b),1\leq i\leq p$.

The proof of Theorem 3.1 is partially based on the lemmas of Section 2. It turns out to be quite similar to the case of sample covariance matrices and Wigner matrices, see , , and . However, the delocalization of the right singular vector $u_{i}$ of $Y$ is an obstacle, owing to the lack of independence between the columns of $Y$.

For the convenience of the reader, we provide a short proof of Theorem 3.1 at first. Our main task in this section is the proof of Theorem 3.2, more precisely, the right singular vector part of the theorem.

We provide the following crude upper bound on $N_{I}$ at first.

Let $\lambda_{\alpha}^{(1)},\alpha=1,\cdots,p-1$ denote the eigenvalues of the $(p-1)\times(p-1)$ matrix $W^{(1)}$. Thus $\lambda_{\alpha}^{(1)},\alpha=1,\cdots,p-1$ are also the eigenvalues of the $n\times n$ matrix $\mathcal{W}^{(1)}$, whose other eigenvalues are all zeros. We further use $\nu_{\alpha}$ to denote the eigenvector of $\mathcal{W}^{(1)}$ corresponding to the eigenvalue $\lambda_{\alpha}^{(1)}$, and introduce the quantity

By Cauchy’s interlacing law, we also have $\lambda_{\alpha}^{(1)}\in[a/2,2b]$ with overwhelming probability. Then for any $z=E+i\eta$ such that $E\in[a/2,2b]$, we have

for any $k\in\{1,\cdots,p\}$. Now we set $I=[E-\eta/2,E+\eta/2]$. Notice that there always exists some positive constant $C_{2}$ such that

If we set $C_{3}=C_{1}C_{2}$, it follows from (3.7) and (3.8) that

The first term of (3.9) is obviously exponential small by the Hoeffding inequality. For the second term, we use Lemma 2.5. Now we specialize $\mathcal{X}$ in Lemma 2.5 to be $\mathbf{x}_{1}$ and the subspace $H$ to be the one generated by eigenvectors $\{\nu_{\alpha}:\lambda_{\alpha}^{(1)}\in I\}$. Thus one has

with overwhelming probability. This implies that the second term of (3.9) is exponential small when $C$ is large enough. So we conclude the proof of Lemma 3.3. ∎

Now we proceed to prove Theorem 3.1. The basic strategy is to compare $s_{p}(z)$ and $s(z)$ with small imaginary part $\eta$. In fact, we have the following proposition.

Let $1/10\geq\eta\geq\frac{1}{n}$, and $L_{1},L_{2},\epsilon,\delta>0$. Suppose that one has the bound

with (uniformly) overwhelming probability for all $z=E+i\eta$ such that $E\in[L_{1},L_{2}]$ and ${\Im z}\geq\eta$. Then for any interval $I\subset[L_{1}+\epsilon,L_{2}-\epsilon]$ with $|I|\geq\max(2\eta,\frac{\eta}{\delta}\log\frac{1}{\delta})$, one has

Proposition 3.1 is an extension of Lemma 29 of up to the edge, whose proof can be found in . In fact, the proof can be taken in the same manner as that of Lemma 64 in for the Wigner matrix.

So in view of Proposition 3.1, to prove Theorem 3.1, we only need to prove that the bound

holds with (uniformly) overwhelming probability for all $z=E+i\eta$ such that $E\in[a/2-\epsilon,2b+\epsilon]$ and $1/10\geq\eta\geq\frac{K^{2}\log^{6}n}{n\delta^{8}}$. To prove (3.10) we need to derive a consistent equation for $s_{p}(z)$, which is similar to the equation (2.6) for $s(z)$.

Firstly by Lemma 2.4 we can rewrite $s_{p}(z)$ as

Then the proof of (3.10) can be taken in the same manner as the counterpart of the sample covariance matrix case (see the proof of formula (4.12) of ). We only state the different parts below and leave the details to the reader. We remark here that we consider the domain $[L_{1},L_{2}]=[a/2-\epsilon,2b+\epsilon]$ rather than $[a,b]$ in . However, if one goes through the proof in , it is not difficult to see that the proof towards any domain $[L_{1},L_{2}]$ containing $[a,b]$ is the same. The only minor difference between our case and the sample covariance matrix in is the estimation of $d_{k}$. We will only deal with $d_{1}$ in the sequel. The others are analogous. By (3.5) and (3.6), we have

is the Stieltjes transform of the ESD of $W^{(1)}$. Then by the Cauchy’s interlacing property, we have

Now we provide the following lemma on the second term of (3.11).

For all $z=E+i\eta$ with $E\in[a/2-\epsilon,2b+\epsilon]$ and $\eta\geq\frac{K^{2}\log^{6}n}{n\delta^{8}}$,

uniformly in $z$ with overwhelming probability.

We set $R_{j}=(\xi_{j}-1)$. By (3.5) and the fact that

holds with overwhelming probability, we have for any $T\subset\{1,\cdots,p-1\}$

where $a\vee b=\max(a,b)$. By inserting (3.13) and (3.15) into (3.14), we have

If we choose $T=\log^{O(1)}n$, we always have

Then the following part of the proof is the same as that in the sample covariance matrix case. One can refer to the proof of Proposition 4.6 of for details. ∎

Now we proceed to the proof of Theorem 3.1. By (3.11), (3.12) and Lemma 3.4 we can get the following equation

By a standard comparison of (3.16) and (2.6) (see for example), we have (3.10). Thus by Proposition 3.1 we conclude the proof of Theorem 3.1. ∎

Now we turn to the proof of Theorem 3.2. At first, we introduce the matrix $\widehat{W}_{(n)}:=\widehat{Y}_{(n)}\widehat{Y}_{(n)}^{T}$ with

We will need the following lemma on eigenvalue collision.

If we assume the random variables $x_{ij}^{\prime}s$ are continuous, we have the following events hold with probability one. $i)$: $W$ has simple eigenvalues, i.e. $\lambda_{1}<\lambda_{2}<\cdots<\lambda_{p}$. $ii)$: $W$ and $W^{(p)}$ have no eigenvalue in common. $iii)$: $W$ and $\widehat{W}_{(n)}$ have no eigenvalue in common.

The proof of Lemma 3.5 will be postponed to Appendix A.

The proof for the left singular vectors is nearly the same as the sample covariance matrix case shown in by using Lemma 2.3, $ii)$ of Lemma 3.5 and Theorem 3.1. Moreover, as we have mentioned in the Remark 3.3, a slightly weaker delocalization property for the left singular vectors has been provided in . So we will only present the proof for the right singular vectors below.

Below we denote the $k$-th column of $Y$ by $h_{k}$, and the remaining $p\times(n-1)$ matrix after deleting $h_{k}$ by $Y_{(k)}$. Note that $Y_{(n)}$ is not independent of the last column $h_{n}$. However, for the sample covariance matrix case, the independence between the column and the corresponding submatrix is essential for one to use the concentration results such as Lemma 2.5. To overcome the inconvenience caused by the dependence, we will use the modified matrix $\widehat{Y}_{(n)}$ defined above. Notice that the matrix $\widehat{Y}_{(n)}$ is independent of the random vector $(x_{1n},x_{2n}\cdots,x_{pn})^{T}$.

The following lemma handles the operator norms of $\Delta_{1}$ and $\Delta_{2}$.

Under the assumption of Theorem 3.2, we have

We only discuss the second term since the first one is analogous. It is easy to see the entries of $\Delta_{2}^{T}$ satisfy

where $\Delta_{3}$ is a $p\times p$ diagonal matrix with $(i,i)$-th entry to be

with overwhelming probability. Together with the fact that $||\widehat{Y}_{(n)}||_{op}\leq C$ holds with overwhelming probability, we can conclude the proof of Lemma 3.6. ∎

Now we proceed to the proof of Theorem 3.2. If we denote

where $x$ is the last component of $u_{i}$. Without loss of generality, we can only prove the theorem for $x$. Notice that $u_{i}$ is the eigenvector of $\mathcal{W}=Y^{T}Y$ corresponding to the eigenvalue $\lambda_{i}$. From

Note that $\widehat{Y}_{(n)}^{T}\widehat{Y}_{(n)}$ share the same nonzero eigenvalues with $\widehat{W}_{(n)}$, so by $iii)$ of Lemma 3.5, we can always view that the matrix $\widehat{Y}^{T}_{(n)}\widehat{Y}_{(n)}-\lambda_{i}$ is invertible. Consequently,

If $x=0$ then Theorem 3.2 is evidently true. Consider $x\neq 0$ below. Together with the fact that $x^{2}=1-||\mathbf{w}||^{2}$, we have

Now if we use $\hat{\lambda}_{j}$ to denote the ordered nonzero eigenvalue of $\widehat{Y}_{(n)}^{T}\widehat{Y}_{(n)}$ and $\hat{u}_{j}$ the corresponding unit eigenvector. And set the projection

Then by the spectral decomposition one has

Therefore to show $|x|\leq n^{-1/2}K^{C_{0}/2}\log^{O(1)}n$, we only need to prove

To prove (3.24), we need to separate the issue into the bulk case and the edge case. Before that, we shall provide the following lemma which will be used in both cases.

If we denote the unit eigenvector of $\widehat{W}_{(n)}$ corresponding to $\hat{\lambda}_{j}$ by $\hat{v}_{j}$, under the assumption of Theorem 3.2 we have for any $J\subseteq\{1,\cdots,p\}$ with $|J|=d\leq nK^{-3}$,

We will postpone the proof of Lemma 3.7 to Appendix B. In fact, it can be viewed as a modification of Lemma 2.5.

Now we decompose the proof of Theorem 3.2 into two parts: bulk case and edge case. $\bullet$ Bulk case: $\lambda_{i}\in[a+\epsilon,b-\epsilon]$ for some $\epsilon>0$

Note that the local MP law (Theorem 3.1) can also be applied to the matrix $\widehat{Y}^{T}_{(n)}\widehat{Y}_{(n)}$. Thus we can find a set $J\subseteq\{1,\cdots,p\}$ with $|J|\geq K^{2}\log^{20}n$ such that $\hat{\lambda}_{j}=\lambda_{i}+O(K^{2}\log^{20}n/n)$ for any $j\in J$ when $\lambda_{i}$ is in the bulk region of the MP law. It follows that

By the singular value decomposition, we have

for any $J\subset\{1,\cdots,p\}$ such that $K^{2}\log^{20}n\leq|J|\leq nK^{-3}$.

If $|x|\leq n^{-1/2}K^{C_{0}/2}\log^{O(1)}n$, then we get the conclusion for the bulk case. So we assume $|x|\geq n^{-1/2}K^{C_{0}/2}\log^{O(1)}n$ below to get (3.24). By Lemma 3.6, if we choose $C_{0}\geq 20$ (say), we have

with overwhelming probability. On the other side, Lemma 3.7 implies

with overwhelming probability. So one has

where $\gg$ means “much larger than”, i.e.

Notice that for any real number sequence $\{S_{1},\cdots,S_{m}\}$ and $\{T_{1},\cdots,T_{m}\}$ with $\sum_{i=1}^{m}S_{i}^{2}\gg\sum_{i=1}^{m}T_{i}^{2}$, there exists some $c$ near 1 such that $\sum_{i=1}^{m}(S_{i}+T_{i})^{2}\geq c\sum_{i=1}^{m}S_{i}^{2}$. Therefore by (3.30),(3.31) and (3.25) we can obtain

which implies (3.24) directly. So we conclude the proof for the bulk case.

Next, we turn to the edge case. $\bullet$ Edge case: $a-o(1)\leq\lambda_{i}\leq a+\epsilon$ or $b-\epsilon\leq\lambda_{i}\leq b+o(1)$ with some $\epsilon>0$.

For the edge case we also begin with the representation (3.23). By (3.22), we have

Inserting (3.32) and (3.18) into (3.21) we find

Similarly to the bulk case, we only need to get (3.24). Below we also assume $|x|\geq Cn^{-1/2}K^{C_{0}/2}\log^{O(1)}n$ to get (3.24). Similar to (3.29), by using Lemma 3.6 we have

Moreover, by Lemma 3.7 and (3.26), we also have

holds with overwhelming probability. Thus to provide (3.24), it suffices to show

instead. By the Cauchy-Schwarz inequality, we only need to prove

with overwhelming probability for some $1\leq T_{-}<T_{+}\leq K^{2}\log^{O(1)}n$.

Notice that under the assumption $|x|\geq Cn^{-1/2}K^{C_{0}/2}\log^{O(1)}n$, by Lemma 3.6 we have

Moreover, it is not difficult to see $h_{n}^{T}h_{n}=y+o(1)$ with overwhelming probability. Thus by (3.33), we have with overwhelming probability

So to prove (3.36) we only need to evaluate

By Theorem 3.1, the interval $I$ with $|d_{I}|<\log n$ contains at most $K^{2}\log^{O(1)}n$ eigenvalues. So we can set $T_{-},T_{+}$ accordingly so that such intervals don’t contain any $\hat{\lambda}_{j}$ if $j<i-T_{-}$ or $j>i+T_{+}$. In the following we only consider $I$ such that $|d_{I}|\geq\log n$ in the estimation of (3.38). Note that for $\hat{\lambda}_{j}\in I$,

when $|x|\geq n^{-1/2}K^{C_{0}/2}\log^{O(1)}n$. Thus we can find

Here we used Lemma 3.3 in the last inequality. Now we partition the real line into intervals $I$ of length $K^{2}\log^{A}n/n$, and sum (3.39) over all intervals $I$ with $|d_{I}|\geq\log n$. Then

instead of (3.38). The evaluation of (3.40) is really the same as the counterpart in the sample covariance matrix case (see (4.5) in ) by inserting Lemma 3.7, so we omit the details here. In fact, we can finally get

Using the formula for the Stieltjes transform $s(z)$, one can get from residue calculus that for $\lambda_{i}\in[a,b]$,

Consequently by the definition of $a$ and $b$, if $|\lambda_{i}-a|\leq o(1)$, we have

And if $|\lambda_{i}-b|\leq o(1)$, we have

Then it is easy to see when $0<y<1$, (3.36) holds with overwhelming probability for the case where $|\lambda_{i}-a|=o(1)$ or $|\lambda_{i}-b|=o(1)$. Moreover by continuity we can adjust the value of $\epsilon$ to get the conclusion for the general case $a-o(1)\leq\lambda_{i}\leq a+\epsilon$ or $b-\epsilon\leq\lambda_{i}\leq b+o(1)$. Thus we complete the proof of the delocalization for $u_{i}$. ∎

Green function comparison theorem

In this section, we provide a Green function comparison theorem for the sample correlation matrices satisfying $\mathbf{C}_{1}$. The proof heavily relies on the recent results of Pillai and Yin on sample covariance matrices and the delocalization property for the right singular vectors proved in the last section. At first, we will borrow some results from directly with only minor notation change. In fact, by Theorem 1.5 in , it is not difficult to see Theorem 1.2 and Theorem 1.3 of also hold for sample correlation matrices under our basic condition $\mathbf{C_{1}}$.

To state the results in , we need to introduce some notation. Define the parameter

Moreover we introduce the “nonasymptotic Marchenko-Pastur law ”

and the corresponding distribution function $F_{W}(x)$ and Stieltjes transform

And we say that an event $\Omega$ holds with $\zeta$-high probability if there exists a constant $C>0$ such that

for large enough $p$. Note that (4.2) implies that the event $\Omega$ holds with overwhelming probability if $\zeta>0$. We further denote

(Theorem 1.5, ) Under the condition $\mathbf{C_{1}}$, for any $\zeta>0$ there exists a constant $C_{\zeta}$ such that the following events hold with $\zeta$-high probability.

(i) The Stieltjes transform of the ESD of $W$ satisfies

(ii) The individual matrix elements of the Green function satisfy

We also need the following lemma on $s_{W}(z)$.

(Lemma 26, ) Set $\kappa:=\min(|\lambda_{+}-E|,|E-\lambda_{-}|)$. For $z=E+i\eta\in\underline{S}(0)$, (see (4.1)) we have the following relations:

where $A\sim B$ means $C^{-1}B\leq A\leq CB$ for some constant $C$. Furthermore

Now we set $Y^{\mathbf{v}}=(\mathbf{y}_{ij}^{\mathbf{v}}):=(x_{ij}^{\mathbf{v}}/||\mathbf{x}_{i}^{\mathbf{v}}||)_{p,n}$, with elements $x_{ij}^{\mathbf{v}}$ satisfying our basic condition $\mathbf{C_{1}}$. Correspondingly we let $W^{\mathbf{v}}=Y^{\mathbf{v}}Y^{\mathbf{v}T}$, $G^{\mathbf{v}}(z)=(W^{\mathbf{v}}-z)^{-1}$ and $s_{p}^{\mathbf{v}}(z)=\frac{1}{p}TrG^{\mathbf{v}}(z)$. Define the matrix $W^{\mathbf{w}}$, the Green function $G^{\mathbf{w}}(z)$ and the Stieltjes transform $s_{p}^{\mathbf{w}}(z)$ analogously for another random sequence $\{x_{ij}^{\mathbf{w}}\}$ satisfying $\mathbf{C}_{1}$ which is independent of $\{x_{ij}^{\mathbf{v}}\}$. The aim in this section is to prove the following Green function comparison theorem.

Below we only state the results and proofs for the largest eigenvalue. The smallest one is just analogous.

with some constant $C_{1}>0$. Then there exists $\epsilon_{0}>0$ depending only on $C_{1}$ such that for any $\epsilon<\epsilon_{0}$ and for any real numbers $E,E_{1}$ and $E_{2}$ satisfying

for some constant $C$ and large enough $p$.

The proof is similar to that of Theorem 6.3 of . Moreover, the proof of (4.9) can be taken in a same manner as that of (4.8), so we will just present the proof for (4.8) below. The basic strategy is to estimate the successive difference of matrices which differ by a row. For $1\leq\gamma\leq p$, we denote by $Y_{\gamma}$ the random matrix whose $j$-th row is the same as that of $Y^{\mathbf{v}}$ if $j\leq\gamma$ and that of $Y^{\mathbf{w}}$ otherwise; in particular $Y_{0}=Y^{\mathbf{v}}$ and $Y_{p}=Y^{\mathbf{w}}$. And we set

We shall compare $W_{\gamma-1}$ with $W_{\gamma}$ by using the following lemma. For simplicity, we denote

For any sample correlation matrix $W$ with elements satisfying the basic assumption $\mathbf{C_{1}}$, if $|E-\lambda_{+}|\leq p^{-2/3+\epsilon}$ and $p^{-2/3}\gg\eta\gg p^{-2/3-\epsilon}$ for some $\epsilon>0$, then we have

where the functional $A(Y^{(i)},m_{1},m_{2})$ only depends on the distribution of $Y^{(i)}$ and the first two moments $m_{1},m_{2}$ of $x_{ij}$.

We always assume $m_{1}=0$, $m_{2}=1$ in our case.

Then the proof of Theorem 4.3 can be completed by the telescoping argument.

Therefore it suffices to prove Lemma 4.4 in the sequel. To do this, we need to provide some bounds about $\mathcal{G}^{(i)}$. We only state the result for $i=1$ as the following lemma since the others are analogous.

Under the assumptions in Lemma 4.4, we have for $\epsilon>0$ small enough,

The proof of Lemma 4.5 will be postponed to the end of this section. Now we begin to prove Lemma 4.4 assuming Lemma 4.5.

The proof is in a similar manner to that of Lemma 6.5 in . At first we rewrite (2.8) as

Moreover, by Schur’s complement, we also have

By $(ii)$ of Lemma 4.1 and (4.7) we can get

with overwhelming probability. Thus we have the expansion

Since $z$ and $s_{W}(z)$ are $O(1)$ by (4.3), by definitions and Lemma 4.5, we have

with overwhelming probability. Thus we have

Similarly to the counterpart proof of Lemma 6.5 in , we only need to show

with some functional $A_{k}$ only depending on the distribution of $Y^{(1)}$, $m_{1}$ and $m_{2}$.

with overwhelming probability. If we write $r_{1}=\Re(\eta zs_{W}(z)),r_{2}=\Im(\eta zs_{W}(z))$, then we have

Notice that if there exists a $k_{i}$ which appears only once in the above product, then by the assumption that $x_{ij}$ is symmetric, we have

So we consider the case where $k_{i}$ appears exactly twice. Firstly, we consider

where the first summation goes through the indices $k_{1},k_{2},k_{3}$ such that they are not equal to each other, and the second summation goes through the left part of the indices. Then it is not difficult to see the number of the terms in the second summation is of the order $O(n^{2})$. By the exponential tail assumption and the Hoeffding inequality, we can see

Furthermore, since $x_{11},\cdots,x_{1n}$ are i.i.d., we have for $k_{1},k_{2},k_{3}$ not equal to each other

Therefore by (4.19), (4.20), (4.21) and the fact that $\mathcal{G}^{(1)}$ only depends on $Y^{(1)}$, we have

By inserting (4.23) into (4.18), we can get (4.17) for $k=3$. The cases of $k=1$ and $k=2$ can be proved similarly by inserting Lemma 4.5. So we conclude the proof. ∎

The proof of (4.10) is the same as the counterpart in , (see (6.36) of ). So we only state the proof of (4.11) below. For the ease of the presentation, we prove (4.11) for $\mathcal{G}=(\mathcal{W}-z)^{-1}:=(Y^{T}Y-z)^{-1}$ instead of $\mathcal{G}^{(1)}$. By the spectral decomposition, we have

where the projection $P=I-\sum_{k=1}^{p}u_{k}u_{k}^{T}$. Consequently, we have

Note that $|P_{ij}|\leq 1,|z|\geq\lambda_{+}/2$. By the delocalization property of $u_{k}$ in Theorem 3.2 one has

with overwhelming probability. For $\alpha=2$, by using $i)$ of Lemma 4.1 and (4.7) we have

with overwhelming probability. Here we used $(iii)$ of Lemma 4.1 in the last inequality. Consequently, we have

It remains to estimate $\int\frac{1}{|x-z|}dF_{W}(x)$. For $E<\lambda_{+}$ such that $\lambda_{+}-E\leq p^{-2/3+\epsilon}$

When $E\geq\lambda_{+}$, we still have (4.24). Therefore, we have

with overwhelming probability. Thus we complete the proof. ∎

Proofs of main theorems

In this section, we provide the proofs of Theorem 1.2 and Theorem 1.3.

The proof of Theorem 1.2 is totally based on Theorem 1.5 of and our Theorem 4.3. Let $W^{\mathbf{v}}$ and $W^{\mathbf{w}}$ be two independent sample correlation matrix satisfying $\mathbf{C}_{1}$. We claim that there is an $\varepsilon>0$ and $\delta>0$ such that for any real number $s$ (which may depend on $p$) one has

for $p\geq p_{0}$ sufficiently large, where $p_{0}$ is independent of $s$. The proof of (5.1) is independent of the matrix model and totally based on Theorem 1.5 of and our Theorem 4.3, we refer to the proof of Theorem 1.7 of for details.

Now if we choose $W^{\mathbf{v}}$ to be the Bernoulli case, it is not difficult to get Theorem 1.2 by combining (5.1) and Theorem 1.1. ∎

It is easy to see $A$ is an orthogonal matrix. Moreover, it is elementary that

where $z_{i1},\cdots,z_{in-1}$ is a sequence of i.i.d $N(0,1)$ variables. Further, if we denote the vector $\mathbf{z}_{i}=(z_{i1},\cdots,z_{in-1})^{T}$, we also have

Consequently, in the Gaussian case, $\mathcal{R}$ is also a $W$-type sample correlation matrix defined in (1.4) with parameters $p,n-1$. Thus by Theorem 1.2, we have

as $p\rightarrow\infty$. Replacing $n-1$ by $n$ in (5.4) and (5.5), we can complete the proof of Theorem 1.3. ∎

Appendix A

At first we prove $i)$. Note that $W=DSD$. For $W$ and $SD^{2}$ share the same eigenvalues, it is equivalent to prove that the eigenvalues of $SD^{2}$ are simple. We further introduce the polynomial $P_{1}(X)$ of $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$ as

It is easy to see $P_{1}(X)$ vanishes with zero Lebesgue measure, so we can always assume $P_{1}(X)\neq 0$. As a consequence, we can reduce our problem to prove the matrix

has no multiple eigenvalue. Now we denote the discriminant of the characteristic polynomial of $Q$ by $P_{Q}(X)$. Observe that all the entries of $Q$ are polynomials of $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$, so $P_{Q}(X)$ is also a polynomial of $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$. For the set of zeros of any non null polynomial in real variables only has zero Lebesgue measure, it suffices to prove that $P_{Q}(X)$ is not a null polynomial. In other words, it suffices to find a family $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$ such that $P_{Q}(X)\neq 0$. It is equivalent to show that $W$ has no multiple eigenvalue for one sample of the collection $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$ such that $P_{1}(X)\neq 0$.

with $1\leq i\leq p,1\leq j\leq n$. Then it is not difficult to see

which is a Jacobi matrix with positive subdiagonal entries. Such a Jacobi matrix has simple eigenvalues, for example, see Proposition 2.40 of .

Next we turn to the proof of $ii)$. We use $X^{(p)}$ to denote the submatrix of $X$ with $p$-th row deleted, and use $D^{(p)}$ to denote the $p-1\times p-1$ upper left corner of $D$. And we set $S^{(p)}=X^{(p)}X^{(p)T}$, thus one has $W^{(p)}=D^{(p)}S^{(p)}D^{(p)}$. Similar to the proof of $i)$, we can prove that $SD^{2}P_{1}(X)$ and $S^{(p)}(D^{(p)})^{2}P_{1}(X)$ have no eigenvalue in common instead. It is easy to see the resultant of the characteristic polynomials of $SD^{2}P_{1}(X)$ and $S^{(p)}(D^{(p)})^{2}P_{1}(X)$ is a polynomial of $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$. Therefore, it suffices to show the resultant is a non null polynomial. Equivalently, we shall provide a sample of $\{x_{ij},1\leq i\leq p,1\leq j\leq n\}$ such that $W$ and $W^{(p)}$ have no eigenvalue in common.

Using $i)$ to $W^{(p)}$ we can denote the ordered eigenvalues of $W^{(p)}$ by $\lambda_{1}^{(p)}<\lambda_{2}^{(p)}<\cdots<\lambda_{p-1}^{(p)}$. By Cauchy’s interlacing property, one has

Moreover, we know that $\mathcal{W}^{(p)}$ shares the same nonzero eigenvalues with $W^{(p)}$. So we can provide an example such that $\mathcal{W}$ and $\mathcal{W}^{(p)}$ have no nonzero eigenvalue in common instead. Note

Taking trace on both side of (6.4), we obtain

Now we prove $iii)$. We set $X_{(n)}$ to be the submatrix of $X$ with the $n$-th column deleted and set

Let $S_{(n)}=X_{(n)}X_{(n)}^{T}$. It is obvious that $S_{(n)}\widehat{D}_{(n)}^{2}$ shares the same eigenvalues with $\widehat{W}_{(n)}$ Now we introduce the polynomials

To prove that $W$ and $\widehat{W}_{(n)}$ have no eigenvalue in common, we only need to show $SD^{2}$ and $S_{(n)}\widehat{D}_{(n)}^{2}$ have no eigenvalue in common. Moreover, if $P_{2}(X)$ does not vanish, it is equivalent to prove that the matrices $T:=SD^{2}P_{2}(X)$ and $\widehat{T}_{(n)}:=S_{(n)}\widehat{D}_{(n)}^{2}P_{2}(X)$ have no eigenvalue in common. Note that the event $P_{2}(X)=0$ has zero Lebesgue measure. What’s more, it is not difficult to see the entries of $T$ and $\widehat{T}_{(n)}$ are all polynomials of the elements of $X$, thus the resultant $R(X)$ of the characteristic polynomials of $T$ and $T_{(n)}$ is also a polynomial of the elements of $X$. Therefore, we only need to show $R(X)$ is a non null polynomial, it suffices to give only one example of $X$ such that $W$ and $\widehat{W}_{(n)}$ do not have eigenvalue in common. For example, we can choose

Then we have $\widehat{W}_{(n)}=I_{p}$ and

Thus it is easy to see $\widehat{W}_{(n)}$ and $W$ have no eigenvalue in common for $\det(W-I)\neq 0$, which implies that $R(X)$ is not a null polynomial, so we conclude the proof. ∎

Appendix B

In this appendix, we prove Lemma 3.7. If we denote

holds with overwhelming probability. It is not difficult to see

Since $(x_{1n}^{3},\cdots,x_{pn}^{3})^{T}$ is also a random vector with mean zero and finite variance entries, Lemma 2.5 can be used to the first part of the right hand side of (7.3). Thus if we set the projection

with overwhelming probability. Here we have used the fact that for any $J$

it suffices to prove the following lemma instead.

Using the notation in Lemma 3.7, we have for any $J\in\{1,\cdots,p\}$ with $|J|=d\leq nK^{-3}$,

We use the following concentration theorem, which is a consequence of Talagrand’s inequality, (see Theorem $69$ of ).

In fact, here we only need the real case of the theorem.

So to conclude the proof of Lemma 7.1, we only need to show that

holds with overwhelming probability for any small $\epsilon>0$. Here we have used the fact that

and $TrP_{J}=d$. By the condition that $d\leq nK^{-3}$, we have

Let $S_{1}:=\sum_{k=1}^{p}m_{kk}(x_{kn}^{2}-1)$. We have

And by the assumption on $K$ we also have

Set $S_{2}:=|\sum_{k\neq l}m_{kl}x_{kn}x_{ln}|$. Then we have

Both terms on the right hand side can be bounded by $1/5$ by the same argument as above. So we conclude the proof. ∎

References