On the principal components of sample covariance matrices

Introduction

In this paper we investigate $M\times M$ sample covariance matrices of the form

Correspondingly, one has to subtract from $A_{i\mu}$ the empirical mean of the $i$-th row of $A$, which we denote by $[A]_{i}\mathrel{\mathop{:}}=\frac{1}{N}\sum_{\mu=1}^{N}A_{i\mu}$. Hence, we replace (1.1) with

By the law of large numbers, if $M$ is fixed and $N$ taken to infinity, the sample covariance matrix $\mathcal{Q}$ converges almost surely to the population covariance matrix $\Sigma$. In many modern applications, however, the population size $M$ is very large and obtaining samples is costly. Thus, one is typically interested in the regime where $M$ is of the same order as $N$, or even larger. In this case, as it turns out, the behaviour of $\mathcal{Q}$ changes dramatically and the problem becomes much more difficult. In principal component analysis, one seeks to understand the correlations by considering the principal components, i.e. the top eigenvalues and associated eigenvectors, of $\mathcal{Q}$. These provide an effective low-dimensional projection of the high-dimensional data set $A$, in which the significant trends and correlations are revealed by discarding superfluous data.

the empirical eigenvalue density of the rescaled matrix $Q=\phi^{-1/2}\mathcal{Q}$ has the same asymptotics for large $M$ and $N$ as

2. Examples and outline of the model

The population covariance matrix is $\Sigma=TT^{*}=I_{M}+UU^{*}$.

Below we shall refer to these examples as Examples (1) and (2) respectively. Motivated by them, we now outline our model. Let $B$ be an $(M+r)\times N$ matrix whose entries are independent with zero mean and unit variance. We choose a deterministic $M\times(M+r)$ matrix $T$, and set $\mathcal{Q}=\frac{1}{N}TBB^{*}T^{*}$. We stress that we do not assume that the underlying randomness is Gaussian. Our key assumptions are (i) $r$ is bounded; (ii) $\Sigma-I_{M}$ has bounded rank; (iii) $\log N$ is comparable to $\log M$; (iv) the entries of $B$ are independent, with zero mean and unit variance, and have a sufficient number of bounded moments. The precise assumptions are given in Section 1.3 below. We emphasize that everything apart from $r$ and the rank of $\Sigma-I_{M}$ is allowed to depend on $N$ in an arbitrary fashion.

3. Definition of model

In this section we give the precise definition of our model and introduce some basic notations. For convenience, we always work with the rescaled sample covariance matrix

The motivation behind this rescaling is that, as observed in (1.6), it ensures that the bulk spectrum of $Q$ has asymptotically a fixed diameter, $4$, for arbitrary $N$ and $M$.

We always regard $N$ as the fundamental large parameter, and write $M\equiv M_{N}$. Here, and throughout the following, in order to unclutter notation we omit the argument $N$ in quantities, such as $M$, that depend on it. In other words, every symbol that is not explicitly a constant is in fact a sequence indexed by $N$. We assume that $M$ and $N$ satisfy the bounds

Fix a constant $r=0,1,2,3,\dots$. Let $X$ be an $(M+r)\times N$ random matrix and $T$ an $M\times(M+r)$ deterministic matrix. For definiteness, and bearing the motivation of sample covariance matrices in mind, we assume that the entries of $X$ and $T$ are real. However, our method also trivially applies to complex-valued $X$ and $T$, with merely cosmetic changes to the proofs. We consider the $M\times M$ matrix

Since $TX$ is an $M\times N$ matrix, we find that $Q$ has

We define the population covariance matrix

for the eigenvalues $\sigma_{i}$. We always order the values $d_{i}$ such that

We suppose that $\Sigma$ is positive definite, so that each $d_{i}$ lies in the interval

Moreover, we suppose that $\Sigma-I_{M}$ has bounded rank, i.e.

We assume that the entries $X_{i\mu}$ of $X$ are independent (but not necessarily identically distributed) random variables satisfying

Our results concern the eigenvalues of $Q$, denoted by

and the associated unit eigenvectors of $Q$, denoted by

4. Sketch of behaviour of the principal components of Q𝑄Q

To guide the reader, we now give a heuristic description of the behaviour of principal components of $Q$. The spectrum of $Q$ consists of a bulk spectrum and of outliers—eigenvalues separated from the bulk. The bulk contains an order $K$ eigenvalues, which are distributed on large scales according to the Marchenko-Pastur law (1.6). In addition, if $\phi>1$ there are $M-K$ trivial eigenvalues at zero. Each $d_{i}$ satisfying $\lvert d_{i}\rvert>1$ gives rise to an outlier located near its classical location

Any $d_{i}$ satisfying $\lvert d_{i}\rvert<1$ does not result in an outlier. We summarize this picture in Figure 1.1.

The creation or annihilation of an outlier as a $d_{i}$ crosses $\pm 1$ is known as the BBP phase transition BBP . It takes place on the scaleWe use the symbol $\asymp$ to denote quantities of comparable size; see “Conventions” at the end of this section for a precise definition. \bigl{\lvert}\lvert d_{i}\rvert-1\bigr{\rvert}\asymp K^{-1/3}. This scale has a simple heuristic explanation (we focus on the right edge of the spectrum). Suppose that $d_{1}\in(0,1)$ and all other $d_{i}$’s are zero. Then the top eigenvalue $\mu_{1}$ exhibits universality, and fluctuates on the scale $K^{-2/3}$ around $\gamma_{+}$ (see Theorem 8.3 and Remark 8.7 below). Increasing $d_{1}$ beyond the critical value $1$, we therefore expect $\mu_{1}$ to become an outlier when its classical location $\theta(d_{1})$ is located at a distance greater than $K^{-2/3}$ from $\gamma_{+}$. By a simple Taylor expansion of $\theta$, the condition $\theta(d_{1})-\gamma_{+}\gg K^{-2/3}$ becomes $d_{1}-1\gg K^{-1/3}$.

for $d_{i}>1$. The function $u$ determines the aperture $2\arccos\sqrt{u(d_{i})}$ of the cone. Note that $u(d_{i})\in(0,1)$ and $u(d_{i})$ converges to $1$ as $d_{i}\to\infty$. See Figure 1.2.

5. Summary of previous related results

There is an extensive literature on spiked covariance matrices. So far most of the results have focused on the outlier eigenvalues of Example (1), with the nonzero $d_{i}$ independent of $N$ and $\phi$ fixed. Eigenvectors and the non-outlier eigenvalues have seen far less attention.

For the uncorrelated case $\Sigma=I_{M}$ and Gaussian $X$ in (1.10) with fixed $\phi$, it was proved in Joh1 for the complex case and in Johnstone for the real case that the top eigenvalue, rescaled as $K^{2/3}(\mu_{1}-\gamma_{+})$, is asymptotically distributed according the Tracy-Widom law of the appropriate symmetry class TW1 ; TW2 . Subsequently, these results were shown to be universal, i.e. independent of the distribution of the entries of $X$, in Sosh2 ; PY . The assumption that $\phi$ be fixed was relaxed in Pe1 ; ElK1 .

The study of covariance matrices with nontrivial population covariance matrix $\Sigma\neq I_{M}$ goes back to the seminal paper of Johnstone Johnstone , where the Gaussian spiked model was introduced. The BBP phase transition was established by Baik, Ben Arous, and Péché BBP for complex Gaussian $X$, fixed rank of $\Sigma-I_{M}$, and fixed $\phi$. Subsequently, the results of BBP were extended to the other Gaussian symmetry classes, such as real covariance matrices, in BV1 ; BV2 . The proofs of BBP ; Pec use an asymptotic analysis of Fredholm determinants, while those of BV1 ; BV2 use an explicit tridiagonal representation of $XX^{*}$; both of these approaches rely heavily on the Gaussian nature of $X$. See also PeBor for a generalization of the BBP phase transition.

For the model from Example (1) with fixed nonzero $\{d_{i}\}$ and $\phi$, the almost sure convergence of the outliers was established in BS . It was also shown in BS that if $\lvert d_{i}\rvert<1$ for all $i$, the top eigenvalue $\mu_{1}$ converges to $\gamma_{+}$. For this model, a central limit theorem of the outliers was proved in BY1 . In BY2 , the almost sure convergence of the outliers was proved for a generalized spiked model whose population covariance matrix is of the block diagonal form $\Sigma=\operatorname{diag}(A,T)$, where $A$ is a fixed $r\times r$ matrix and $T$ is chosen so that the associated sample covariance matrix has no outliers.

In BGN , the almost sure convergence of the projection of the outlier eigenvectors onto the finite-dimensional spike subspace was established, under the assumption that $\phi$ and the nonzero $d_{i}$ are fixed, and that $B$ and $T$ are both random and one of them is orthogonally invariant. In particular, the cone concentration from (1.18) was established in BGN . In Paul , under the assumption that $X$ is Gaussian and $\phi$ and the nonzero $d_{i}$ are fixed, a central limit theorem for a certain observable, the so-called sample vector, of the outlier eigenvectors was established. The result of Paul was extended to non-Gaussian entries for a special class of $\Sigma$ in Shi .

Moreover, in BGN2 ; NA1 result analogous to those of BGN were obtained for the model from Example (2). Finally, a related class of models, so-called deformed Wigner matrices, have been the subject of much attention in recent years; we refer to SoshPert ; SoshPert2 ; KY2 ; KY3 for more details; in particular, the joint distribution of all outliers was derived in KY3 .

6. Overview of results

In this subsection we give an informal overview of our results.

Our results on the eigenvalues of $Q$ consist of two parts. First, we derive large deviation bounds on the locations of the outliers (Theorem 2.3). Second, we prove eigenvalue sticking for the non-outliers (Theorem 2.7), whereby each non-outlier “sticks” with high probability and very accurately to the eigenvalues of a related covariance matrix satisfying $\Sigma=I_{M}$ and whose top eigenvalues exhibit universality. As a corollary (Remark 8.7), we prove that the top non-outlier eigenvalue of $Q$ has asymptotically the Tracy-Widom-1 distribution. This sticking is very accurate if all $d_{i}$’s are separated from the critical point $1$, and becomes less accurate if a $d_{i}$ is in the vicinity of $1$. Eventually, it breaks down precisely on the BBP transition scale $\lvert d_{i}-1\rvert\asymp K^{-1/3}$, at which the Tracy-Widom-1 distribution is known not to hold for the top non-outlier eigenvalue. These results generalize those from (KY2, , Theorem 2.7).

If the outlier $\mu_{i}$ approaches the bulk spectrum or another outlier, the cone concentration becomes less accurate. For the case of two nearby outlier eigenvalues, for instance, the cone concentration (1.18) of the eigenvectors breaks down when the distributions of the outlier eigenvalues have a nontrivial overlap. In order to understand this behaviour in more detail, we introduce the deterministic projection

Finally, the proofs of universality of the non-outlier eigenvalues and eigenvectors require the universality of $Q$ for the uncorrelated case $\Sigma=I_{M}$ as input. This universality result is given in Theorem 8.3, which is also of some independent interest. It establishes the joint, fixed-index, universality of the eigenvalues and eigenvectors of $Q$ (and hence, as a special case, the quantum unique ergodicity of the eigenvectors of $Q$ mentioned in Section 1.1). It works for all eigenvalue indices $i$ satisfying $i\leqslant K^{1-\tau}$ for any fixed $\tau>0$.

We conclude this subsection by outlining the key novelties of our work.

We introduce the general models $Q$ from (1.10) and $\dot{Q}$ from (2.23) below, which subsume and generalize several models considered previously in the literatureIn particular, the current paper is the first to study the principal components of a realistic sample covariance matrix (1.3) instead of the zero mean case (1.1).. We allow the entries of $X$ to be arbitrary random variables (up to a technical assumption on their tails). All quantities except $r$ and the rank of $\Sigma-I_{M}$ may depend on $N$. We make no assumption on $T$ beyond the bounded-rank condition of $TT^{*}-I_{M}$. The dimensions $M$ and $N$ may be wildly different, and are only subject to the technical condition (1.9).

We obtain quantitative bounds (i.e. rates of convergence) on the outlier eigenvalues and the generalized components of the eigenvectors. We believe these bounds to be optimal.

We establish the joint, fixed-index, universality of the eigenvalues and eigenvectors for the case $\Sigma=I_{M}$. This result holds for any eigenvalue indices $i$ satisfying $i\leqslant K^{1-\tau}$ for an arbitrary $\tau>0$. Note that previous works KY1 ; TV3 (established in the context of Wigner matrices) required either the much stronger condition $i\leqslant(\log K)^{C\log\log K}$ or a four-moment matching condition.

We remark that the large deviation bounds derived in this paper also allow one to derive the joint distribution of the generalized components of the outlier eigenvectors; this will be the subject of future work.

Conventions

The fundamental large parameter is $N$. All quantities that are not explicitly constant may depend on $N$; we almost always omit the argument $N$ from our notation.

We use $\tau>0$ in various assumptions to denote a positive constant that may be chosen arbitrarily small. A smaller value of $\tau$ corresponds to a weaker assumption. All of our estimates depend on $\tau$, and we neither indicate nor track this dependence.

Results

In this section we state our main results. The following notion of a high-probability bound was introduced in EKY2 , and has been subsequently used in a number of works on random matrix theory. It provides a simple way of systematizing and making precise statements of the form “$A$ is bounded with high probability by $B$ up to small powers of $N$”.

be two families of nonnegative random variables, where $U^{(N)}$ is a possibly $N$-dependent parameter set. We say that $A$ is stochastically dominated by $B$, uniformly in $u$, if for all (small) $\varepsilon>0$ and (large) $D>0$ we have

for large enough $N\geqslant N_{0}(\varepsilon,D)$. Throughout this paper the stochastic domination will always be uniform in all parameters (such as matrix indices) that are not explicitly fixed. Note that $N_{0}(\varepsilon,D)$ may depend on the constants from (1.9) and (1.16) as well as any constants fixed in the assumptions of our main results. If $A$ is stochastically dominated by $B$, uniformly in $u$, we use the notation $A\prec B$. Moreover, if for some complex family $A$ we have $\lvert A\rvert\prec B$ we also write $A=O_{\prec}(B)$.

Because of (1.9), all (or some) factors of $N$ in Definition (2.1) could be replaced with $M$ without changing the definition of stochastic domination.

We begin with results on the locations of the eigenvalues of $Q$. These results will also serve as a fundamental input for the proofs of the results on eigenvectors presented in Sections 2.2 and 2.3.

Recall that $Q$ has $M-K$ zero eigenvalues. We shall therefore focus on the $K$ nontrivial eigenvalues $\mu_{1}\geqslant\cdots\geqslant\mu_{K}$ of $Q$. On the global scale, the eigenvalues of $Q$ are distributed according to the Marchenko-Pastur law (1.6). This may be easily inferred from the fact that (1.6) gives the global density of the eigenvalues for the uncorrelated case $\Sigma=I_{M}$, combined with eigenvalue interlacing (see Lemma 4.1 below). In this section we focus on local eigenvalue information.

As explained in Section 1.4, each $i\in\mathcal{O}$ gives rise to an outlier of $Q$ near the classical location $\theta(d_{i})$ defined in (1.17). In the definition (2.2), the lower bound $1+K^{-1/3}$ is chosen for definiteness; it could be replaced with $1+aK^{-1/3}$ for any fixed $a>0$. We denote by

the number of outliers to the left ($s_{-}$) and right ($s_{+}$) of the bulk spectrum.

The function $\Delta(d)$ will be used to give an upper bound on the magnitude of the fluctuations of an outlier associated with $d$. We give such a precise expression for $\Delta$ in order to obtain sharp large deviation bounds for all $d\in\mathcal{D}\setminus$. (Note that the discontinuity of $\Delta$ at $d=2$ is immaterial since $\Delta$ is used as an upper bound with respect to $\prec$. The ratio of the right- and left-sided limits at $2$ of $\Delta$ lies in $$.)

Our result on the outlier eigenvalues is the following.

Fix $\tau>0$. Then for $i\in\mathcal{O}$ we have the estimate

provided that $d_{i}>0$ or $\lvert\phi-1\rvert\geqslant\tau$.

Furthermore, the extremal non-outliers $\mu_{s_{+}+1}$ and $\mu_{K-s_{-}}$ satisfy

and, assuming in addition that $\lvert\phi-1\rvert\geqslant\tau$,

Theorem 2.3 gives large deviation bounds for the locations of the outliers to the right of the bulk. Since $\tau>0$ may be arbitrarily small, Theorem 2.3 also gives the full information about the outliers to the left of the bulk except in the case $1>\phi=1+o(1)$. Although our methods may be extended to this case as well, we exclude it here to avoid extraneous complications.

By definition of $s_{-}$ and $\mathcal{D}$, if $\phi>1$ then $s_{-}=0$. Hence, by (2.6), if $\phi>1$ there are no outliers on the left of the bulk spectrum.

Previously, the model from Example (1) in Section 1.2 with fixed nonzero $\{d_{i}\}$ and $\phi$ was investigated in BS ; BY1 . In BS , it was proved that each outlier eigenvalue $\mu_{i}$ with $i\in\mathcal{O}$ convergences almost surely to $\theta(d_{i})$. Moreover, a central limit theorem for $\mu_{i}$ was established in BY1 .

The locations of the non-outlier eigenvalues $\mu_{i}$, $i\notin\mathcal{O}$, are governed by eigenvalue sticking, whereby the eigenvalues of $Q$ “stick” with high probability to eigenvalues of a reference matrix which has a trivial population covariance matrix. The reference matrix is $Q$ from (1.10) with uncorrelated entries. More precisely, we set

Fix $\tau>0$. Then we have for all $i\in[\![{1,(1-\tau)K}]\!]$

Similarly, if $\lvert\phi-1\rvert\geqslant\tau$ then we have for all $i\in[\![{\tau K,K}]\!]$

As outlined above, in Theorem 8.3 below we prove that the asymptotic joint distribution of the non-bulk eigenvalues of $H$ is universal, i.e. it coincides with that of the Wishart matrix $H_{\text{\rm{Wish}}}=XX^{*}$ with $r=0$ and $X$ Gaussian. As an immediate corollary of Theorems 2.7 and 8.3, we obtain the universality of the non-outlier eigenvalues of $Q$ with index $i\leqslant K^{1-\tau}\alpha_{+}^{3}$. This condition states simply that the right-hand side of (2.9) is much smaller than the scale on which the eigenvalue $\lambda_{i}$ fluctuates, which is $K^{-2/3}i^{-1/3}$. See Remark 8.7 below for a precise statement.

Theorem 2.7 is analogous to Theorem 2.7 of KY2 , where sticking was first established for Wigner matrices. Previously, eigenvalue sticking was established for a certain class of random perturbations of Wigner matrices in BGGM1 ; BGGM2 . We refer to (KY2, , Remark 2.8) for a more detailed discussion.

Aside from holding for general covariance matrices of the form (1.10), Theorem 2.7 is stronger than its counterpart from KY2 because it holds much further into the bulk: in (KY2, , Theorem 2.7), sticking was established under the assumption that $i\leqslant(\log K)^{C\log\log K}$.

The edge universality following from Theorem 2.7 (as explained in Remark 2.8) generalizes the recent result BaoPanZhou2 . There, for the model from Example (1) in Section 1.2 with fixed nonzero $\{d_{i}\}$ and $\phi$, it was proved that if $d_{i}<1$ for all $i$ and $\Sigma$ is diagonal, then $\mu_{1}$ converges (after a suitable affine transformation) in distribution to the Tracy-Widom-1 distribution.

2. Outlier eigenvectors

For $i\in[\![{1,M}]\!]$ we define $\nu_{i}\geqslant 0$ through

For definiteness, we only state our results for the outliers on the right-hand side of the bulk spectrum. Analogous results hold for the outliers on the left-hand side. Since the behaviour of the fluctuating error term is different in the regimes $\mu_{i}-\gamma_{+}\ll 1$ (near the bulk) and $\mu_{i}-\gamma_{+}\gg 1$ (far from the bulk), we split these two cases into separate theorems.

Fix $\tau>0$. Suppose that $A\subset\mathcal{O}$ satisfies $1+K^{-1/3}\leqslant d_{i}\leqslant\tau^{-1}$ for all $i\in A$. Define the deterministic positive quadratic form

We emphasize that the set $A$ in Theorem 2.11 may be chosen at will. If all outliers are well-separated, then the choice $A=\{i\}$ gives the most precise information. However, as explained at the beginning of this subsection, the indices of outliers that are close to each other should be included in the same set $A$. Thus, the freedom to chose $\lvert A\rvert\geqslant 2$ is meant for degenerate or almost degenerate outliers. (In fact, as explained after (2.14) below, the correct notion of closeness of outliers is that of overlapping.)

This gives a precise version of the cone concentration from (1.18). Note that the cone concentration holds provided the error is much smaller than the main term $u(d_{i})$, which leads to the conditions

here we used that $d_{i}\asymp 1$ and $M\asymp(1+\phi)K$.

We claim that both conditions in (2.14) are natural and necessary. The first condition of (2.14) simply means that $\mu_{i}$ is an outlier. The second condition of (2.14) is a non-overlapping condition. To understand it, recall from (2.4) that $\mu_{i}$ fluctuates on the scale $(d_{i}-1)^{1/2}K^{-1/2}$. Then $\mu_{i}$ is a non-overlapping outlier if all other outliers are located with high probability at a distance greater than this scale from $\mu_{i}$. Recalling the definition of the classical location $\theta(d_{i})$ of $\mu_{i}$, the non-overlapping condition becomes

After a simple estimate using the definition of $\theta$, we find that this is precisely the second condition of (2.14). The degeneracy or almost degeneracy of outliers discussed at the beginning of this subsection is hence to be interpreted more precisely in terms of overlapping of outliers.

Provided $\mu_{i}$ is well-separated from both the bulk spectrum and the other outliers, we find that the error in (2.13) is of order $M^{-1/2}$.

As $d_{j}$ approaches $d_{i}$ the delocalization bound from (2.16) deteriorates, and eventually when $\mu_{i}$ and $\mu_{j}$ start overlapping, i.e. the second condition of (2.14) is violated, the right-hand side of (2.16) has the same size as the leading term of (2.13). This is again a manifestation of the fact that the individual eigenspaces of overlapping outliers cannot be distinguished.

Suppose that we have an $\lvert A\rvert$-fold degenerate outlier, i.e. $d_{i}=d_{j}$ for all $i,j\in A$. Then from Theorem 2.11 and Remark 2.12 (see the estimate (5.2)) we get, for all $i,j\in A$,

This is the correct generalization of (2.13) from Example 2.13 to the degenerate case. The error term is the same as in (2.13), and its size and relation to the main term is exactly the same as in Example 2.13. Hence the discussion following (2.13) may be take over verbatim to this case.

We conclude this example by remarking that a similar discussion also holds for a group of outliers that is not degenerate, but nearly degenerate, i.e. $\lvert d_{i}-d_{j}\rvert\ll\lvert d_{i}-d_{k}\rvert$ for all $i,j\in A$ and $k\notin A$. We omit the details.

The next result is the analogue of Theorem 2.11 for outliers far from the bulk.

We leave the discussion on the interpretation of the error in (2.17) to the reader; it is similar to that of Examples 2.13, 2.14, and 2.15.

3. Non-outlier eigenvectors

This quantity should be interpreted as a deterministic version of $\lvert\mu_{a}-\gamma_{-}\rvert\wedge\lvert\mu_{a}-\gamma_{+}\rvert$ for $a\notin\mathcal{O}$; see Theorem 3.5 below.

Suppose that $a\leqslant C$ ($\mu_{a}$ is near the edge), which implies that $\kappa_{a}\asymp K^{-2/3}$. Suppose moreover that $d_{i}$ is near the transition point $1$. Then we get

An analogous statement holds near the left spectral edge provided $\lvert\phi-1\rvert\geqslant\tau$; we omit the details.

More generally, our method also yields the asymptotic joint distribution of the family

(after a suitable affine rescaling of the variables, as in Theorem 8.3 below), where $a_{1},\dots,a_{k},b_{1},\dots,b_{k}\in[\![{1,K^{1-\tau}\alpha_{+}^{3}}]\!]\setminus\mathcal{O}$. We omit the precise statement, which is a universality result: it says essentially that the asymptotic distribution of (2.22) coincides with that under the standard Wishart ensemble (i.e. an uncorrelated Gaussian sample covariance matrix). The proof is a simple corollary of Theorem 2.7, Proposition 6.2, Proposition 6.3, and Theorem 8.3.

Finally, instead of $Q$ defined in (1.10), we may also consider

Preliminaries

The rest of this paper is devoted to the proofs of the results from Sections 2.1–2.3. To clarify the presentation of the main ideas of the proofs, we shall first assume that

We make the assumption (3.1) throughout Sections 3–7. The additional arguments required to relax the assumption (3.1) are presented in Section 8. Under the assumption (3.1) we have

Moreover, the extension of our results from $Q$ to $\dot{Q}$, and hence the proof of Theorem 2.23, is given in Section 9.

In this section we collect the key tool of our analysis: the isotropic Marchenko-Pastur law from BEKYY .

It is well known that the empirical distribution of the eigenvalues of the $N\times N$ matrix $X^{*}X$ has the same asymptotics as the Marchenko-Pastur law

where we recall the edges $\gamma_{\pm}$ of the limiting spectrum defined in (1.7). Similarly, as noted in (1.6), the empirical distribution of the eigenvalues of the $M\times M$ matrix $XX^{*}$ has the same asymptotics as $\varrho_{\phi^{-1}}$.

Note that (3.3) is normalized so that its integral is equal to one. The Stieltjes transform of the Marchenko-Pastur law (3.3) is

where the square root is chosen so that $m_{\phi}$ is holomorphic in the upper half-plane and satisfies $m_{\phi}(z)\to 0$ as $z\to\infty$. The function $m_{\phi}=m_{\phi}(z)$ is also characterized as the unique solution of the equation

satisfying $\operatorname{Im}m(z)>0$ for $\operatorname{Im}z>0$. The formulas (3.3)–(3.5) were originally derived for the case when $\phi=M/N$ is independent of $N$ (or, more precisely, when $\phi$ has a limit in $(0,\infty)$ as $N\to\infty$). Our results allow $\phi$ to depend on $N$ under the constraint (1.9), so that $m_{\phi}$ and $\varrho_{\phi}$ may also depend on $N$ through $\phi$.

Throughout the following we use a spectral parameter

with $\eta>0$, as the argument of Stieltjes transforms and resolvents. Define the resolvent

Throughout the following we regard the quantities $E(z)$, $\eta(z)$, and $\kappa(z)$ as functions of $z$ and usually omit the argument unless it is needed to avoid confusion.

Sometimes we shall need the following notion of high probability.

Fix a (small) $\omega\in(0,1)$ and define the domain

Beyond the support of the limiting spectrum, one has stronger control all the way down to the real axis. For fixed (small) $\omega>0$ define the region

of spectral parameters separated from the asymptotic spectrum by $K^{-2/3+\omega}$, which may have an arbitrarily small positive imaginary part $\eta$. Throughout the following we regard $\omega$ as fixed once and for all, and do not track the dependence of constants on $\omega$.

Suppose that (1.15), (1.9), and (1.16) hold. Then

for all $\varepsilon>0$, $D>0$, and $N\geqslant N_{0}(\varepsilon,D)$. See (BEKYY, , Remark 2.6).

The next results are on the nontrivial (i.e. nonzero) eigenvalues of $H\mathrel{\mathop{:}}=XX^{*}$ as well as the corresponding eigenvectors. The matrix $H$ has $K$ nontrivial eigenvalues, which we order according to

(The remaining $M-K$ eigenvalues of $H$ are zero.) Moreover, we denote by

the unit eigenvectors of $H$ associated with the nontrivial eigenvalues $\lambda_{1}\geqslant\lambda_{2}\geqslant\dots\geqslant\lambda_{K}$.

Fix $\tau>0$, and suppose that (1.15), (1.9), and (1.16) hold. Then for $i\in[\![{1,K}]\!]$ we have

if either $i\leqslant(1-\tau)K$ or $\lvert\phi-1\rvert\geqslant\tau$.

The following result is on the rigidity of the nontrivial eigenvalues of $H$. Let $\gamma_{1}\geqslant\gamma_{2}\geqslant\cdots\geqslant\gamma_{K}$ be the classical eigenvalue locations according to $\varrho_{\phi}$ (see (3.3)), defined through

Fix $\tau>0$, and suppose that (1.15), (1.9), and (1.16) hold. Then for $i\in[\![{1,M}]\!]$ we have

if $i\leqslant(1-\tau)K$ or $\lvert\phi-1\rvert\geqslant\tau$.

2. Link to the semicircle law

Note that this is nothing but Wigner’s semicircle law centred at $\phi^{1/2}+\phi^{-1/2}$. Thus,

where in the last step we used (3.4). Note that

The estimates (3.19) and (3.20) follow from the explicit expressions in (3.4) and (3.17). In fact, these estimates have already appeared in previous works. Indeed, for $m_{\phi}$ the estimates (3.19) and (3.20) were proved in (BEKYY, , Lemma 3.3). In order to prove them for $w_{\phi}$, we observe that the estimates (3.19) and (3.20) follow from the corresponding ones for the semicircle law, which were proved in (EKYY4, , Lemma 4.3). The estimates (3.21) follow from (3.20) and the elementary identity

which can be derived from (3.18); the estimates for $m_{\phi}$ are derived similarly. Finally, (3.22) follows easily from

which may itself be derived from (3.5). ∎

In analogy to $w_{\phi}$ (see (3.17)), we define the matrix-valued function

Theorem 3.2 has the following analogue, which compares $F$ with $m_{\phi}$.

Suppose that (1.15), (1.9), and (1.16) hold. Then

3. Extension of the spectral domain

In this section we extend the spectral domain on which Theorem 3.2 and Lemma 3.7 hold. The argument relies on the Helffer-Sjöstrand functional calculus Davies . Define the domains

The basic idea of the proof is to apply the Helffer-Sjöstrand formula to the function

where $x_{0}$ is chosen below. To that end, we need a smooth compactly supported cutoff function $\chi$ on the complex plane satisfying $\chi(w)\in$ and $\lvert\partial_{\bar{w}}\chi(w)\rvert\leqslant C(\omega,\tau)$. We distinguish the three cases $\phi<1-\tau$, $\lvert\phi-1\rvert\leqslant\tau$, and $\phi>1+\tau$.

Let us first focus on the case $\phi<1-\tau$. Set $x_{0}\mathrel{\mathop{:}}=\phi^{1/2}+\phi^{-1/2}$ and choose a constant $\omega^{\prime}=\omega^{\prime}(\omega,\tau)\in(0,\omega)$ small enough that $\gamma_{-}\geqslant 4\omega^{\prime}$. We require that $\chi$ be equal to $1$ in the $\omega^{\prime}$-neighbourhood of $[\gamma_{-},\gamma_{+}]$ and outside of the $2\omega^{\prime}$-neighbourhood of $[\gamma_{-},\gamma_{+}]$. By Theorem 3.5 we have $\operatorname{supp}\rho^{\Delta}\subset\{\chi=1\}$ with high probability. Now choose $z$ satisfying $\operatorname{dist}(z,[\gamma_{-},\gamma_{+}])\geqslant 3\omega^{\prime}$. Then the Helffer-Sjöstrand formula Davies yields, for $x\in\operatorname{supp}\rho^{\Delta}$,

Finally, suppose that $\phi>1+\tau$. Now we set $x_{0}\mathrel{\mathop{:}}=0$. We choose the same $\omega^{\prime}$ and cutoff function $\chi$ as in the case $\phi<1-\tau$ above. Suppose that $\operatorname{dist}(z,[\gamma_{-},\gamma_{+}])\geqslant 3\omega^{\prime}$ and $z\neq 0$. Thus, (3.30) holds with high probability for $x\in\operatorname{supp}\rho^{\Delta}\setminus\{0\}$. Since $f_{w}(0)=0$, we therefore find that (3.31) holds. As above, we find that for $w\in\{\partial_{\bar{w}}\chi\neq 0\}$ we have

and $\lvert m^{\Delta}(w)\rvert\prec\phi^{-1}K^{-1/2}$. Recalling (3.10), we find that (3.28) follows easily. ∎

Proposition 3.8 yields the following result for $F$ defined in (3.24).

4. Identities for the resolvent and eigenvalues

In this section we derive the identities on which our analysis of the eigenvalues and eigenvectors relies. Recall the definition of the set $\mathcal{R}$ from (1.14). We write the population covariance matrix $\Sigma$ from (1.12) as

where $H=XX^{*}$ and $Q=\Sigma^{1/2}H\Sigma^{1/2}$. We introduce the $\lvert\mathcal{R}\rvert\times\lvert\mathcal{R}\rvert$ matrix

We also denote by $\sigma(A)$ the spectrum of a square matrix $A$.

The following lemma collects the basic identities for analysing $\sigma(Q)$ and $\widetilde{G}$. We remark that versions of its part (i) have already appeared in several previous works on finite-rank deformations of random matrix ensembles BGGM1 ; BY1 ; KY2 ; SoshPert .

Suppose that $\mu\notin\sigma(H)$. Then $\mu\in\sigma(Q)$ if and only if

To prove (i), we write the condition $\mu\in\sigma(Q)$ as

where we used that $\mu\notin\sigma(H)$. Using

the matrix identity $\det(1+XY)=\det(1+YX)$, and $\det(\Sigma)\neq 0$, we find

with $A=H-z$, $B=D(\phi^{-1/2}+D)^{-1}$, $S=V$, and $T=zV^{*}$. ∎

The result (3.35), when restricted to the range of $V$, has an alternative form (3.37) which is often easier to work with, since it collects all of the randomness in the single quantity $W(z)$ on its right-hand side.

Eigenvalue locations

In this section we prove Theorems 2.3 and 2.7. The arguments are similar to those of (KY2, , Section 6), and we therefore only sketch the proofs. The proof of (KY2, , Section 6) relies on three main steps: (i) establishing a forbidden region which contains with high probability no eigenvalues of $Q$; (ii) a counting estimate for the special case where $D$ does not depend on $N$, which ensures that each connected component of the allowed region (complement of the forbidden region) contains exactly the right number of eigenvalues of $Q$; and (iii) a continuity argument where the counting result of (ii) is extended to arbitrary $N$-dependent $D$ using the gaps established in (i) and the continuity of the eigenvalues as functions of the matrix entries. The steps (ii) and (iii) are exactly the same as in KY2 , and will not be repeated here. The step (i) differs slightly from that of KY2 , and in the proofs below we explain these differences.

Let $\lvert\mathcal{R}\rvert=1$ and $D=d\in\mathcal{D}$. For $d>0$ we have

Writing this in spectral decomposition yields

As above, a simple perturbation argument implies that we may without loss of generality assume that all scalar products in (4.1) are nonzero. Now take $z\in(0,\infty)$. Note that $b(z)$ and $d$ have the same sign.

To conclude the proof, we observe that the left-hand side of (4.1) defines a function of $z\in(0,\infty)$ with $M-1$ singularities and $M$ zeros, which is smooth and decreasing away from the singularities. Moreover, its zeros are the eigenvalues $\lambda_{1},\dots,\lambda_{M}$. The interlacing property now follows from the fact that $z$ is an eigenvalue of $Q$ if and only if the left-hand side of (4.1) is equal to $-b(z)$. ∎

For the rank-$\lvert\mathcal{R}\rvert$ model (1.10) we have

with the convention that $\lambda_{i}=0$ for $i>K$ and $\lambda_{i}=\infty$ for $i<1$.

Throughout the following we shall make use of the subsets of outliers

for $\tau\geqslant 0$. Note that $\mathcal{O}\;=\;\mathcal{O}_{0}^{+}\cup\mathcal{O}_{0}^{-}$.

The proof of Proposition 2.3 is similar to that of (KY2, , Equation (2.20)). We focus first on the outliers to the right of the bulk spectrum. Let $\varepsilon>0$. We shall prove that there exists an event $\Xi$ of high probability (see Definition 3.1) such that for all $i\in\mathcal{O}_{4\varepsilon}^{+}$ we have

and for $i\in[\![{\lvert\mathcal{O}_{4\varepsilon}^{+}\rvert+1,\lvert\mathcal{O}_{4\varepsilon}^{+}\rvert+r}]\!]$ we have

Before proving (4.3) and (4.4), we show how they imply (2.4) for $d_{i}>0$ and (2.5). From (4.4) we get for $i$ satisfying $K^{-1/3}\leqslant d_{i}-1\leqslant K^{-1/3+4\varepsilon}$

Since $\varepsilon>0$ was arbitrary, (2.4) for $d_{i}>0$ and (2.5) follow from (4.3) and (4.5).

What remains is the proof of (4.3) and (4.4). As in (KY2, , Proposition 6.5), the first step is to prove that with high probability there are no eigenvalues outside a neighbourhood of the classical outlier locations $\theta(d_{i})$. To that end, we define for each $i\in\mathcal{O}_{\varepsilon}^{+}$ the interval

Moreover, we set $I_{0}\mathrel{\mathop{:}}=[0,\theta(1+K^{-1/3+2\varepsilon})]$.

We now claim that with high probability the complement of the set $I(D)\mathrel{\mathop{:}}=I_{0}\cup\bigcup_{i\in\mathcal{O}_{\varepsilon}^{+}}I_{i}(D)$ contains no eigenvalues of $Q$. Indeed, from Theorem 3.5 and Corollary 3.9 combined with Remark 3.3 (with small enough $\omega\equiv\omega(\varepsilon)$), we find that there exists an event $\Xi$ of high probability such that $\lvert\lambda_{i}-\gamma_{+}\rvert\leqslant K^{-2/3+\varepsilon}$ for $i\in[\![{1,2r}]\!]$ and

for all $x\notin I_{0}$, where we defined

is singular. Since $-d_{i}^{-1}=w_{\phi}(\theta(d_{i}))$ for $i\in\mathcal{O}_{\varepsilon}^{+}$, we conclude from the definition of $I(D)$ that it suffices to show that if $x\notin I(D)$ then

We prove (4.6) using the two following observations. First, $w_{\phi}$ is monotone increasing on $(\gamma_{+},\infty)$ and

We omit further details, which may be found e.g. in (KY2, , Section 6). Thus we conclude that on the event $\Xi$ the complement of $I(D)$ contains no eigenvalues of $Q$.

The next step of the proof consists in making sure that the allowed neighbourhoods $I_{i}(D)$ contain exactly the right number of outliers; the counting argument (sketched in the steps (ii) and (iii) at the beginning of this section) follows that of (KY2, , Section 6). First we consider the case $D=D(0)$ where for all $i\neq j\in\mathcal{O}_{\varepsilon}^{+}$ we have $d_{i}(0),d_{j}(0)\geqslant 2$ and $\lvert d_{i}(0)-d_{j}(0)\rvert\geqslant 1$, and show that each interval $\{I_{i}(D(0))\mathrel{\mathop{:}}i\in\mathcal{O}_{\varepsilon}^{+}\}$ contains exactly one eigenvalue of $Q$ (see (KY2, , Proposition 6.6)). We then deduce the general case by a continuity argument, by choosing an appropriate continuous path $(D(t))_{t\in}$ joining the initial configuration $D(0)$ to the desired final configuration $D=D(1)$. The continuity argument requires the existence of a gap in the set $I(D)$ to the left of $\bigcup_{i\in\mathcal{O}_{4\varepsilon}^{+}}I_{i}(D)$. The existence of such a gap follows easily from the definition of $I(D)$ and the fact that $\lvert\mathcal{R}\rvert$ is bounded. The details are the same as in (KY2, , Section 6.5). Hence (4.3) follows. Moreover, (4.4) follows from the same argument combined with Corollary 4.2 for a lower bound on $\mu_{i}$. This concludes the analysis of the outliers to the right of the bulk spectrum.

The case of outliers to the left of the bulk spectrum is analogous. Here we assume that $\phi<1-\tau$. The argument is exactly the same as for $d_{i}>0$, except that we use the bound (3.32) to the left of the bulk spectrum as well as $\lvert\lambda_{i}-\gamma_{-}\rvert\leqslant K^{-2/3+\varepsilon}$ for $i\in[\![{K-2r,K}]\!]$ with high probability. ∎

We only give the proof of (2.9); the proof of (2.10) is analogous. Fix $\varepsilon>0$. By Theorem 2.3, Theorem 3.5, Theorem 3.2, Lemma 3.7, and Remark 3.3, there exists a high-probability event $\Xi\equiv\Xi_{N}(\varepsilon)$ satisfying the following conditions.

For the following we fix a realization $H\in\Xi$. We suppose first that

and define $\eta\mathrel{\mathop{:}}=K^{-1+2\varepsilon}\alpha_{+}^{-1}$. Now suppose that $x$ satisfies

We shall show, using (3.34), that any $x$ satisfying (4.11) cannot be an eigenvalue of $Q$. First we deduce from (4.8) that

The estimate (4.12) follows by spectral decomposition of $F(\cdot)$ together with the estimate $2\lvert\lambda_{i}-x\rvert\geqslant\sqrt{(\lambda_{i}-x)^{2}+\eta^{2}}$ for all $i$. We get from (4.12) and Lemma 3.6 that

where we use the notation $A=B+O(t)$ to mean $\lVert A-B\rVert\leqslant Ct$. Recalling (3.34), we conclude that on the event $\Xi$ the value $x$ is not an eigenvalue of $Q$ provided

It is easy to check that this condition is satisfied if

where we used (4.10). Recalling (4.7), we therefore conclude that for $i\leqslant K^{1-2\varepsilon}\alpha_{+}^{3}$ the set

The next step of the proof is a counting argument (sketched in the steps (ii) and (iii) at the beginning of this section), which uses the eigenvalue interlacing from Lemma 4.1. They details are the same as in (KY2, , Section 6), and hence omitted here. The counting argument implies that for $i\leqslant K^{1-2\varepsilon}\alpha_{+}^{3}$ and assuming (4.10) we have

What remains is to check (4.13) for the cases $\alpha_{+}<K^{-1/3+\varepsilon}$ and $i>K^{1-2\varepsilon}\alpha_{+}^{3}$.

Suppose first that $\alpha_{+}<K^{-1/3+\varepsilon}$. Then using the rigidity from (4.7) and interlacing from Corollary 4.2 we find

where we used the trivial bound $i\geqslant 1$. Similarly, if $i>K^{1-2\varepsilon}\alpha_{+}^{3}$ satisfies $i\leqslant(1-\tau)K$, we may repeat the same estimate.

We conclude that (4.13) under the sole assumption that $i\leqslant(1-\tau)K$. Since $\varepsilon>0$ was arbitrary, (2.9) follows. ∎

Outlier eigenvectors

In this section we focus on the outlier eigenvectors $\xi_{a}$, $a\in\mathcal{O}$. Here we in fact prove Theorem 2.11 under the stronger assumption

instead of $1+K^{-1/3}\leqslant d_{i}\leqslant\tau^{-1}$. How to improve the lower bound from $1+K^{-1/3+\tau}$ to the claimed $K^{-1/3}$ requires a completely different approach, relying on eigenvector delocalization bounds, and is presented in Section 6 in conjunction with results for the non-outlier eigenvectors $\xi_{a}$, $a\notin\mathcal{O}$.

The proof of Theorem 2.16 is similar to that of Theorem 2.11; one has to adapt the proof to cover the range $d_{i}\in[1+\tau,\infty)$ instead of $d_{i}\in[1+K^{-1/3},\tau^{-1}]$. The key input is the extension of the spectral domain from Corollary 3.9. For the sake of brevity we omit the details of the proof of Theorem 2.16, and focus solely on Theorem 2.11.

The following proposition is the main result of this section.

Fix $\tau>0$. Suppose that $A$ satisfies (5.1). Then for all $i,j=1,\dots,M$ we have

where the symbol $(i\leftrightarrow j)$ denotes the preceding terms with $i$ and $j$ interchanged.

Note that, under the assumption (5.1), Theorem 2.11 is an easy consequence of Proposition 5.1. As explained above, the proof of Theorem 2.11 in full generality is given in Section 6, where we give the additional argument required to relax (5.1).

The rest of this section is devoted to the proof of Proposition 5.1.

We first prove a slightly stronger version of (5.2) under the additional non-overlapping condition

for all $i\in A$, where $\delta>0$ is a positive constant. This is a precise version of the second condition of (2.14), whose interpretation was given below (2.14): an outlier indexed by $A$ cannot overlap with an outlier indexed by $A^{c}$. Note, however, that there is no restriction on the outliers indexed by $A$ overlapping among themselves. The assumption (5.3) will be removed in Section 5.2. The main estimate for non-overlapping outliers is the following.

Fix $\tau>0$ and $\delta>0$. Suppose that $A$ satisfies (5.1) and (5.3) for all $i\in A$. Then for all $i,j=1,\dots,M$ we have

The rest of this subsection is devoted to the proof of Proposition 5.2. We begin by defining $\omega\mathrel{\mathop{:}}=\tau/2$ and letting $\varepsilon<\min\{{\tau/3,\delta}\}$ be a positive constant to be determined later. We choose a high-probability event $\Xi\equiv\Xi_{N}(\varepsilon,\tau)$ (see Definition 3.1) satisfying the following conditions.

for $i,j\in\mathcal{R}$, large enough $K$, and all $z$ in the set

For all $i$ satisfying $1+K^{-1/3}\leqslant d_{i}\leqslant\omega^{-1}$ we have

Note that such an event $\Xi$ exists. Indeed, (5.7) and (5.8) may be satisfied using Theorem 2.3, and (5.5) using Theorem 3.7 combined with Remark 3.3.

For the sequel we fix a realization $H\in\Xi$ satisfying the conditions (i)–(iii) above. Hence, the rest of the proof of Proposition 5.2 is entirely deterministic, and the randomness only enters in ensuring that $\Xi$ has high probability. Our starting point is a contour integral representation of the projection $P_{A}$. In order to construct the contour, we define for each $i\in A$ the radius

We define the contour $\Gamma\mathrel{\mathop{:}}=\partial\Upsilon$ as the boundary of the union of discs $\Upsilon\mathrel{\mathop{:}}=\bigcup_{i\in A}B_{\rho_{i}}(d_{i})$, where $B_{\rho}(d)$ is the open disc of radius $\rho$ around $d$. We shall sometimes need the decomposition $\Gamma=\bigcup_{i\in A}\Gamma_{i}$, where $\Gamma_{i}\mathrel{\mathop{:}}=\Gamma\cap\partial B_{\rho_{i}}(d_{i})$. See Figure 5.1 for an illustration of $\Gamma$.

We shall have to use the estimate (5.5) on the set $\overline{\theta(\Upsilon)}\!\,$. Its applicability is an immediate consequence of the following lemma.

The set $\overline{\theta(\Upsilon)}\!\,$ lies in (5.6).

It is easy to check that $\theta(\zeta)\leqslant\omega^{-1}$ for all $\zeta\in\Upsilon$. In order to check the lower bound on $\operatorname{Re}\theta(\zeta)$, we note that for any $\alpha\in(0,1)$ there exists a constant $c\equiv c(\alpha,\tau)$ such that

for $\operatorname{Re}\zeta\geqslant 1$, $\lvert\operatorname{Im}\zeta\rvert\leqslant\alpha(\operatorname{Re}\zeta-1)$, and $\lvert\zeta\rvert\leqslant\tau^{-1}$. Now the claim follows easily from $\operatorname{Re}\zeta\geqslant 1+K^{-1/3+\tau}/2$ for all $\zeta\in\Upsilon$, by choosing $\alpha=1/\sqrt{3}$. ∎

Each outlier $\{\mu_{i}\}_{i\in A}$ lies in $\theta(\Upsilon)$, and all other eigenvalues of $Q$ lie in the complement of $\overline{\theta(\Upsilon)}\!\,$.

It suffices to prove that (a) for each $i\in A$ we have $\mu_{i}\in\theta(B_{\rho_{i}}(d_{i}))$ and (b) all the other eigenvalues $\mu_{j}$ satisfy $\mu_{j}\notin\theta(B_{\rho_{i}}(d_{i}))$ for all $i\in A$.

for $i\in A$, as follows from (5.3) and (5.1). Using

it is then not hard to get (a) from (5.10) and (5.7).

In order to prove (b), we consider the two cases (i) $1+K^{-1/3}\leqslant d_{j}\leqslant\omega^{-1}$ with $j\notin A$, and (ii) and $j\geqslant s_{+}+1$. In the case (i), the claim (b) follows using (5.7), (5.11), and (5.3). In the case (ii), the claim (b) follows from (5.8) and the estimate

Using the spectral decomposition of $\widetilde{G}(z)$, Lemma 5.5, and the residue theorem, we may write the projection $P_{A}$ as

This is the desired integral representation of $P_{A}$.

We now perform a resolvent expansion on the denominator

where we used Cauchy’s theorem, (4.2), and the fact that $d_{i}$ lies in $\Upsilon$ if and only if $i\in A$.

using the fact that $f_{ij}$ is holomorphic inside $\Gamma$ and satisfies the bounds

The first bound of (5.21) follows from (5.5), (5.11), and (5.12). The second bound of (5.21) follows by plugging the first one into

where the contour $\mathcal{C}$ is the circle of radius $\lvert\zeta-1\rvert/2$ centred at $\zeta$. (By assumptions on $\varepsilon$ and $\omega$, the function $f_{ij}$ is holomorphic in a neighbourhood of the closed interior of $\mathcal{C}$.)

In order to estimate (5.20), we consider the three cases (i) $i,j\in A$, (ii) $i\in A$, $j\notin A$, (iii) $i\notin A$, $j\in A$. Note that (5.20) vanishes if $i,j\notin A$. We start with the case (i). Suppose first that $i\neq j$ and $d_{i}\neq d_{j}$. Then we find

A simple limiting argument shows that this bound is also valid for $d_{i}=d_{j}$ and $i=j$. Next, in the case (ii) we get from (5.21)

A similar estimate holds for the case (iii). Putting all three cases together, we find

What remains is the estimate of $S_{ij}^{(2)}$. Here residue calculations are unavailable, and the precise choice of the contour $\Gamma$ is crucial. We use the following basic estimate to control the integral.

For $k\in A$, $l\in\mathcal{R}$, and $\zeta\in\Gamma_{k}$ we have

The upper bound $\lvert\zeta-d_{l}\rvert\leqslant\rho_{k}+\lvert d_{k}-d_{l}\rvert$ is trivial, so that we only focus on the lower bound. Suppose first that $l\notin A$. Then we get $\lvert\zeta-d_{l}\rvert\geqslant\lvert d_{k}-d_{l}\rvert-\rho_{k}$, from which the claim follows since $\lvert d_{k}-d_{l}\rvert\geqslant 2\rho_{k}$ by (5.9).

For the remainder of the proof we may therefore suppose that $l\in A$. Define $\delta\mathrel{\mathop{:}}=\lvert d_{k}-d_{l}\rvert-\rho_{k}-\rho_{l}$, the distance between the discs $D_{\rho_{k}}(d_{k})$ and $D_{\rho_{l}}(d_{l})$ (see Figure 5.1). We consider the two cases $4\delta\leqslant\lvert d_{k}-d_{l}\rvert$ and $4\delta>\lvert d_{k}-d_{l}\rvert$ separately.

Suppose first that $4\delta\leqslant\lvert d_{k}-d_{l}\rvert$. Then by definition of $\delta$ we have $\lvert d_{k}-d_{l}\rvert\leqslant\frac{4}{3}(\rho_{k}+\rho_{l})$. Now a simple estimate using the definition of $\rho_{i}$ yields $\rho_{k}/5\leqslant\rho_{l}\leqslant 5\rho_{k}$, from which we conclude $\lvert d_{k}-d_{l}\rvert\leqslant 8\rho_{k}$. The claim now follows from the bound $\lvert\zeta-d_{l}\rvert\geqslant\rho_{l}$.

Suppose now that $4\delta>\lvert d_{k}-d_{l}\rvert$. Hence $\rho_{k}+\rho_{l}\leqslant\frac{3}{4}\lvert d_{k}-d_{l}\rvert$, so that in particular $\rho_{k}\leqslant\lvert d_{k}-d_{l}\rvert$. Thus we get

From (5.18), (5.5), (5.11), and (5.12) we get

where we also used the estimate $\lvert\theta(\zeta)\rvert\asymp\phi^{-1/2}(1+\phi)$ for $\zeta\in\Gamma$.

In order to estimate the matrix norm, we observe that for $\zeta\in\Gamma_{k}$ we have on the one hand

for any $l\in\mathcal{R}$, where in the last step we used (5.10). Since $\varepsilon<\delta$, these estimates combined with a resolvent expansion give the bound

for $\zeta\in\Gamma_{k}$. Decomposing the integration contour in (5.23) as $\Gamma=\bigcup_{k\in A}\Gamma_{k}$, and recalling that $\Gamma_{k}$ has length bounded by $2\pi\rho_{k}$, we get from Lemma 5.6

We estimate the right-hand side using Cauchy-Schwarz. For $i\notin A$ we find, using (5.9),

For $i\in A$ we use (5.9) and the estimate $\rho_{k}+\lvert d_{i}-d_{k}\rvert\geqslant\rho_{i}$ for all $k\in A$ to get

Recall that $M\asymp(1+\phi)K$. Hence, plugging (5.19), (5.22), and (5.27) into (5.15), we find

We have proved (5.28) under the assumption that $i,j\in\mathcal{R}$. The general case is an easy corollary. For general $i,j\in[\![{1,M}]\!]$, we define $\widehat{\mathcal{R}}\mathrel{\mathop{:}}=\mathcal{R}\cup\{i,j\}$ and consider

where $\widehat{d}_{k}\mathrel{\mathop{:}}=d_{k}$ for $k\in\mathcal{R}$ and $\widehat{d}_{k}\in(0,1/2)$ for $k\in\widehat{\mathcal{R}}\setminus\mathcal{R}$. Since $\lvert\widehat{\mathcal{R}}\rvert\leqslant r+2$ and $\widehat{D}$ is invertible, we may apply the result (5.28) to this modified model. Now taking the limit $\widehat{d}_{k}\to 0$ for $k\in\widehat{\mathcal{R}}\setminus\mathcal{R}$ in (5.28) concludes the proof in the general case. Now Proposition 5.2 follows since $\varepsilon$ may be chosen arbitrarily small. This concludes the proof of Proposition 5.2.

2. Removing the non-overlapping assumption

In this subsection we complete the proof of Proposition 5.1 by extending Proposition 5.2 to the case where (5.3) does not hold.

Let $\delta<\tau/4$. We say that $i,j\in\mathcal{O}_{\tau/2}^{+}$ overlap if $\lvert d_{i}-d_{j}\rvert\leqslant(d_{i}-1)^{-1/2}K^{-1/2+\delta}$ or $\lvert d_{i}-d_{j}\rvert\leqslant(d_{j}-1)^{-1/2}K^{-1/2+\delta}$. For $A\subset\mathcal{O}_{\tau}^{+}$ we introduce sets $S(A),L(A)\subset\mathcal{O}_{\tau/2}^{+}$ satisfying $S(A)\subset A\subset L(A)$. Informally, $S(A)\subset A$ is the largest subset of indices of $A$ that do not overlap with its complement. It is by definition constructed by successively choosing $k\in A$, such that $k$ overlaps with an index of $A^{c}$, and removing $k$ from $A$; this process is repeated until no such $k$ exists. One can check that the result is independent of the choice of $k$ at each step. Note that $S(A)$ may be empty.