Random matrices: Universal properties of eigenvectors
Terence Tao, Van Vu
Introduction
Consider a random Hermitian matrix with real eigenvalues (counting multiplicity)
By the spectral theorem, one can find an orthonormal basis
Unfortunately, the eigenvectors are not unique in either the Hermitian or real symmetric cases; even if one assumes that the spectrum of is simple, in the sense that
one has the freedom to rotate each by a unitWe use to denote the imaginary unit, in order to free up the symbol as an index variable. phase . In the real symmetric case, in which we force the eigenvectors to have real coefficients, one only has the freedom to multiply each by a sign . However, one can eliminate this phase ambiguity or sign ambiguity (in the case of simple spectrum) by a adopting a variety of viewpoints:
One can consider the rank one projection operators
instead of the eigenvectors , thus the coefficient of is given by the formula
One can perform the ad hoc normalization of requiring to be positive real, where is the first index for which (generically we will have , and as we will see shortly, for Wigner matrices we will also have with high probability).
One can perform the random normalization of replacing with a randomly chosen rotation (in the Hermitian case) or (in the real symmetric case) (each the random phase or sign being chosen independently of each other, and (if is itself random) of ).
Note that , , the ad hoc normalized , and the randomly normalized in viewpoints (i)-(iv) respectively will be uniquely defined as long as the spectrum is simple (indeed, it suffices to have ). In the proofs of our main results, we shall adopt viewpoint (i) (which is natural from the perspective of spectral theory). However, in order to express our results in explicit coordinates, we will adopt the more ad hoc viewpoint (iii) or the random viewpoint (iv) in the statements of our results.
Notations. We consider as an asymptotic parameter tending to infinity. We use , , , or to denote the bound for all sufficiently large and for some constant . Notations such as mean that the hidden constant depend on another constant . or means that as ; the rate of decay here will be allowed to depend on other parameters.
We will need some definitions that capture the intuition that a certain event occurs very frequently.
holds asymptotically almost surely if .
holds with high probability if for some constant .
holds with overwhelming probability if for every constant (or equivalently, that ).
holds almost surely if .
The goal of this paper is to understand the distribution of the eigenvectors (as normalized using viewpoint (iii) or (iv), for sake of concreteness) of a class of random matrix ensembles known as Wigner random matrices.
Let us first identify the class of matrices we are working with.
Let be an integer (which we view as a parameter going off to infinity). An Wigner Hermitian matrix is defined to be a random Hermitian matrix , in which the for are jointly independent with . For , we require that the have mean zero and variance one, while for we require that the (which are necessarily real) have mean zero and variance for some independent of . (Note that we do not require the to be identically distributed, either on or off the diagonal.)
We say that the Wigner matrix ensemble obeys condition C0 if we have the exponential decay condition
for all and , and some constants (independent of ). We say that the Wigner matrix ensemble obeys condition C1 with constant if one has
for some constant (independent of ).
Of course, Condition C0 implies Condition C1 for any , but not conversely.
The distribution of coefficients of a matrix distributed using the Haar measure on the unitary or orthogonal groups has been studied by many authors , , , , , , , , . It is known (see ) that each coefficient, after multiplication by , is asymptotically complex normal (in the unitary case) or real normal (in the orthogonal case). In fact the same is true for the joint distribution of multiple coefficients:
If instead we make the weaker assumption that , then it is possible to couple together and such that converges to zero in probability.
In it is also shown that the hypotheses , in the above two results are best possible. Of course, by symmetry, one can replace the top left minor of the orthogonal matrix with any other minor and obtain the same results.
As a corollary of Theorem 3, we obtain an asymptotic for the distribution of eigenvector coefficients of GUE or GOE (normalizing using viewpoint (iii)).
If , then and differ by in variation norm.
If instead we make the weaker assumption that , then it is possible to couple together and such that converges to zero in probability.
The main objective of this paper is to develop analogues of Corollary 4 for the more general Wigner ensembles from Definition 2. In particular, we would like to consider ensembles which are allowed to be discrete instead of continuous.
One immediate difficulty that arises in the discrete setting is that one can now have a non-zero probability that the spectrum is non-simple. However, we have the following gap theorem from , , :
Suppose that is a Wigner random matrix obeying Condition C1 with a sufficiently large constant , and let be independent of . Write for the rescaled matrix. Then for any , one has
for all sufficiently large , where depends only on .
In the bulk case assuming Condition C0, see [17, Theorem 19]. For the extension to the edge case, see [18, Theorem 1.14]. For the relaxation of Condition C0 to Condition C1 (with a sufficiently large ), see [19, Section 2]. See also for some related level repulsion estimates (assuming some additional regularity hypotheses on ). ∎
Of course, one has . From the above theorem and the union bound, we see that there is an absolute constant such that if and , and obeys Condition C1 with a sufficiently large , then with probability , the eigenvalues will all occur with multiplicity one, so that one can meaningfully normalise the eigenvectors according to any of the viewpoints (i), (ii), (iii), (iv) mentioned previously, outside of an exceptional event of probability . On that exceptional event, we define the orthonormal eigenvector basis (and related objects such as the rank one projection ) in some arbitrary (measurable) fashion.
We now turn to the distribution of the coefficients of the eigenvectors. As the ensembles may be discrete, the eigenvector coefficients may be discrete also, and so one does not expect to have convergence in variation distance any more. Instead, we will consider the weaker notion of vague convergence, which resembles the condition (1), but with now required to be compactly supported, and either continuous or smooth. We need the following definition:
Let . Two Wigner random matrices and are said to match to order off the diagonal, and match to order on the diagonal, if one has whenever and are integers such that (if ) or (if ).
We can now give our first main result, which partially extends Corollary 4 to other Wigner ensembles:
(Vague convergence) If , then one has
If instead we make the stronger assumption that , then it is possible to couple together and such that converges to zero in probability. In particular, this implies that converges to in distribution.
If one uses the random normalization (iv) instead of (iii), then the conclusions are the same, except that the absolute values are not present in the definition of .
The bound in (ii) can be extended by our method (with some effort) to , but this still falls far short of the analogous range in Corollary 4. It is reasonable to expect that these bounds are not best possible (particularly if one places some additional regularity hypotheses on the coefficients of ).
We will deduce Theorem 7 from Corollary 4 by establishing a four moment theorem for eigenvectors, which is the main technical result of this paper:
The bounds are uniform in the choice of .
The deduction of Theorem 7 from Corollary 4 and Theorem 8 is routine and is performed in Section 2.
Theorem 8 is an extension of the four moment theorem for eigenvalues established in , , . Indeed, the latter theorem is essentially the special case of Theorem 8 in which , and only depends on the first components of . Unsurprisingly, the proof of Theorem 8 will rely heavily on the machinery developed in , , .
Theorem 8 was announced at the AIM workshop “Random matrices” in December 2010. We have found out that recently a result in the same spirit has been proved by Knowles and Yin using a somewhat different method. Knowles and Yin handled the case with Condition C1 replaced by the stronger Condition C0. Furthermore, they needeed control on derivatives of rather than just derivatives, and also a level repulsion hypothesis similar to (6), (7) below. On the other hand, their result holds for generalized Wigner matrices.
The need for four matching moments in Theorem 8 is believed to be necessary; see . However, we conjecture that Corollary 4 continues to hold without the matching moment hypothesis.
We can use Theorem 8 (together with other tools) to obtain a four moment theorem for the resolvent (or Green’s function) coefficients
where is the rescaled matrix (so in particular and ).
for some constant independent of .
We isolate the case of this theorem as a corollary:
Under the conditions of Theorem 9, we have
We prove Theorem 9 in Section 5; it is established by combining Theorem 8 with an eigenvalue rigidity result from and a local semicircle law from . This result generalizes a similar four moment theorem from ( ( [9, Theorem 2.3]). Its main strengths as compared against that result are that is allowed to go all the way to zero. In particular, one can take to be zero, thus giving control of the coefficients of the inverse matrix . On the other hand, the result in [9, Theorem 2.3] does not require the hypothesis (6), (7), and allows the entries in (or ) to have different variances, and the bounds are slightly sharper.
The same proof allows one to control the joint distribution of coefficients of several resolvents with for some sufficiently small , assuming a level repulsion estimate at each energy .
The hypotheses (6), (7) are natural, as when these claims fail one would expect the resolvent to be unusually large. However, in practice such hypotheses are in fact automatic and can thus be omitted. For instance, we have
If the real and imaginary parts of the off-diagonal coefficients of are iid and supported on at least three points, and obeys Condition C1for a sufficiently large , then (6) holds.
If and obeys Condition C1for , then (6) holds.
For (i), see ; an earlier result assuming smoothness and decay on the coefficients (but without the requirement of iid real and imaginary parts) was established in (see also for a more refined result). The claim (ii) was recently established (by a rather different method) in ∎
It is in fact likely that (6) and (7) in fact always hold whenever Condition C1is satisfied for a sufficiently large , but we do not attempt to establish this fact here.
As another application, we can use our new results to obtain central limit theorems concerning eigenvectors. Here is a sample result.
is normalized using the procedure (iii), and ; or
is normalized using the procedure (iv).
As an example to illustrate Theorem 13, we can take , and . Then Theorem 13 asserts that the sum of the entries of the middle eigenvector (using either normalization (iii) or normalization (iv)) is gaussian in the limit.
We prove Theorem 13 in Section 4 as a consequence of Corollary 4 and a general central limit theorem on averages of approximately independent symmetric random variables (Proposition 25) which may be of independent interest. It should be possible to extend this result to more general ensembles (and to obtain the analogous results for ensembles that match GUE rather than GOE), and to obtain central limit theorems for the joint distribution of several statistics of the form , but we do not pursue this matter here.
The four moment theorem can be extended to handle the singular values of iid covariance matrices, instead of the eigenvalues of Wigner matrices; see . It is possible to use that extension to establish an analogue of Theorem 8 for the singular values and singular vectors of such matrices, and an analogue of Theorem 9 for the inverses of such matrices (which, as is well known, can be expressed in terms of the singular value decomposition of the matrix). We omit the details.
Proof of Theorem 7
We establish the claim in the GOE case only, as the GUE case is similar. We shall also establish the claim just for the normalization (iii), as the normalization (iv) can be treated similarly (or deduced directly from the results for (iii)).
Let be as in Theorem 8, let be sufficiently small, and let , , , , , and be as in that theorem. Let be drawn from GOE, and write and . We initially assume that . By adding a dummy index and relabeling if necessary, we may assume without loss of generality that .
and thus by Corollary 4, we have an analogous estimate for the quantities :
Applying Theorem 8 (assuming that is sufficiently small depending on , so that the losses of coming from bounding the derivatives of can be absorbed into the factor) we conclude that
Now we can prove part (i) of Theorem 7. From (8) one has
Thus, to prove (2), it suffices by the triangle inequality to show that
with the understanding that the right-hand side vanishes when any of the vanish. Similarly for . From construction we can easily verify that
for all . The claim (i) now follows from Theorem 8 (assuming sufficiently small depending on ).
Finally, we prove Claim (ii) of Theorem 7. Assume that . Let be a slowly decaying function of to be chosen later. Observe that asymptotically almost surely, one has
and thus by (i), we conclude that asymptotically almost surely, we also have
Write . Applying Claim (i) (approximating the indicator functions from above and below by smooth functions) and using the continuous nature of , we see that
if is sufficiently slowly decaying in . From this, we see that we may couple and together in such a fashion that with probability , and lie in the same cube , which in particular implies that . The claim follows.
Proof of four moment theorem
In this section we prove Theorem 8. We will follow closely the arguments from , , .
Let us first review the general strategy from for handling the eigenvalues. As in , we introduce the normalised matrices
(whose mean eigenvalue spacing is comparable to ). For sake of exposition let us restrict attention to the case , thus we wish to show that the expectation of the random variable only changes by if one replaces with another random matrix with moments matching up to fourth order off the diagonal (and up to second order on the diagonal). To further simplify the exposition, let us suppose that the coefficients of (or ) are real-valued rather than complex-valued.
Let us freeze (or condition on) all the entries of except for the and entries. For any complex number , let denote the matrix which equals except at the , , entries, where it equals and respectively. (Actually, with our hypotheses, we only need to consider real-valued .) Thus it would suffice to show that
for all (or at least most) choices of the frozen entries of , where . (A standard argument allows us to restrict attention to values of of size .)
Suppose we could show the derivative estimates
for . Then by Taylor’s theorem with remainder, we would have
and so in particular (using the hypothesis )
and similarly for . Since for and large enough and small enough, we thus obtain the claim (10) thanks to the hypothesis that the first four moments of and match. (Note how this argument barely fails if only three moments are assumed to match, though it is possible that some refinement of this argument might still succeed by exploiting further cancellations in the fourth order term .)
We can use exactly this strategy for eigenvectors, replacing by (say). A key point here is that the derivatives of is computed based on the basic relation and the chain rule. Thus, in order to bound the derivatives of (for the four moment theorem for eigenvalues), we needed to obtain estimates for the derivatives of both and . The same estimates can be used to bound the derivatives of in the proof of the four moment theorem for eigenvectors. In order to make as large as , one needs to follow the proof closely and make certain adjustments.
2. Formal proof
We now turn to the details. The first step is to truncate away the event that an eigenvalue gap is unexpectedly small. Define
This quantity is usually bounded from above:
with high probability for all .
See [17, Lemma 49]. Strictly speaking, this lemma assumed Condition C0 and was restricted to the bulk of the spectrum, but this was solely because at the time of writing of that paper, the gap theorem (Theorem 5) was only established in that setting. Inserting Theorem 5 as a replacement for [17, Theorem 19] in the proof of [17, Lemma 49], we obtain the claim. ∎
In view of this lemma (and its obvious counterpart for ), we can (if for a sufficiently small ) deduce the four moment theorem from the following truncated version:
Then for any and , and for sufficiently large depending on (and the constant in Definition 2) we have
The reduction of Theorem 8 to Theorem 17 using Lemma 16 proceeds exactly as in [17, §3.3] and is omittedNote that now that is as large as , some factors of may be lost in the bounds, but this can be absorbed by the gain in the conclusion of Theorem 17..
As in , we adopt the Lindeberg strategy of swapping each matrix entry of (and its transpose) one at a time. A key definition is that of a good configuration, which is a slightly modified version of the same concept from . Let be parameters (independent of ) to be selected later. For a given , we fix as in Theorem 17.
For a complex parameter , let be a (deterministic) family of Hermitian matrices of the form
where are unit vectors. We say that is a good configuration if for every and every whose real and imaginary parts are multiples of , we have the following properties:
(Eigenvalue separation) For any with , we have
(Delocalization) There exists an orthonormal eigenfunction basis such that
To show Theorem 17, it then suffices to establish the following two claims. The first claim, which we shall establish shortly, is an analogue of [17, Proposition 46], and involves a single (deterministic) good configuration :
Suppose that is sufficiently large, and let . Let be a good configuration. Then one has
whenever are random complex variables that match to order for some , and bounded almost surely by .
The second claim is an analogue of [17, Proposition 48]:
Let and , and assume that is sufficiently large depending on . Let be a random Hermitian matrix with independent upper-triangular entries and for all , with , but with having mean zero and variance for all other , and also being distributed continuously in the complex plane. Then is a good configuration with overwhelming probability.
This is almost identical to the argumentsThe hypotheses in [17, §5] did not have the loss of in the bound for , but such bounds only cause losses of in the final bounds, which can be absorbed into the factors in the definition of a good configuration if is sufficiently large depending on . in [17, §5]. Those arguments already give (14) with overwhelming probability. To obtain the delocalization of eigenvectors, one can use [18, Proposition 1.12] (see also the proof of [17, Corollary 63] to deal with the fact that the and entries are not random). ∎
With these two propositions, we can establish Theorem 17 by first using Condition C1 to truncate to the case when have entries of size (say), and perturbing them to be continuous, and then replacing the entries of with one at a time just as in [17, §3.3]. Because the entries of and match to order off the diagonal and to order on the diagonal, each of the off-diagonal replacements costs an error of , while each of the diagonal replacements costs an error of , and so the net error is acceptable if are sufficiently small (and if is large enough that Proposition 20 applies).
It remains to establish Proposition 19. As in , we will need derivative bounds on various spectral statistics of . For each , we introduce the resolvent-type matrices
Let , and let be a Hermitian matrix which has a simple eigenvalue at . Then , , , and depend smoothly on in a neighborhood of .
Now we turn to more quantitative bounds on derivatives. If is a scalar, vector, or matrix-valued function depending smoothly (but not holomorphically) on a complex parameter , we define the derivatives of to be the vector (or tensor)-valued quantity
Let be an Hermitian matrix varying (real)-linearly in (thus for ), with
for some . Let . At some fixed value of , suppose we have the spectral gap condition
for all and some (in particular, is a simple eigenvalue). Then for all we have (at this fixed choice of )
In practice, this crude bound is insufficient, and we will need the following more advanced bound:
Let be an matrix depending on a complex parameter of the form
for some vectors . We abbreviaate , , etc.
Let . At some fixed value of , suppose that is a simple eigenvalue, and that we have a partition
where is a finite index set, and are orthogonal projections to invariant spaces on (i.e. to spans of eigenvectors not corresponding to ). Suppose that on the range of each , the eigenvalues of have magnitude at least for some . Suppose also that we have the incompressibility bounds
for all and some and , with . Then at this value of , and for all , we have the bounds
for all and all at this value of . Here is the Frobenius norm of .
See [17, Corollary 58], which is a special case of [17, Lemma 57]. The bounds (22), (23), (24) do not appear explicitly in [17, Corollary 58], but come directly from the equations [17, (73), (74), (75)] in [17, Lemma 57] after plugging in the parameters indicated in the proof of [17, Corollary 58]. ∎
We can now prove Proposition 19 and thus Theorem 8. This will be a repetition of the material in [17, Section 4.3]; we sketch the main points here.
Fix , and , and suppose that is sufficiently large. We assume are as in the proposition.
We may of course assume that for at least one with , since the claim is vacuous otherwise.
Using Taylor expansion and the chain rule exactly as in [17, Section 4.3], it suffices to show that
Suppose that for at least one with . Then for all with , and all , we have
for all with and all .
The bounds (26), (28) were already proven in [17, Lemma 59]; the arguments in the proof of that lemma also show that
uniformly for all with .
Now we prove (27). Arguing inductively as in the proof of [17, Lemma 59], it suffices to establish the claim for in the ball . Let us first establish this for whose real and imaginary parts are a multiple of . At this value of , we can apply Proposition 23 exactly as in [17, Section 4.3] to obtain the bounds
for all , , and , where is the minimal value of for , and is the spectral projection to those eigenvalues with .
for all , where we adopt the convention that . Now, we expand
Applying the triangle and Cauchy-Schwarz inequalities we conclude that
From the hypothesis (15) and Pythagoras’ theorem we have
so from (32) we obtain (27) as required, at least when the real and imaginary parts of are multiples of . The general case can then be handled (for large enough) by an appeal to Lemma 22 by arguing exactly as in [17, Section 4.3].
Proof of Theorem 13
We now prove Theorem 13. The main tool is the following general central limit theorem:
(Exchangeability) The distribution of is symmetric with respect to the permutation group .
For any distinct with fixed, converges jointly in distribution to iid copies of as .
(Symmetry) The distribution of is symmetric with respect to the reflection group .
We now prove Proposition 25. Let be a quantity growing slowly to infinity that we will choose later. We truncate
where and . We then split correspondingly. It will suffice to show that, for a suitable choice of ,
converges in probability to zero.
We first consider the second claim. Here we use the second moment method. It suffices to show that
Set . The left-hand side can be expanded as
Using the symmetry hypothesis (iv), we see that if , then has a symmetric distribution and thus has mean zero. Thus only the diagonal terms contribute. Using hypothesis (i) and the unit normalization of , the second moment becomes
for any fixed , where , and thus (since goes to infinity)
Now we turn to Claim 1. Here we use the moment method. By Carleman’s theorem (see e.g. ), it suffices to show that for each fixed positive integer ,
By the symmetry hypothesis (iii), the expectation vanishes unless each index appears an even number of times. Using hypothesis (ii) (and (iii)), we see that
where are iid copies of , uniformly in , if grows sufficiently slowly to infinity. Observe that
since , and that (as before) the summands vanish unless each index appears an even number of times. Thus it suffices to show that
where the sum is over all -tuples in which each index appears an even number of times, and the implied constants in the notation are allowed to depend on .
Suppose there are distinct indices appearing, then the contribution of this case is at most
(since we have whenever is a positive even number). But as is a unit vector, this sums to as required. This concludes the proof of Proposition 25 and hence Theorem 13.
Proof of Theorem 9
We now prove Theorem 9. Let be as in that theorem. Let be a small constant to be chosen later, and let be an even smaller constant (depending on ) to be chosen later. Write and .
Let us call an expression depending on (or ) stable if it only changes by for some if is replaced by . Our task is thus to show that is stable.
We first subtract off the “global” portion of the resolvent . Set . Applying the local semicircle law from [9, Theorem 2.1], one hasStrictly speaking, the statement of [9, Theorem 2.1] only controls the diagonal component of the resolvent. However, an inspection of the proof of that theorem (see in particular [9, (3.13)] and [9, Proposition 3.3]) reveals that the off-diagonal components were also controlled by the argument.
with overwhelming probability for some , where is the Kronecker delta function and is semicircular Stieltjes transform
This type of result already gives the claim when , so we may assume that . After shifting by , and using the regularity bounds on , it thus suffices to show that the expression
Note that decays quadratically in rather than linearly, due to the subtraction of the comparison term ; this will allow us to easily neglect the contribution of the spectrum that is far from in the rest of the argument.
We now invoke the eigenvalue rigidity result from [10, Theorem 2.2]. Among other things, this theorem gives an interval of length (depending only on , , and ) which contains most of the eigenvalues close to , in the sense that we have
with overwhelming probability. In fact, we have the stronger assertion that
for all with overwhelming probability. In particular, this implies that
Also, by the delocalization of eigenvalues (established in the bulk in [7, Theorem 4.8] (see also the earlier result [6, Theorem 1.2] handling the smooth case), and up to the edge in [18, Proposition 1.12]), we have
with overwhelming probability for all , and hence
with overwhelming probability for all . As such, we see that with overwhelming probability, we have
with overwhelming probability. Using the regularity bounds on , it thus suffices to show that the quantity
of . From the hypothesis (6), we see that for each , one has
with probability at least (say), if is sufficiently small depending on and on the implied constant in the high probability event (6). In particular, by the union bound, we have
Similarly for using (7). It thus suffices to show that the quantity
is stable. But this follows directly from Theorem 8 (if are small enough). The proof of Theorem 9 is now complete.
The exponential decay hypothesis (Condition C0) in Theorem 9 is needed only to be able to use the eigenvalue rigidity result from and the local semicircle law from . It is quite likely that in both cases, one can relax Condition C0 to Condition C1 (conceding some factors of in the process), which would then allow one to achieve a similar relaxation in Theorem 9.