Invertibility and Robustness of Phaseless Reconstruction
Radu Balan, Yang Wang
Introduction
This paper is concerned with the question of reconstructing a vector in a finite-dimensional real Hilbert space of dimension when only the magnitudes of the coefficients of the vector under a redundant linear map are known.
Specifically our problem is to reconstruct up to a global phase factor from the magnitudes where is a frame (complete system) for .
A previous paper described the importance of this problem to signal processing, in particular to the analysis of speech. Of particular interest is the case when the coefficients are obtained from a Windowed Fourier Transform (also known as Short-Time Fourier Transform), or an Undecimated Wavelet Transform (in audio and image signal processing). A similar problem appears in Quantum Information (QI) literature (see e.g. ). However some important differences are notable: first the unknown objects to be reconstructed are quantum states (meaning nonnegative symmetric operators of unit trace); secondly, measurements are performed by taking Hilbert-Schmidt inner products against some (nonnegative) symmetric operators of rank not necessarily one. In QI language our problem is to reconstruct rank one nonnegative symmetric operators from measurements against a set of rank one nonnegative symmetric operators.
While presents some necessary and sufficient conditions for reconstruction, the general problem of finding fast/efficient algorithms is still open. In we describe one solution in the case of STFT coefficients.
For vectors in real Hilbert spaces, the reconstruction problem is easily shown to be equivalent to a combinatorial problem. In this problem is further proved to be equivalent to a (nonconvex) optimization problem.
A different approach (which we called the algebraic approach) was proposed in . While it applies to both real and complex cases, noisless and noisy cases, the approach requires solving a linear system of size exponentially in space dimension. This algebraic approach generalizes the approach in where reconstruction is performed with complexity (plus computation of the principal eigenvector for a matrix of size ). However this method requires frame vectors.
Recently the authors of developed a convex optimization algorithm (a SemiDefinite Program called PhaseLift) and proved its ability to perform exact reconstruction in the absence of noise, as well as its stability under noise conditions. In a separate paper , the authors further developed a similar algorithm in the case of windowed DFT transforms. Inspired by the PhaseLift and MaxCut algorithms, but operating in the coefficients space, the authors of proposed a SemiDefinite Program called PhaseCut. They show the algorithm yields the exact solution in the absence of noise under similar conditions as PhaseLift.
The paper presents an iterative regularized least-square algorithm for inverting the nonlinear map and compares its performance to a Cramer-Rao lower bound for this problem in the real case. The paper also presents some new injectivity results which are incorporated into this paper.
A different approach is proposed in . There the authors use a 4-term polarization identity together with a family of spectral expander graphs to design a frame of bounded redundancy () that yields an exact reconstruction algorithm in the absence of noise.
The authors of study several robustness bounds to the phase recovery problem in the real case. However their approach is different than ours in several respects. First they consider a probabilistic setup of this problem, where data and frame vectors ’s are random vectors with probabilities from a class of subgaussian distributions. Additionally, their focus is on classes of -sparse signals. Their results show that, with high probability, recovery is possible from a number of measurements that has a similar asymptotic behavior with respect to and as in the case of linear measurements (that is with the phase). In our paper we analyze stability bounds of reconstruction for a fixed frame using deterministic analytic tools. After that we present asymptotic behavior of these bounds for random frames.
Finally, the authors of analyze the phaseless reconstruction problem for both the real and complex case. In the real case the authors obtain the exact upper Lipschitz constant for the nonlinear map , namely where is the upper frame bound. For the lower Lipschitz constant, they give an estimate between two computable singular eienvalues. Our results have overlaps with their results somewhat here. However, in this paper we improve the improve the lower Lipschitz constant by giving its exact value. There are some significant differences between this paper and . In addition to studying of the Lipschitz property of the map we focus also on two related but different settings. First we study the robustness of the reconstruction given a fixed error allowance in measurements. Second we also consider the Lipschitz property of the map . The authors of point out that the map is not bi-Lipschitz. However in our paper we show becomes bi-Lipschitz for a different metric on the domain. With this metric (the one induced by the nuclear norm on the set of symmetric operators) the nonlinear map is bi-Lipschitz with constants indicated in Theorem 4.5. Furthermore the same conclusion holds true in the complex case, although this will be studied elsewhere.
The organization of the paper is as follows. Section 2 formally defines the problem and reviews existing inversion results in the real case. Section 3 establishes information theoretic performance bounds, namely the Cramer-Rao lower bound. Section 4 contains robustness measures of any reconstruction algorithm. Section 5 presents a stochastic analysis of these bounds, and is followed by references.
Background
When we can choose the frame is said tight. For the frame is called Parseval. The frame matrix corresponding to is defined as with the vectors as its columns. We shall frequently identify with its corresponding frame matrix . The largest and smallest in (2.1) are called the lower frame bound and upper frame bound of , and they are given by
Thus the phaseless reconstruction problems aims to reconstruct from the map . We say a frame is phase retrievable if one can reconstruct for all , or in other words, is injective on . The main objective of this paper is to analyze robustness and stability of the inversion map, and to give performance bounds of any reconstruction algorithm.
If is phase retrievable in then . Furthermore, for a generic with the map is phase retrievable in .
Let . Then is phase retrievable in if an only if is full spark.
Then is phase retrievable in if and only if .
Proof. The results (1)-(3) are in , and (4)-(5) are in .
Information Theoretic Performance Bounds
In this section we derive expressions for the Fisher Information Matrix and obtain performance bounds for reconstruction algorithms in the noisy case.
Consider the following noisy measurement process:
where the noise model is AWGN (additive white Gaussian noise): each random variable is independent and normally distributed with zero mean and variance.
Under these assumptions we compute the Fisher Information matrix (see ). This is given by
where the likelihood function is given by
Note the matrix is exactly the same as the matrix introduced in (2.6). Thus we obtain the following results:
This allows to state the following performance bound result (see for details on the Cramer-Rao lower bound).
Furthermore, any efficient estimator (that is, any unbiased estimator that achieves the Cramer-Rao Lower Bound (3.6)) has the covariance matrix bounded from above by
Robustness Measures for Reconstruction
In this section we analyze the robustness of deterministic phaseless reconstruction. Additionally we connect the constant introduced earlier in Theorem 2.1 to quantities directly computable from the frame .
The size of measures the worst case stability of the reconstruction for the vector , under the assumption that the total noise level is controlled by . We also study the global stability by analyzing the measures
Here denotes usual Euclidian norm. Note that has the scaling property for any real . Thus it is natural to focus on unit vectors .
We introduce now some quantities that play key roles in the estimation of these robustness measures. For the frame let be its frame matrix. Denote by the subset of indexed by a subset , and by the frame matrix corresponding to (which is the matrix with vectors in as its columns). Set
All of them depend of course on . However since we fix throughout the paper, we shall without confusion not explicitly reference in the notation for simplicity as there will not be any confusion. Clearly
Let . Then the stability measurement function is given by
under the constraints and
where .
Let be the frame matrix of . We thus have
Note that . The proposition now follows.
The above proposition allows us to establish the following stability result for the worst case scenario.
Assume that the frame is phase retrievable. Let be the lower frame bound for the frame and let .
If then . Consequently .
The upper bound equals the reciprocal of :
under the constraints and
for some . Now assume without loss of generality that . Then
Thus .
Thus . Now let
It follows that . Now by taking sufficiently small we have .
This is a contradiction. So and hence
Thus . Proposition 4.1 now yields , proving part (B).
Now we prove (C). We go back to the formulation in Proposition 4.1.
under the constraints and
where . Since is injective, either or by Theorem 2.1 (1). Without loss of generality we assume . Thus \varepsilon\geq\|F_{S}^{*}w_{1}\bigr{\|}\geq\tau\|w_{1}\|. So . We show that for any we must have . Assume otherwise and write , . Then
This is a contradiction. Thus for we have and
Thus and hence . Now we show the bound can be achieved. Let satisfy . Such a always exists. Then clearly and satisfy the required constraints, and it is easy to check that .
Finally we prove (D). By the result at part (A), . It is therefore sufficient to shoow that for some and . Let be the subset that achieves the minimum in (4.4). Let be unit eigenvectors corresponding to the lowest eigenvalues of and respectively. Thus
Let and , and set , . Then by Proposition 4.1
Remark. It may seem strange that for all and sufficiently small while , where is typically much smaller than . The reason is that for to hold, depends on . Thus we cannot exchange the order of and .
We first investigate the bounds for . For this the upper bound is relatively straightforward. Let and . We have already shown in the proof of Theorem 4.2 using (4.9) that
To study the lower bound we now consider the following quantities:
where again and . Now fix and let . Without loss of generality we may assume . Thus and . Let and set
Note for any with and we have
Hence for all whenever . It follows that
Thus where is the lower frame bound for the frame . Furthermore this lower bound is achieved whenever is an eigenvector corresponding to the smallest eigenvalue of . This implies that
whenever . Consequently . We have the following theorem:
Assume that . Then . Consequently .
.
The map is bi-Lipschitz with (optimal) upper Lipschitz bound and lower Lipschitz bound :
Proof. We have already proved (1) and (2) of the theorem in the discussion. It remains only to prove (3) since (4) is just a restatement of (1) and (3). Note that
For any , assume without loss of generality that . Let . Set , and . Then
Let and be normalized (eigen) vectors that achieve the bound , that is:
Remark. The two quantities, and satisfy . However there are subtle differences between and so that the simple relationship does not usually hold.
Remark. The upper Lipschitz bound has been obtained independently in . The lower Lipschitz bound we obtained here strenghtens the estimates given in . Specifically their estimate for reads where
Clearly .
We conclude this section by turning our attention to the analysis of . A motivation for studying it is that in practical problems the noise is often added directly to as in (3.1) rather than to . Such noise model is used in many studies of phaseless reconstruction, e.g. in the Phaselift algorithm , or in the IRLS algorithm in .
The following lemma will be useful in this analysis
Furthermore, for its nuclear norm is .
(B) (C) is proved directly by setting and .
We now prove (C) (A) by computing the eigenvalues of . Obviously . Let be the two (possibly) nonzero eigenvalues of . Then
Hence, by Cauchy-Schwartz inequality, which proves . Furthermore, it also shows that the nuclear norm of is .
Now we analyze . Parallel to the study of we consider the following quantities:
as well as the upper bound . By (4.15) we have where and . Hence
Set and and apply Lemma 4.4 we obtain
We can immediately obtain the upper bound:
where was defined in (2.6). An immediate bound is with the upper frame bound of .
Fix and let . Then either or . Without loss of generality we assume that . Thus and . However can be any unit vector. Thus
where was introduced in (2.6). Thus we obtain
where was introduced in (2.4). Thus we proved:
Assume the frame is phase retrievable. Then
Furthermore is bi-Lipschitz with upper Lipschitz bound and lower Lipschitz bound :
Remark. Note that the distance is not equivalent to . Theorem 4.5 now also implies that is not bi-Lipschitz with respect to the distance on . This fact was pointed out in .
Robustness and Size of Redundancy
Previous sections establish results on the robustness of phaseless reconstruction for the worst case scenario. A natural question is to ask: can “reasonable” robustness be achieved for a given frame, and in particular with small number of samples? We shall examine how scales as the dimension increases.
Consider the case where . This is the minimal redundancy required for phaseless reconstruction. In this case any frame would have . Hence we have . The stability of the reconstruction is thus mostly controlled by the size of . The question is: how big is , especially as increases?
It follows that , and hence
Note that here we have considered only the first vectors of the frame . The actual value of will likely decay much faster as increases. In a preliminary work we are able to establish the bound where is independent of . But even this estimate is likely far from optimal.
Let and for all . Then there exist constants and independent of such that
A related problem is as follows: Consider an matrix . Let . Assume that all . How large can be? For we have already seen that it is bounded from above by . The preliminary work shows that for it is bounded from above by .
There exists a constant such that
Thus in the minimal setting with it is impossible to achieve scale independent stability for phaseless reconstruction. The same arguments can be used to show that even when for some fixed scale independent stability is not possible. A natural question is whether scale independent stability is possible when we increase the redundancy of the frame. As it turns out this is possible via a recent work by Wang . More precisely, the following result follows from the main results in :
Let and let where is an random matrix whose elements are i.i.d. normal random variables such that . Then there exist constants dependent only on and not on such that with high probability we have
Proof. Part of the main theorem of proves the following result: Let be fixed. Assume that where is an random Gaussian matrix with i.i.d entries such that . Then there exists a constant depending continuously and only on and such that
The theorem now readily follows. Observe that because , in the expression for we may choose and clearly we have
for some independent of . For we may choose and . Again the theorem of implies that
In the theorem the values and can be estimated explicitly. Here with high probability is in the standard sense that the probability is at least for some . Thus scale independent stable phaseless reconstruction is possible whenever the redundancy is greater than , , at least for random Gaussian matrices.
Acknowledgement. The authors would like to thank Matt Fickus, Dustin Mixon and Jeffrey Schenker for very helpful discussions.