Approximating the Expansion Profile and Almost Optimal Local Graph Clustering

Introduction

Let $G=(V,E)$ be an undirected graph, with $n:=|V|$ vertices, and let $d(v)$ denote the degree of vertex $v\in V$. The measure (volume) of a set $S\subseteq V$ is defined as the sum of the degree of vertices in $S$,

The conductance of a set $S$ is defined as

where $\partial(S)$ denotes the number of edges that leaves $S$. Let

be the conductance (uniform sparsest cut) of $G$. The Cheeger inequalities [AM85, Alo86] prove that the spectral partitioning algorithms finds, in nearly linear time, a $O(1/\sqrt{\phi(G)})$ approximation to the uniform sparsest cut problem. Most notably, the approximation factor does not depend of the size of the graph; in particular the Cheeger inequalities imply a constant factor approximation, if $\phi(G)$ is constant. Variants of the spectral partitioning algorithm are widely used in practice [Kle99, SM00, TM06].

Often, one is interested in applying a sparsest cut approximation algorithm iteratively, that is, first find an approximate sparsest cut in the graph, and then recurse on one or both of the subgraphs induced by the set found by the algorithm and by its complement. Such iteration might be used to find a balanced sparse cut if one exists (c.f. [OSV12]), or to find a good clustering of the graph, an approach that lead to approximate clusterings with good worst-case guarantees, as shown by Kannan, Vempala and Vetta [KVV04]. Even though each application of the spectral partitioning algorithm runs in nearly linear time, iterated applications of the algorithm can result in a quadratic running time.

Spielman and Teng [ST04], and subsequently [ACL06, AP09] studied local graph partitioning algorithms that find a set $S$ of approximately minimal conductance in time nearly linear in the size of the output set $S$. Note that the running time can be sub linear in the size of the input graph if the algorithm finds a small output set $S$. When iterated, such an algorithm finds a balanced sparse cut in nearly linear time in the size of the graph, and can be used to find a good clustering in nearly linear time as well.

Another advantage of such “local” algorithms is that if there are both large and small sets of near-optimal conductance, the algorithm is more likely to find the smaller sets. Thus, such algorithms can be used to approximate the “small-set expander” problem, which is related to the unique games conjecture [RS10] and the expansion profile of a graph (that is, what is the cut of smallest conductance among all sets of a given volume). Finding small, low-conductance, sets is also interesting in clustering applications. In a social network, for example, a low-conductance set of users in the “friendship” graph represents a “community” of users who are significantly more likely to be friends with other members of the community than with non-members, and discovering such communities has several applications. While large communities might correspond to large-scale, known, factors, such as the fact that American users are more likely to have other Americans as friends, or that people are more likely to have friends around their age, small communities contain more interesting information.

A local graph clustering algorithm, is a local graph algorithm that finds a non-expanding set in the local neighborhood of a given vertex $v$, in time proportional to the size of the output set. The work/volume ratio of such an algorithm, which is the ratio of the the computational time of the algorithm in a single run, and the volume of the output set, may depend only poly logarithmically to the size of the graph.

The problem first studied in the remarkable work of Spielman and Teng [ST04]. Spielman and Teng design an algorithm Nibble such that for any set $U\subseteq V$, if the initial vertex, $v$, is sampled randomly according to the degree of vertices in $U$, with a constant probability, Nibble finds a set of conductance $O(\phi^{1/2}(U)\log^{3/2}{n})$, with a work/volume ratio of $O(\phi^{-2}(U)\operatorname{polylog}(n))$, Nibble finds the desired set by looking at the threshold sets of the probability distribution of a $t$-step random walk started at $v$. To achieve the desired computational time they keep the support of the probability distribution small by removing a small portion of the probability mass at each step.

Andersen, Chung and Lang [ACL06], used the approximate PageRank vector rather than approximate random walk distribution, and they managed to improve the conductance of the output set to $O(\sqrt{\phi(U)\log{n}})$, and the work/volume ratio to $O(\phi^{-1}(U)\operatorname{polylog}{n})$. More recently, Andersen and Peres [AP09], use the evolving set process developed in the work of Diaconis and Fill [DF90], and they improved the work/volume ratio to $O(\phi^{-1/2}(U)\operatorname{polylog}{n})$, while achieving the same guarantee as [ACL06] on the conductance of the output set.

It has been a long standing open problem to design a local variant of the Cheeger’s inequalities: that is to provide a sublinear time algorithm with an approximation guarantee that does not depend on the size of $G$, assuming that the size of the optimum set is sufficiently smaller than $n$, and a randomly chosen vertex of the optimum set is given. In this work we answer this question, and we prove the following theorem:

${\textup{{ParESP}}}(v,\gamma,\phi,{\epsilon})$ takes as input a starting vetex $v\in V$, a target conductance $\phi\in(0,1)$, a target size $\gamma$, and $0<{\epsilon}<1$. For a given run of the algorithm it outputs a set $S$ of vertices with the expected work per volume ratio of $O(\gamma^{{\epsilon}}\phi^{-1/2}\log^{2}{n})$. If $U\subseteq V$ is a set of vertices that satisfy $\phi(U)\leq\phi$, and $\mu(U)\leq\gamma$, then there is a subset $U^{\prime}\subseteq U$ with volume at least $\mu(U)/2$, such that if $v\in U^{\prime}$, with a constant probability $S$ satisfies,

We remark that unlike the previous local graph clustering algorithms, the running time of the algorithm is slightly super linear in the size of the optimum.

As a byproduct of the above result we give an approximation algorithm for the expansion profile of $G$. Lovasz and Kannan [LK99] defined the expansion profile of a graph $G$ as follows:

Lovasz and Kannan used expansion profile as a parameter to prove strong upper-bounds on the mixing time of random walks. The notion of expansion profile recently received significant attention in the literature because of its close connection to the small set expansion problem and the unique games conjecture [RS10]. Raghavendra, Steurer, Tetali [RST10], and Bansal et al. [BFK+11] use semidefinite programming and designed algorithms that approximate $\phi(\gamma)$ within $O(\sqrt{\phi(\gamma)^{-1}\log\frac{\mu(V)}{\gamma}})$, and $O(\sqrt{\log{n}\log\frac{\mu(V)}{\gamma}})$ of the optimum, respectively. However, in the interesting regime of $\gamma=o(\mu(V))$, which is of interests to the small set expansion problem, the quality of both approximation algorithms is not independent of $\gamma$.

Here, we prove $\gamma$ independent approximation of $\phi(\gamma)$ as a function of $\phi(\gamma^{1-{\epsilon}})$, without any dependency in the size of the graph; specifically we prove the following theorem:

There is a polynomial time algorithm that takes as input a target conductance $\phi$, and $0<{\epsilon}<1/4$, and outputs a set $S$, s.t. if $\phi(U)\leq\phi$, for $U\subseteq V$, then $\mu(S)\leq 2\mu(U)^{1+{\epsilon}}$, and $\phi(S)\leq\sqrt{2\phi/{\epsilon}}$.

Our theorem indicates that the hard instance of the small set expansion problem are those where $\phi(\gamma)\approx 1$, for $\gamma\leq n^{1-\Omega(1)}$.

We remark that one can also use ParESP to approximate $\phi(\gamma)$ with slightly worse guarantees (up to constant factors), in sub linear time. Here, for the sake of clarity and simplicity of the arguments, we prove the theorem using random walks. Independent of our work, Kwok and Lau [KL12] have obtained a somewhat different proof of Theorem 1.2.

Our analysis techniques also provide a simpler proof of the structural result of Arora, Barak, Steurer [ABS10], that can be applied to non-regular graphs. Let $A$ be the adjacency matrix of $G$, and $D$ be the diagonal matrix of vertex degrees. Let the threshold rank of $G$, denoted by $\operatorname{rank}_{1-\eta}(D^{-1}A)$, be the number (with multiplicities) of eigenvalues $\lambda$ of $D^{-1}A$, satisfying $\lambda>1-\eta$.

For any graph $G$, and $0<{\epsilon}\leq 1$, if $\operatorname{rank}_{1-\eta}(D^{-1}A)\geq n^{(1+{\epsilon})\eta/\phi}$, then there exists a set $S\subseteq V$ of volume $\mu(S)\leq 4\mu(V)n^{-\eta/\phi}$ and $\phi(S)\leq\sqrt{2\phi/{\epsilon}}$. Such a set can be found by finding the smallest threshold set of conductance $\sqrt{2\phi/{\epsilon}}$, among the rows of $(D^{-1}A)^{t}$, for $t=O(\log{n}/\phi)$.

We remark that Arora et al. [ABS10] prove a variant of the above theorem for regular graphs with the stronger assumption that $\operatorname{rank}_{1-\eta}(D^{-1}A)\geq n^{100\eta/\phi}$. This essentially resolves their question of whether the factor $100$ can be improved to $1+{\epsilon}$. Independent of our work, O’Donnell and Witmer [OW12] obtained a different proof of the above theorem.

Our main technical result is that if $S$ is a set of vertices, and we consider a $t$-step lazy random walk started at a random element of $S$, then the probability that the walk is entirely contained in $S$ is at least $(1-\phi(S)/2)^{t}$. Previously, only the lower bound $1-t\phi(S)/2$ was known, and the analysis of other local clustering algorithms implicitly or explicitly depended on such a bound.

For comparison, when $t=1/\phi(S)$, the known bound would imply that the walk has probability at least $1/2$ of being entirely contained in $S$, with no guarantee being available in the case $t=2/\phi(S)$, while our bound implies that for $t=(\alpha\ln n)/\phi$ the probability of being entirely contained in $S$ is still at least $1/n^{\alpha}$. Roughly speaking, the $\Omega(\log n)$ factor that we gain in the length of walks that we can study corresponds to our improvement in the expansion bound, while the $1/n^{\alpha}$ factor that we lose in the probability corresponds to the factor that we lose in the size of the non-expanding set. We also use this bound to prove stronger lower bounds on the uniform mixing time of reversible markov chains.

Our polynomial time algorithm to approximate the expansion profile of a graph is the same as the algorithm used by Arora, Barak and Steurer [ABS10] to find small non-expanding sets in graphs of a given threshold rank, but our analysis is different. (Our analysis can be also be used to give a different proof of their result.) Arora, Barak and Steurer use the threshold rank to argue that a random walk started from a random vertex of $G$ will be at the initial vertex after $t$ steps with probability at least $\operatorname{rank}_{1-\eta}(G)(1-\eta)^{t}/n$. Then, they argue that if all sets of a certain size have large conductance, this probability must be small, which is a contradiction. To make this quantitative they use the second norm of the probability distribution vector ($\lVert{\bf p}_{t}\rVert$) as a potential function, where ${\bf p}_{t}$ is the distribution of the walk after $t$ steps, and they choose $t$ to get a sufficiently small potential function and argue that the probability of being at the initial vertex after $t$ steps must be small. In our analysis, we use the fact that for any set $S$, there is a vertex $v\in S$ such that the probability that the walk started at $v$ remains in $S$ is at least $(1-\phi(S)/2)^{t}$. Then, we use the potential function $I({\bf p}_{t},\gamma)$ introduced in the work of Lovasz and Simonovits [LS90]. Roughly speaking, $I({\bf p}_{t},\gamma)$ is defined as follows: consider the distribution ${\bf p}_{t}$ of the vertex reached in a $t$-step random walk started from a random element of $S$, take the $k$ vertices of highest probability under ${\bf p}_{t}$, where $k$ is chosen so that their total volume is about $\gamma$, then $I({\bf p}_{t},\gamma)$ is the total probability under ${\bf p}_{t}$ of those $k$ vertices.

Using the machinery of Lovasz and Simonovits, we can upper-bound $I({\bf p}_{t},\gamma)$ by $\frac{\gamma}{\Gamma}+\sqrt{\gamma}(1-\phi^{2}/2)^{t}$ conditioned on all of the threshold sets of volume at most $\Gamma$ of the probability distribution vectors up to time $t$ having conductance less than $\phi$. Letting $t=\Omega(\alpha\log\mu(S)/\phi(S))$, $\Gamma=O(\mu(S)^{1+\alpha})$, and $\phi=O(\sqrt{\phi(S)/\alpha})$ since the walk remains in $S$ with probability $\mu(S)^{-\alpha}<I({\bf p}_{t},\gamma)$, at least one of the threshold sets of volume at most $O(\mu(S)^{1+\alpha})$, must have conductance $O(\sqrt{\phi(S)/\alpha})$.

Our local algorithm uses the evolving set process. The evolving set process starts with a vertex $v$ of the graph, and then produces a sequence of sets $S_{1},S_{2},\ldots,S_{\tau}$, with the property that at least one set $S_{t}$ is such that $\partial S_{t}/\mu(S_{t})\leq O(\sqrt{\log\mu(S_{\tau})/\tau})$. If one can show that up to some time $T$ the process constructs sets all of volume at most $\gamma$, then we get a set of volume at most $\gamma$ and conductance at most $O(\sqrt{\log\gamma/T})$. Andersen and Peres were able to show that if the graph has a set $S$ of conductance $\phi$, then the process is likely to construct sets all of volume at most $2\mu(S)$ for at least $T=\Omega(1/\phi)$ steps, if started from a random element of the $S$, leading to their $O(\sqrt{\phi\log n})$ guarantee. We show that for any chosen $\alpha<1/2$, the process will construct sets of volume at most $O(\mu(S)^{1+\alpha})$ for $T=\Omega(\alpha\log\mu(S)/\phi)$ steps, with probability at least $1/\mu(S)^{\alpha}$. This is enough to guarantee that, at least with probability $1/\mu(S)^{\alpha}$, the process constructs at least one set of conductance $O(\sqrt{\phi/\alpha})$. To obtain this conclusion, we also need to strengthen the first part of the analysis of Andersen and Peres: they show that the process has at least a constant probability of constructing a set of low conductance in the first $t$ steps, while we need to show that this happens with probability at least $1-1/\mu(S)^{\Omega(1)}$, because we need to take a union bound with the event that $t$ is large, for which probability we only have a $\mu(S)^{-\Omega(1)}$ lower bound. Finally, to achieve a constant probability of success, we run $\mu(S)^{\alpha}$ copies of the evolving set process simultaneously, and stop as soon as one of the copies finds a small non-expanding set.

Preliminaries

Let $G=(V,E)$ be an undirected graph, with $n:=|V|$ vertices and $m:=|E|$ edges. Let $A$ be the adjacency matrix of $G$, $D$ be the diagonal matrix of vertex degrees, and $d(v)$ be the degree of vertex $v\in V$. The volume of a subset $S\subseteq V$ is defined as the summation of the degree of vertices in $S$,

Let $E(S,V\setminus S):=\left\{\{u,v\}:u\in S,v\notin S\right\}$ be the set of the edges connecting $S$ to $V\setminus S$, and we use $\partial(S)$ to denote the number of those edges, we also let $E(S):=\left\{\{u,v\}:u,v\in S\right\}$ be the set of edges inside $S$. The conductance of a set $S\subseteq V$ is defined to be

Observe that $\phi(V)=0$. In the literature, the conductance of a set is sometimes defined to be $\partial(S)/\min(\mu(S),\mu(V\setminus S))$. Notice that the quantities are within a constant factor of each other if $\mu(S)=O(\mu(V\setminus S))$. Since, here we are interested in finding small non-expanding sets, we would rather work with the above definition.

We define the following probability distribution vector on a set $S\subseteq V$ of vertices:

In particular, we use $\pi(v)\equiv\pi_{V}(v)$ as the stationary distribution of a random walk in $G$.

Throughout the paper, let $I$ be the identity matrix, and for any subset $S\subseteq V$, let $I_{S}$ be the diagonal matrix such that $I_{S}(v,v)=1$, if $v\in S$ and otherwise. Also, let ${\bf 1}$ be all one vector, and ${\bf 1}_{S}$ be the indicator vector of the set $S$. We may abuse the notation and use ${\bf 1}_{v}$ instead of ${\bf 1}_{\{v\}}$ for a vertex $v\in V$.

We use lower bold letters to denote the vectors, and capital letters for the matrices/sets. For a vector ${\bf x}:V\rightarrow R$, and a set $S\subseteq V$, we use $x(S):=\sum_{v\in S}x(v)$. Unless otherwise specified, ${\bf x}$ is considered to be a column vector, and ${\bf x}^{\prime}$ is its transpose.

For a square matrix $A$, we use $\lambda_{\min}(A)$ to denote the minimum eigenvalue of $A$, and $\lambda_{\max}(A)$ to denote the maximum eigenvalue of $A$.

2 Random Walks

We will consider the lazy random walk on $G$ that each time step stays at the current vertex with probability $1/2$ and otherwise moves to the endpoint of a random edge attached to the current vertex. We abuse notation, and we use ${\cal G}:=(D^{-1}A+I)/2$ as the transition probability matrix of this random walk, and $\pi(.)$ is the unique stationary distribution, that is ${{\bm{\pi}}^{\prime}}{\cal G}={{\bm{\pi}}^{\prime}}$. We write ${\cal{P}}_{v}\left[.\right]$ to denote the probability measure of the lazy random walk started from a vertex $v\in V$.

Let $X_{t}$ be the random variable indicating the $t^{th}$ step of the random walk started at $v$. Observe that the distribution of $X_{t}$ is exactly, ${{\bf 1}_{v}^{\prime}}{\cal G}^{t}$. For a subset $S\subseteq V$, and $v\in V$, and integer $t>0$, we write $\operatorname{esc}(v,t,S):={\cal{P}}_{v}\left[\cup_{i=0}^{t}X_{i}\notin S\right]$ to denote the probability that the random walk started at $v$ leaves $S$ in the first $t$ steps, and $\operatorname{rem}(v,t,S):=1-\operatorname{esc}(v,t,S)$ as the probability that the walk stays entirely inside $S$. It follows that,

3 Spectral Properties of the Transition Probability Matrix

Although the transition probability matrix, ${\cal G}$, is not a symmetric matrix, it features many properties of the symmetric matrices. First of all, ${\cal G}$ can be transformed to a symmetric matrix simply by considering $D^{1/2}{\cal G}D^{-1/2}$. It follows that any eigenvector of $D^{1/2}{\cal G}D^{-1/2}$ can be transformed into a left (right) eigenvector of ${\cal G}$, once it is multiplied by $D^{1/2}$ ($D^{-1/2}$), respectively. Henceforth, the left and right eigenvalues of ${\cal G}$ are the same, and they are real.

Furthermore, since $\lVert D^{-1}A\rVert_{\infty}\leq 1$, and ${\cal G}$ is the average of $D^{-1}A$, and the identity matrix, we must have $\lambda_{\min}({\cal G})\geq 0$ and $\lambda_{\max}({\cal G})\leq 1$. Thus $D^{1/2}{\cal G}D^{-1/2}$ is a positive semidefinite matrix, symmetric matrix, whose largest eigenvalue is at most 1.

4 The Evolving Set Process

The evolving set process is a markov chain on the subsets of the vertex set $V$. The process together with the closely related volume biased evolving set process is introduced in the work of Diaconis and Fill [DF90] as the strong stationary dual of a random walk. Morris and Peres [MP03] use it to upper-bound the mixing time of random walks in terms of isoperimetric properties.

Given a subset $S_{0}\subseteq V$, the next subset $S_{1}$ is chosen as follows: first we choose a threshold $R\in$ uniformly at random. Then, we let

For any set $S\subseteq V$, $\psi(S)\geq\phi(S)^{2}/8$.

The volume-biased evolving set process is a special case of the evolving set process where the markov chain is conditioned to be absorbed in $V$. In particular, the transition kernel is defined of a volume biased ESP as follows:

Given a state $S_{0}$, we write ${\bf P}_{S_{0}}\left[.\right]:={\bf P}\left[.|S_{0}\right]$ to denote the probability measure of the volume biased ESP started at state $S_{0}$, and we use ${\bf E}_{S_{0}}\left[.\right]:={\bf E}\left[.|S_{0}\right]$ for the expectation.

Andersen and Peres used the volume biased ESP as a local graph clustering algorithm [AP09]. They show that for any non-expanding set $U$, if we run the volume biased ESP from a randomly chosen vertex of $U$, with a constant probability, there is a set in the sample path of expansion $O(\sqrt{\phi(U)\log n})$, and volume at most $2\mu(U)$. As a part of their proof, they designed an efficient simulation of the volume biased ESP, called GenerateSample. They prove the following theorem,

There is an algorithm, GenerateSample, that simulates the volume biased ESP such that for any vertex $v\in V$, any sample path $(S_{0}=\{v\},\ldots,S_{\tau})$, is generated with probability ${\bf P}_{v}\left[S_{0},\ldots,S_{\tau}\right]$. Furthermore, for a stopping time $\tau$ that is bounded above by $T$, let $W(\tau)$ be the time complexity of GenerateSample if it is run up to time $\tau$. Then, the expected work per volume ratio of the algorithm is

Upper Bounds on the Escaping Probability of Random Walks

In this section we establish strong results on the escaping probability of the random walks. Spielman and Teng [ST08] show that for any set $S\subseteq V$, $t>0$, the random walk started at a randomly (proportional to degree) chosen vertex of $S$, remain in $S$ for $t$ steps with probability at least $1-t\phi(S)/2$. We strengthen this result, by improving the lower bound to $(1-\phi(S)/2)^{t}$,

For any set $S\subseteq V$, and integer $t>0$,

Furthermore, there is a subset $S^{t}\subseteq S$, such that $\mu(S^{t})\geq\mu(S)/2$, and for all $v\in S^{t}$

We remark that the second statement does not follow from a simple application of the Markov Inequality to the first statement, as this is the case in [ST08]. Whence, here both of the results incorporate non-trivial spectral arguments.

As a corollary, we prove strong lower bounds on the uniform mixing time of random walks in section 6. In the rest of this section we prove 3.1. We start by proving (2).

Using equation (1), and a simple induction on $t$, (2) is equivalent to the following equation:

Let $P:=D^{1/2}I_{S}{\cal G}I_{S}D^{-1/2}$. First we show that (4) is equivalent to the following equation:

Then, we use 3.2 that shows the above equation holds for any symmetric positive semidefinite matrix $P$, and any norm one vector ${\bf x}=\sqrt{{\bm{\pi}}_{S}}$. First observe that by the definition of $P$, for any $t>0$,

Equation (5) is derived simply from equation (4), by putting (6), (7) together. Next we prove equation (5) using 3.2. First observe that $\sqrt{{\bm{\pi}}_{S}}$ is a norm one vector. On the other hand, by definition $P=\frac{1}{2}(D^{-1/2}I_{S}AI_{S}D^{-1/2}+D^{-1/2}I_{S}D^{-1/2})$ is a symmetric matrix.

It remains to show that $P$ is positive semidefinite. This follows from the same reason that $D^{1/2}{\cal G}D^{-1/2}$ is positive semidefinite. In particular, since eigenvectors of $P$ can be transformed to the eigenvectors of $I_{S}{\cal G}I_{S}$, the eigenvalues of $I_{S}{\cal G}I_{S}$ are the same as the eigenvalues of $P$. Finally, since $\lVert I_{S}D^{-1}AI_{S}\rVert_{\infty}\leq 1$, and $I_{S}{\cal G}I_{S}$ is the average of $I_{S}D^{-1}AI_{S}$ and $I_{S}$, we must have $\lambda_{\min}(I_{S}{\cal G}I_{S})\geq 0$, and $\lambda_{max}(I_{S}{\cal G}I_{S})\leq 1$. Thus $P$ is positive semidefinite. Now, (5) simply follows from 3.2. This completes the proof of (2)

It remains to prove (3). We prove it by showing that for any set $X\subseteq S$, of volume $\mu(X)\geq\mu(S)/2$, the random walk started at a randomly (proportional to degree) chosen vertex of $X$, remains in $X$ (and $S$), with probability at least $\frac{1}{200}(1-3\phi(S)/2)^{t}$,

Therefore, in any such set $X$, there is a vertex that satisfy (3), hence the volume of the set of vertices that satisfy (3) is at least half of $\mu(S)$.

Using equations (6) and (7), (8) is equivalent to the following equation,

We prove the above equation using 3.3. Let $Y=S\setminus X$, and define

Since $X\cap Y=\emptyset$, $\langle{\bf x},{\bf y}\rangle=0$, and $\lVert{\bf x}+{\bf y}\rVert=\lVert\sqrt{{\bm{\pi}}_{S}}\rVert=1$. Furthermore, since $\mu(X)\geq\mu(S)/2\geq\mu(Y)$, $\lVert{\bf x}\rVert\geq\lVert{\bf y}\rVert$. Therefore, $P$, ${\bf x},{\bf y}$ satisfy the requirements of 3.3. Finally, since ${\sqrt{{\bm{\pi}}_{X}}^{\prime}}P^{t}\sqrt{{\bm{\pi}}_{X}}\geq{{\bf x}^{\prime}}P^{t}{\bf x}$, (8) follows from 3.3. This completes the proof of Proposition 3.1.

Since all of the inequalities in lemma’s statement follows from the first inequality, we only prove the first inequality. Let ${\bf v}_{1},{\bf v}_{2},\ldots,{\bf v}_{n}$ be the set of orthonormal eigenvectors of $P$ with the corresponding eigenvalues $\lambda_{1},\lambda_{2},\ldots,\lambda_{n}$. For any $k\geq 1$, we have

On the other hand, since $\{{\bf v}_{1},\ldots,{\bf v}_{n}\}$ is an orthornormal system, we have

For any $k>0$, Let $f_{k}(\lambda)=\lambda^{k}$; it follows that,

Since $P$ is positive semidefinite, $\lambda_{\min}(P)\geq 0$. Thus, for all $t>0$, the function $f_{t}(.)$ is increasing in the support of ${\cal D}$. The above inequality follows from the Chebyshev’s sum inequality. ∎

Let ${\bf z}:={\bf x}+{\bf y}$. Since ${\bf x}$ is orthogonal to ${\bf y}$, we have $\lVert{\bf y}\rVert^{2}\leq 1/2\leq\lVert{\bf x}\rVert^{2}$. Let ${\bf v}_{1},{\bf v}_{2},\ldots,{\bf v}_{n}$ be the set of orthonormal eigenvectors of $P$ with the corresponding eigenvalues $\lambda_{1},\lambda_{2},\ldots,\lambda_{n}$. Let $\alpha>0$ be a constant that will be fixed later in the proof. Define $B:=\{i:|\langle{\bf x},{\bf v}_{i}\rangle|\geq\alpha|\langle{\bf y},{\bf v}_{i}\rangle|\}$. First observe that,

where the equality follows from equation (11), the first inequality uses $\lambda_{\min}(P)\geq 0$, and the last inequality follows from the definition of $B$, that is for any $i\in B$, $\langle{\bf x},{\bf v}_{i}\rangle^{2}\geq(\langle{\bf z},{\bf v}_{i}\rangle/(1+1/\alpha))^{2}$. Let $L:=\sum_{i\in B}\langle{\bf z},{\bf v}_{i}\rangle^{2}$. Then, since $\lambda_{\min}(P)\geq 0$, by Jensen’s inequality,

where the second inequality follows by the assumptions that $\lambda_{\max}(P)\leq 1$, and that $\lVert{\bf z}\rVert=1$, and the last inequality follows from the fact that ${{\bf z}^{\prime}}P{\bf z}\leq 1$, and that

Putting equations (12) and (13), and letting $\alpha=0.154$ we get,

Approximating the Expansion Profile

In this section we use the machinery developed in the works of Lovasz and Simonovits [LS90, LS93] to prove Theorem 1.2, Theorem 1.3. We start by introducing some notations.

Let ${\bf p}$ be a probability distribution vector on the vertices of $V$, and let $\sigma(.)$ be the permutation of the vertices that is decreasing with respect to $p(v)/d(v)$, and breaking ties lexicographically. That is, suppose

We use $T_{i}({\bf p}):=\{\sigma(1),\ldots,\sigma(i)\}$ to denote the threshold set of the first $i$ vertices. Following Spielman, Teng [ST08] (c.f. Lovasz, Simonovits [LS90]), we use the following potential function,

Observe that for $I({\bf p},x)$ is a non-decreasing piecewise linear concave function of $x$, that is $I({\bf p},x)=p(T_{j}({\bf p}))$, for $x=\mu(T_{j}({\bf p}))$, and is linear in other values of $x$. We use $I({\bf p},x)$ as a potential function to measure the distance of the distribution ${\bf p}$ from the stationary distribution ${\bm{\pi}}$.

We find the small non-expanding set in Theorem 1.2, by running ${\textup{{Threshold}}}(\sqrt{2\phi/{\epsilon}},{\epsilon}\ln{\mu(V)}/\phi)$. The algorithm simply returns the smallest non-expanding set among the threshold sets of the rows of ${\cal G}^{t}$, for $t=O({\epsilon}\log\mu(V)/\phi)$. The details are described in Algorithm 1.

If none of the sets $T_{i}({{\bf 1}_{v}^{\prime}}{\cal G}^{t})$ is a non-expanding set, then Lovasz and Siminovits [LS90, LS93] prove that the curve $I({{\bf 1}_{v}^{\prime}}{\cal G}^{t},x)$ lies far below $I({{\bf 1}_{v}^{\prime}}{\cal G}^{t-1},x)$. This is quantified in the following lemma:

Let ${\cal G}$ be a transition probability matrix of a lazy random walk on a graph. For any probability distribution vector ${\bf p}$ on $V$, if $\phi(T_{i}({{\bf p}^{\prime}}{\cal G}))\geq\Phi$, then for $x=\mu(T_{i}({{\bf p}^{\prime}}{\cal G}))$,

By repeated application of the above lemma, Lovasz and Simonovits [LS90] argue that, if all of the sets $T_{i}({{\bf 1}_{v}^{\prime}}{\cal G}^{t})$ are expanding, then $I({{\bf 1}_{v}^{\prime}}{\cal G}^{t},.)$ approaches the straight line. In the next lemma we show that, if all of the small threshold sets (i.e., $\mu(T_{i}({{\bf 1}_{v}^{\prime}}{\cal G}^{t}))\leq\Gamma$), are expanding, then $I({{\bf 1}_{v}^{\prime}}{\cal G}^{t},.)$ approaches the curve $x/\Gamma$.

For any vertex $v\in V$, $t\geq 0$, and $0\leq\Gamma\leq m$, $0\leq\Phi\leq 1/2$, if for all $t\leq T$, all of threshold sets $T_{i}({{\bf 1}_{v}^{\prime}}{\cal G}^{t})$ of volume at most $\Gamma$, has expansion at least $\Phi$, then for any $0\leq t\leq T$,

We prove by induction. The lemma trivially holds for $t=0$. This is because the LHS is $x/\mu(v)$ for $0\leq x\leq\mu(v)$ and $1$ for larger values of $x$, while the RHS is $\sqrt{x/\mu(v)}$ for all $x\geq 0$. Next, we prove the lemma’s statement holds for $t$, assuming that it holds for $t-1$. Let ${\bf p}:={{\bf 1}_{v}^{\prime}}{\cal G}^{t-1}$. First of all, since $I({{\bf p}^{\prime}}{\cal G},.)$ is a piecewise-linear concave function of $x$, it is sufficient to prove the statement for values of $x=\mu(T_{i}({{\bf p}^{\prime}}{\cal G}))$. For $x\geq\Gamma$, the statement holds trivially, because the RHS is at least 1, while the LHS is less than or equal to 1. Now, suppose $x<\Gamma$, and $x=\mu(T_{i}({{\bf p}^{\prime}}{\cal G}))$. Using 4.1, we have

where the first inequality uses the assumption that $x<\Gamma\leq m$, the second inequality uses the induction hypothesis, and the last inequality uses the inequality

Now we are ready to prove Theorem 1.2: using 3.1 we show that $I({{\bf 1}_{v}^{\prime}}{\cal G}^{t})$ does not converge to $x/\Gamma$, for $t\approx\log\gamma/\phi$. Therefore, by the previous lemma, at least one of the small threshold sets is non-expanding.

Let, $\gamma=\mu(U)$, $T={\epsilon}\ln{\gamma}/\phi$, $\Gamma=2\gamma^{1+{\epsilon}}$, and $\Phi=\sqrt{2\phi/{\epsilon}}$. We show that ${\textup{{Threshold}}}(\phi,{\epsilon})$ returns a set of volume at most $\Gamma$, and conductance at most $\Phi$. Wlog we may assume that $\Gamma<m$, otherwise the statement is trivial. We prove by contradiction; assume that the output of the algorithm has volume larger than $\Gamma$, we show that 4.2 and 3.1 can not hold simultaneously. First of all, by 3.1, there exists a vertex $u\in U$, such that

Let ${\bf p}={{\bf 1}_{u}^{\prime}}{\cal G}^{T}$. Let $w(v)=1$, for all $v\in U$, and $w(v)=0$ for the rest of the vertices. By equation (14), we have

which is a contradiction. The last inequality uses the fact that ${\epsilon}<1/2$. ∎

Next we show that using 4.2 we can provide a simpler, and yet stronger proof of the result of Arora, Barak and Steurer [ABS10]. See 1.3

Wlog we assume $\phi\leq 1/2$, $\eta<\phi$, and $n^{\eta/\phi}>4$. Let $T={\epsilon}\ln{n}/\phi$, $\Gamma=4\mu(V)n^{-\eta/\phi}$, and $\Phi=\sqrt{2\phi/{\epsilon}}$. We show that ${\textup{{Threshold}}}(\Phi,T)$ finds a set of volume $\Gamma$ and conductance at most $\Phi$. We prove by contradiction; suppose that Threshold does not find such a set. Since ${\cal G}=\frac{1}{2}(D^{-1}A+I)$, $\operatorname{rank}_{1-\eta/2}({\cal G})\geq n^{(1+{\epsilon})\eta/\phi}$. Therefore, by the next claim, there is a vertex $u$ such that,

Let ${\bf p}={{\bf 1}_{u}^{\prime}}{\cal G}^{T}$, $x=\mu(u)$, $w(v)=1$, for $v=u$ and $w(v)=0$ for the rest of the vertices. By equation (14), we have

which is a contradiction, since $n^{\eta/\phi}>4$.

For any graph $G$, if $\operatorname{rank}_{1-\eta}({\cal G})\geq r$, then there is a vertex $u\in V$, such that

Let $0\leq\lambda_{1},\ldots,\lambda_{n}\leq 1$ be the eigenvalues of ${\cal G}$. We use the trace formula,

Now, let $U_{1}:=\{v:{\bf 1}_{v}{\cal G}^{t}{\bf 1}_{v}<r(1-\eta)^{t}/2n\}$, and $U_{2}:=\{v:{\bf v}1_{v}{\cal G}^{t}{\bf 1}_{v}<\frac{\mu(v)}{2\mu(V)}r(1-\eta)^{t}\}$. It follows that,

Therefore, there is a vertex $u\notin U_{1},U_{2}$ that satisfies claim’s statement. ∎

Almost Optimal Local Graph Clustering

In this section we use the volume biased ESP to design a local graph clustering algorithm with a worst case guarantee on the conductance of output set that is independent of the size of $G$.

Let $(S_{0},S_{1},\ldots,S_{\tau})$ be a sample path of the volume biased ESP, for a stopping time $\tau$. Andersen and Peres show that with a constant probability the conductance of at least one of the sets in the sample path is at most $O(\sqrt{\frac{1}{\tau}\log\mu(S_{\tau})})$,

For any starting set $S_{0}$, and any stopping time $\tau$, and $\alpha>0$,

Here, we strengthen the above result, and we show the event occurs with much higher probability. In particular, we show the with probability at least $1-1/\alpha$, the conductance of at least one of the sets in the sample path is at most $O(\sqrt{\frac{1}{\tau}\log(\alpha\cdot\mu(S_{\tau}))})$.

For any starting set $S_{0}\subseteq V$, and any stopping time $\tau$, and $\alpha\geq 0$,

Andersen, Peres [AP09, Lemma 1] show that $M_{t}$ is a martingale in the volume biased ESP. It follows from the optional sampling theorem [Wil91] that ${\bf E}\left[M_{\tau}\right]=M_{0}=1$. Thus, by the Markov inequality, for any $\alpha>0$, we have

By taking logarithm, from both sides of the event in the above equation we obtain,

On the other hand, by the definition of $M_{\tau}$,

where the first inequality follows by the fact that $1/(1-\psi(S_{i}))\geq e^{\psi(S_{i})}$, and the last inequality follows by 2.1. Putting (15) and (16) together proves the lemma. ∎

The previous lemma shows that for any $\gamma,\phi>0$, if we can run the process for $T\approx{\epsilon}\log\gamma/\phi$ steps without observing a set larger than $\gamma^{O(1)}$, then, with probability $1-1/\gamma$, one of the sets in the sample path must have an expansion of $O(\sqrt{\phi/{\epsilon}})$, which is what we are looking for. Next we use the following lemma by Andersen and Peres, together with 3.1, to show that event occurs with some non-zero probability. That is, for any ${\epsilon}<1$, with probability at least $\approx\gamma^{-{\epsilon}}$, the volume of all sets in the sample path of the process are at most $O(\gamma^{1+{\epsilon}})$. Then, by the union bound we can argue both event occur with probability at least $\Omega(\gamma^{-{\epsilon}})$.

For any set $U\subset V$, $v\in U$, and integer $T>0$, the following holds,

Given $\gamma>0$, $0<\phi<1/4$, $0<{\epsilon}<1$, such that $G$ has a set $U\subset V$ of volume $\mu(U)\leq\gamma$, and conductance $\phi(U)\leq\phi$. Let $T={\epsilon}\ln{\gamma}/3\phi$. There is a constant $c>0$, and a subset $U^{T}\subseteq U$ of volume $\mu(U^{T})\geq\mu(U)/2$ such that for any $v\in U^{T}$, with probability at least $c\gamma^{-{\epsilon}}/8$, a sample path $(S_{1},S_{2},\ldots,S_{T})$ of the volume biased ESP started from $S_{0}=\{v\}$ satisfies the following,

For some $t\in[0,T]$, $\phi(S_{t})\leq O(\sqrt{100(1-\ln c)\phi/{\epsilon}})$,

For all $t\in[0,T]$, $\mu(S_{t}\cap U)\geq c\gamma^{-{\epsilon}}\mu(S_{t})/2$, and henceforth, $\mu(S_{t})\leq 2\gamma^{1+{\epsilon}}/c$.

First of all, we let $U^{T}$ be the set of vertices $v\in U$ such that

By 3.1, there exists a constant $c>0$ such that $\mu(U^{T})\geq\mu(U)/2$. In the rest of the proof let $v$ be a vertex in $U^{T}$. We have,

Now, let $\beta:=1+c\gamma^{-{\epsilon}}/2$. By 5.3, we have

Since for any $S\subset V$, $\mu(S\setminus U)+\mu(S\cap U)=\mu(S)$, we have

On the other hand, let $\alpha:=\gamma$. By 5.2, with probability $1-1/\gamma$, for some $t\in[0,T]$,

Therefore, since ${\epsilon}<1$, by the union bound we have

Finally, since for any set $S\subseteq V$, $\mu(S\cap U)\leq\mu(U)\leq\gamma$, in the above event, $\mu(S_{T})\leq\frac{2\gamma^{1+{\epsilon}}}{c}$. Therefore,

To prove Theorem 1.1, we can simply run $\gamma^{{\epsilon}}$ copies of the volume biased ESP in parallel. Using the previous lemma with a constant probability at least one of the copies finds a non-expanding set. Moreover, we may bound the time complexity of the algorithm using Theorem 2.2. The details of the algorithm is described in Algorithm 2.

Now we are ready to prove Theorem 1.1. See 1.1

Let $U^{\prime}=U^{T}$ as defined in 5.4. First of all, for any $v\in U^{\prime}$, by 5.4, each copy of volume biased ESP, with probability $\Omega(\gamma^{-{\epsilon}/2})$, finds a set $S$ such that $\mu(S)\leq 2\gamma^{{\epsilon}/2}/c$, and $\phi(S)\leq\sqrt{200(1-\ln c)\phi/{\epsilon}}$; but, since $\gamma^{{\epsilon}/2}$ copies are executed independently, at least one of them will succeed with a constant probability. Therefore, with a constant probability the output set will satisfy properties (1), (2) in theorems’ statement. This proves the correctness of the algorithm.

It remains to compute the time complexity. Let, $k:=\gamma^{{\epsilon}/2}$ be the number of copies, and $W_{1},\ldots,W_{k}$ be random variables indicating the work done by each of the copies in a single run of ParESP, thus $\sum_{i}W_{i}$ is the time complexity of the algorithm. Let $M$ be the random variable indicating the volume of the output set of the algorithm; we let $M=0$ if the algorithm does not return any set. Also, for $1\leq i\leq k$, let $X_{i}$ be $1/M$ if the output set is chosen from the $i^{th}$ copy, and otherwise, and let $X:=\sum X_{i}$. We write ${\bf P}^{k}_{v}\left[.\right]$ to denote the probability measure of the $k$ independent volume biased ESP all started from $S_{0}=\{v\}$, and ${\bf E}^{k}_{v}\left[.\right]$ for the expectation. To prove the theorem it is sufficient to show

By linearity of expectation, it is sufficient to show that for all $1\leq i\leq k$,

By symmetry of the copies, it is sufficient to show that only for $i=1$. Furthermore, since conditioned on $X_{1}\neq 0$, $W_{1}=\max_{i}W_{i}$, we just need to show,

Let $\tau$ be a stopping time, bounded from above by $T$, indicating the first time where a set $S_{\tau}$ of volume $\mu(S_{\tau})\leq 2\gamma^{1+{\epsilon}}/c$, and conductance $\phi(S_{\tau})\leq\sqrt{200(1-\log c)\phi/{\epsilon}}$ is observed in the first copy if it is executed up to time $\tau$, and $W_{1}(\tau)$ as the amount of work done by that time. Observe that for any element of the joint probability space, $X_{1}W_{1}\leq W_{1}(\tau)/\mu(S_{\tau})$. This is because, we always have $W_{1}\leq W_{1}{\tau}$, and $X_{1}\leq\mu(S_{\tau})$. Therefore,

where the second to last equation follows from Theorem 2.2. ∎

Lower Bounds on Uniform Mixing Time of Random Walks

In this section we prove lower bounds on the mixing time of reversible markov chains. Since any reversible finite state markov chain can be realized as a random walk on a weighted undirected graph, for simplicity of notations, we model the markov chain as a random walk on a weighted graph $G$.

The ${\epsilon}$-mixing time of a random walk in total variation distance is defined as

The mixing time of the chain is usually defined as $\tau_{V}(1/4)$. The ${\epsilon}$-uniform mixing time of the chain is defined as

It is worth noting that the uniform mixing time can be considerably larger than the mixing time in total variation distance.

Let $\phi(G):=\min_{S:\mu(S)\leq\mu(V)/2}\phi(S)$. Jerrum and Sinclair [JS89] prove that the ${\epsilon}$-uniform mixing time of any lazy random walk is bounded from above by,

On the other hand, one can use $\phi(G)$ as the bottleneck ratio to provide lower-bound on the mixing time of the random walks. It follows from the Cheeger’s inequality that (see e.g. [LPW06]),

In the next proposition we prove stronger lower bounds on the uniform mixing time of any reversible markov chain.

For any graph $G=(V,E)$, $1\geq\gamma\leq\mu(V)/2$, and $0<{\epsilon}<1$,

Let $S\subseteq V$ such that $\mu(S)\leq\gamma$, and $\phi(S)=\phi(\gamma)$. Let $t\geq-\ln(2\pi(S))/2\phi(S)-2$ be an even integer. By the next claim, and equation (1), there exists a vertex $u\in S$ such that

Since ${\cal{P}}_{u}\left[X_{t}\in S\right]\geq\operatorname{rem}(u,t,S)$, there is a vertex $v\in S$ such that,

where the first inequality uses ${\cal{P}}_{u}\left[X_{t}\in S\right]=\sum_{v\in S}{\cal{P}}_{u}\left[X_{t}=v\right]$. Therefore, $\frac{|{\cal{P}}_{u}\left[X_{t}=v\right]-\pi(v)|}{\pi(v)}\geq 1,$ and by equation (17), for any ${\epsilon}<1$, $\tau({\epsilon})\geq t.$ The proposition follows from the choice of $S$; that is $t>\frac{\ln(\mu(V)/2\gamma)}{\phi(\gamma)}-2$.

For any (weighted) graph $G$, $S\subseteq V$, and integer $t>0$,

The proof is very similar to 3.1, except, here $I_{S}D^{-1}AI_{S}$ is not (necessarily) a positive semidefinite matrix. This is the reason that we prove the inequality only for even time steps of the walk.

Let $P:=D^{-1/2}I_{S}AI_{S}D^{-1/2}$ be a symmetric matrix. By equation (7), $1-\phi(S)={{\bm{\pi}}_{S}^{\prime}}(I_{S}D^{-1}AI_{S}){\bf 1}_{S}$. Using equation (6), the claim’s statement is equivalent to the following equation:

The above inequality can be proved using techniques similar to 3.2. Let ${\bf v}_{1},\ldots,{\bf v}_{n}$ be the eigenvectors of $P$, corresponding to the eigenvalues $\lambda_{1},\ldots,\lambda_{n}$. By equation (11), (18) is equivalent to the following equation:

Since $\sqrt{{\bm{\pi}}_{S}}$ is a norm one vector, and $f(\lambda)=\lambda^{2t}$ is a convex function, the above equation holds by the Jensen’s inequality. ∎

We remark that the above bound only holds for the uniform mixing time, and it can provide much stronger lower bound than the bottleneck ratio, if $\gamma\ll\mu(V)$.

We would like to thank Or Meir and Amin Saberi for stimulating discussions. We also thank anonymous reviewers for helpful comments on the earlier version of this document.

References