Distributed optimization over time-varying directed graphs

I Introduction

We consider the problem of distributed convex optimization by a network of nodes when knowledge of the objective function is scattered throughout the network and unavailable at any singe location. There has been much recent interest in multi-agent optimization problems of this type that arise whenever a large collections of nodes - which may be processors, nodes of a sensor network, vehicles, or UAVs - desire to collectively optimize a global objective by means of local actions taken by each node without any centralized coordination.

Specifically, we will study the problem of optimizing a sum of $n$ convex functions by a network of $n$ nodes when each function is known to only a single node. This problem frequently arises when control and signal processing protocols need to be implemented in sensor networks. For example, the problems including robust statistical inference , formation control , non-autonomous power control , distributed message routing , and spectrum access coordination , can be reduced to variations of this problem. We will be focusing on the case when communication between nodes is directed and time-varying.

Distributed optimization of a sum of convex functions has received a surge of interest in recent years . There is now a considerable theory justifying the use of distributed subgradient methods in this setting, and their performance limitations and convergence times are well-understood. Moreover, distributed subgradient methods have been used to propose new solutions for a number of problems in distributed control and sensor networks . However, the works cited above assumed communications among nodes are either fixed or undirected.

Our paper is the first to demonstrate a working subgradient protocol in the setting of directed time-varying communications. We develop a broadcast-based protocol, termed the subgradient-push, which steers every node to an optimal value under a standard assumption of subgradient boundedness. The subgradient-push requires each node to know its out-degree at all times, but beyond this it needs no knowledge of the graph sequence of even of the number of agents to implement. Our results show that it converges at a rate of $O\left(\ln t/\sqrt{t}\right)$, where the constant depends, among other factors, on the information diffusion speed of the corresponding directed graph sequence and a measure of the influence imbalance among the nodes.

Our work is closest to the recent papers . The papers proposed a distributed subgradient algorithm which is very similar to the one we study here, involving the introduction of subgradients into an information aggregation procedure known as “push-sum.” The convergence of this scheme for directed but fixed topologies was shown in ; implementation of the protocol proposed in these papers appears to require knowledge of the graph or of the number of agents. By contrast, our results work in time-varying networks and are fully distributed, requiring no knowledge of either the graph sequence or the number of agents. The paper shows the convergence of a distributed optimization protocol in continuous time, also for directed but fixed graphs; moreover, an additional assumption is made in that the graph is “balanced.”

All the prior work in distributed optimization, except for , requires time-varying communications with some form of balancedness, often reflected in a requirement of having a sequence of doubly stochastic matrices that are commensurate with the sequence of underlying communication graphs. In contrast, our proposed method removes the need for the doubly stochastic matrices. The proposed distributed optimization model is motivated by applications that are characterized by time-varying directed communications, such as those arising in a mobile sensor network where the links among nodes will come and go as nodes move in and out of line-of-sight or broadcast range of each other. Moreover, if different nodes are capable of broadcasting messages at different power levels, then communication links connecting the nodes will necessarily be unidirectional.

The remainder of this paper is organized as follows. We begin in Section II where we describe the problem of interest, outline the subgradient-push algorithm, and state the main convergence results. Section III is devoted to the proof of a key lemma, namely the convergence rate result for a perturbed version of the so-called push-sum protocol; this lemma is then used in the subsequent proofs of convergence and convergence rate for the subgradient-push in Section IV. Finally, some conclusions are offered in Section VI.

II Problem, Algorithm and Main Results

We consider a network of $n$ nodes whose goal is to solve distributedly the following minimization problem:

We will assume that, at each time $t$, node $i$ can only send messages to its out-neighbors in some directed graph $G(t)$. Naturally, the graph $G(t)$ will have vertex set $\{1,\ldots,n\}$, and we will use $E(t)$ to denote its edge set. Also, naturally, the sequence $\{G(t)\}$ should posses some good long-term connectivity properties. A standard assumption, which we will be making, is that the sequence $\{G(t)\}$ is uniformly strongly connected (or, as it is sometimes called, $B$-strongly-connected), namely, that there exists some ineger $B>0$ (possibly unknown to the nodes) such that the graph with edge set

is strongly connected for every $k\geq 0$. This is a typical assumption for many results in multi-agent control: it is considerably weaker than requiring each $G(t)$ be connected for it allows the edges necessary for connectivity to appear over a long time period and in arbitrary order; however, it is still strong enough to derive bounds on the speed of information propagation from one part of the network to another.

Finally, we introduce the notation $N^{\rm in}_{i}(t)$ and $N^{\rm out}_{i}(t)$ for the in- and out-neighborhoods of node $i$, respectively, at time $t$. We will require these neighborhoods to include the node $i$ itselfAlternatively, one may define these neighborhoods in a standard way of the graph theory, but require that each graph in the sequence $\{G(t)\}$ has a self-loop at every node.; formally, we have

and $d_{i}(t)$ for the out-degree of node $i$, i.e.,

Crucially, we will be assuming that every node $i$ knows its out-degree $d_{i}(t)$ at every time $t$.

Our main contribution is the design of an algorithm which successfully accomplishes the task of distributed minimization of $F({\mathbf{z}})$ assumptions we have laid out above. Our scheme is a combination of the subgradient method and the so-called push-sum protocol, recently studied in the papers . We will refer to our protocol as the subgradient-push method. A subgradient method using the push-sum protocol has been considered in for a fixed communication network, whose implementation requires some knowledge of the graph or the number of agents. In contrast, our algorithm can handle time-varying networks and its implementation requires node $i$ knowing its local out-degree $d_{i}(t)$ only.

We note that the above equations have simple broadcast-based implementation: each node $j$ broadcasts the quantities ${\mathbf{x}}_{j}(t)/d_{j}(t),y_{j}(t)/d_{j}(t)$ to all of the nodes $i$ in its out-neighborhoodWe note that we make use here of the assumption that node $i$ knows its out-degree $d_{i}(t)$., which simply sum all the messages they receive to obtain ${\mathbf{w}}_{i}(t+1)$ and $y_{i}(t+1)$. The update equations for ${\mathbf{z}}_{i}(t+1),{\mathbf{x}}_{i}(t+1)$ can be executed without any further communications among nodes during step $t$.

Without the subgradient term in the final equation, our protocol would be a version of the push-sum protocol for average computation studied recently in . For intuition on the precise form of these equations, we refer the reader to these three papers; roughly speaking, the somewhat involved form of the updates is intended to ensure that every node receives an equal weighting after all the linear combinations and ratios have been taken. In this case the vectors ${\mathbf{z}}_{i}(t+1)$ converge to some common point, i.e., a consensus is achieved. The inclusion of the subgradient terms in the updates of ${\mathbf{x}}_{i}(t+1)$ is intended to steer the consensus point towards the optimal set $Z^{*}$, while the push-sum updates steer the vectors ${\mathbf{z}}_{i}(t+1)$ towards each other. Our main results, which we describe in the next section, demonstrate that this scheme succeeds in steering all vectors ${\mathbf{z}}_{i}(t+1)$ towards the same point in the solution set $Z^{*}$.

II-B Our results

Our first theorem demonstrates the correctness of the subgradient-push method for an arbitrary stepsize $\alpha(t)$ satisfying Eq. (8); this holds under the assumptions we have laid out above, as well as an additional technical assumption on the subgradient boundedness.

The graph sequence $\{G(t)\}$ is uniformly strongly connected.

Then, the distributed subgradient-push method of Eq. (1) with the stepsize satisfying the conditions in Eq. (8) has the following property

Our second theorem makes explicit the rate at which the objective function converges to its optimal value. As standard with subgradient methods, we will make two tweaks in order to get a convergence rate result: (i) we take a stepsize which decays as $\alpha(t)=1/\sqrt{t}$ (stepsizes which decay at faster rates usually produce inferior convergence rates), and (ii) each node $i$ will maintain a convex combination of the values ${\mathbf{z}}_{i}(1),{\mathbf{z}}_{i}(2),\ldots$ for which the convergence rate will be obtained. We then demonstrate that the subgradient-push converges at a rate of $O(\ln t/\sqrt{t})$; this is formally stated in the following theorem. The theorem makes use of the matrix $A(t)$ that captures the weights used in the construction of ${\mathbf{w}}_{i}(t+1)$ and $y_{i}(t+1)$ in Eq. (1), which are defined by

where $S(0)=0$ and $S(t)=\sum_{s=0}^{t-1}\alpha(s+1)$ for $t\geq 1$. Then, we have for all $t\geq 1$, $i=1,\ldots,n$, and any ${\mathbf{z}}^{*}\in Z^{*}$,

The scalars $\lambda$ and $\delta$ are functions of the graph sequence $G(1),G(2),\ldots,$ which are given by

If each of the graphs $G(t)$, $t\geq 1$, is regularA directed graph $G(t)$ is regular if every out-degree and every in-degree of a node in $G(t)$ equals $d(t)$ for some $d(t)$., then

where $A(t)$ is defined by Eq. (10) and $\sigma_{2}(A)$ is the second-largest singular value of a matrix $A$.

Theorem 2 implies that, along the time-averages $\widetilde{\mathbf{z}}_{i}(t)$ for each node $i$, the network objective function $F(z)$ converges to the optimal objective value $F^{*}$, i.e.,

However, the theorem does not state anything about the convergence of the sequences $\{\widetilde{\mathbf{z}}_{i}(t)\}$ for $i=1,\ldots,n$.

Theorem 2 provides the rate at which $F\left(\widetilde{\mathbf{z}}_{i}(t)\right)$ converges to $F^{*}$ for any $i$, which is expected. Specifically, it is standard for a distributed subgradient method to converge at the rate of $O(\ln t/\sqrt{t})$ with the constant depending on the subgradient-norm upper bounds $L_{i}$ and the initial conditions ${\mathbf{x}}_{i}(0)$ . Moreover, it is also standard for the convergence rate of the distributed methods over networks to depend on some measure of the connectivity of the directed graph sequence $G(1),G(2),\ldots$. Namely, here, the closeness of $\lambda$ to $1$ measures the speed at which the (connectivity) graph sequence $\{G(t)\}$ diffuses the information among the nodes over time. Additionally, our rate results also include the parameter $\delta$, which is a measure of the imbalance of influences among the nodes, as we will later see. Time-varying directed regular networks are uniform in influence and will have $\delta=1$, so that $\delta$ will disappear from the bounds entirely; however, networks which have a row very close to zero, corresponding nodes which are only weakly influenced by all others, will suffer a corresponding blow-up in the convergence time of the subgradient-push algorithm. Consequently, $\delta$ may be thought of as a measure of the uniformity of long-term influence among the nodes.

Moreover, while the term $1/(\delta(1-\lambda))$ appearing in our rate estimate is bounded exponentially by $n^{2nB}$ in the worst case, the term need not be this large for every graph sequence. Indeed, Theorem 2 shows that for a class of time-varying regular directed graphs, $1/(\delta(1-\lambda))$ scales polynomially in $n$. Our work therefore motivates the search for effective bounds on consensus speed and the influence imbalances for time-varying directed graphs. Finally, we note that previous research has studied the case when the matrices $A(t)$ in (10) are doubly stochastic, which occurs when the directed graph sequence $\{G(t)\}$ is regular. In this case, our polynomial bounds match the previously known results.

We conclude by briefly summarizing the idea of the proofs as well as the organization of the remainder of this paper. As previously remarked, our protocol is a perturbation of the so-called “push-sum” protocol for averaging studied in . We begin in Section III by showing that as a consequence of well-known facts about consensus protocols, such perturbed push-sum protocols are guaranteed to converge when the perturbations are well behaved (in a sense). In the subsequent Section IV, we interpret the subgradient-push algorithm as a special perturbation of the push-sum protocol and use the convergence results of Section III to show that, after a transient period, our subgradient-push algorithm begins to approximate a centralized subgradient scheme. Finally some simulations are performed in Section V and conclusions are drawn in Section VI.

III Perturbed Push-Sum Protocol

This section is dedicated to the analysis of a perturbed version of the so-called push-sum protocol, originally introduced in the groundbreaking work and recently analyzed in time-varying directed graphs in . The push-sum is a protocol for node interaction which allows nodes to compute averages and other aggregates in the network with directed communication links.

The work in demonstrates the convergence of the push-sum protocol. Here, we generalize this result by showing that the protocol remains convergent even if the state of the nodes is perturbed at each step, as long as the perturbations decay to zero. However, while the iterate sequences (at the nodes) obtained by the push-sum method converge to the average of the initial values of the nodes, the iterate sequences produced by the perturbed push-sum method need not converge to a common point; instead, they converge to each other over time. We will later use this result to prove Theorems 1 and 2. Since the result has a self-contained interpretation and analysis, we sequester it to this section.

We begin with a statement of the perturbed push-sum update rule. Every node $i$ maintains scalar variables $x_{i}(t),y_{i}(t),z_{i}(t),w_{i}(t)$, where $y_{i}(0)=1$ for all $i$. These variables are updated as follows: for $t\geq 0$,

where $\epsilon_{i}(t)$ is a perturbation at time $t$, perhaps adversarially chosen. Recall that $N_{i}^{\rm in}(t)$ is the in-neighborhood of node $i$ in a directed graph $G(t)$ and $d_{j}(t)$ is the out-degree of node $j$, as defined in Section II.

Without the perturbation term $\epsilon_{i}(t)$, the method in Eq. (11) reduces to the push-sum protocol. Moreover, our subgradient-push method of Eq. (1) is simply a vector-space analog of the perturbed pus-sum method of Eq. (11) with a specific choice of the perturbations.

The intuition behind the push-sum equations of Eq. (11) is somewhat involved. This dynamic has been introduced for the purpose of average computation (in the case when all the perturbations $\epsilon_{i}(t)$ are zero) and has a simple motivating intuition. The push-sum is a variation of a consensus-like protocol wherein every node updates its values by taking linear combinations of the values of its neighbors. Due to taking linear combinations, some nodes are bound to be more influential than others (meaning that other nodes end up placing larger weights on their information), for example by virtue of being more centrally placed. To cancel out the effect of these influence imbalances of the nodes, the ratios $z_{i}(t)=w_{i}(t)/y_{i}(t)$ are used, which ensures that each $z_{i}(t)$ converges to $(1/n)\sum_{i=1}^{n}x_{i}(0).$ We refer the reader to for more details.

Next, we rewrite the perturbed push-sum equations more compactly by using the definition of the matrix $A(t)$ in Eq. (10). Then, the relations in Eq. (11) assume the following form: for all $t\geq 0$,

where $\epsilon(t)=(\epsilon_{1}(t),\ldots,\epsilon_{n}(t))^{\prime}$. Each of matrices $A(t)$ is column-stochastic but not necessarily row-stochastic.

We are concerned with demonstrating a convergence result and a convergence rate for the updates given in Eq. (11), or equivalently Eq. (18). Specifically, the bulk of this section is dedicated to proving the following lemma.

Consider the sequences $\{z_{i}(t)\}$, $i=1,\ldots,n,$ generated by the method in Eq. (11). Assuming that the graph sequence $\{G(t)\}$ is uniformly strongly connected, the following statements hold:

where $\delta>0$ and $\lambda\in(0,1)$ satisfy

If each of the matrices $A(t)$ in Eq. (10) are doubly stochastic, then

If $\lim_{t\to\infty}\epsilon_{i}(t)=0$ for all $i=1,\ldots,n$, then

If $\{\alpha(t)\}$ is a non-increasing positive scalar sequence with $\sum_{t=1}^{\infty}\alpha(t)|\epsilon_{i}(t)|<\infty$ for all $i$, then

For part (b) of Lemma 1, observe that each of the matrices $A(t)$ is doubly stochastic if each of the graphs $G(t)$ is regular. Furthermore, observe that if $\epsilon_{i}(t)=0$, this lemma implies that the push-sum method converges at a geometric rate. In this case, it is easy to see that ${\bf 1}^{\prime}x(t)/n={\bf 1}^{\prime}x(0)/n$ and, therefore $z_{i}(t)\rightarrow{\bf 1}^{\prime}x(0)/n$, so that the push-sum protocol successfully computes the average of the initial values. When the perturbations are nonzero, Lemma 1 states that if these perturbations decay to zero, then the push-sum method still converges. Of course, it will no longer be true that the convergence is necessarily to the average of the initial values.

We will prove a series of auxiliary lemmas before beginning the proof of Lemma 1. We first remark that the matrices $A(t)$ have a special structure that allows us to efficiently analyze their products. Specifically, we have the following properties of the matrices $A(t)$ (see for proofs of this and similar statements).

Suppose that the graph sequence $\{G(t)\}$ is uniformly strongly connected. Then for each integer $s\geq 0$, there is a stochastic vector $\phi(s)$ such that for all $i,j$ and $t\geq s,$

There are known bounds on the parameters $C,\lambda$ in Lemma 2, which characterize how large $C$ is and how far away $\lambda$ is from $1$. Moreover, these bounds improve if the sequence $\{G(t)\}$ has some nice properties. The following lemma is a formal statement to this effect.

Let the graph sequence $\{G(t)\}$ be uniformly strongly connected. Then, in the statement of Lemma 2(b) we have

If in addition every graph $G(t)$ is regular, we have

We remark that this lemma is not novel relative to the previous literature, but rather is an explicit statement of the bounds that have been implicitly used in the previous papers on the subject.

From , under the assumption of the uniform strong connectivity of the graphs and our definition of neighborhoods, we have that: if

The preceding relation implies that for all $t>s\geq 0$,

This holds for every $x(s)$. By choosing $x(s)$ to be each of the $n$ basis vectors, we see that for every $j=1,\ldots,n$,

Since each matrix $A^{\prime}(t)$ is row-stochastic, the entry $\phi_{j}(s)$ is a limit of the convex combinations of the $n$ numbers $[A^{\prime}(t)\cdots A^{\prime}(s)]_{ij},i=1,\ldots,n$, as $t\to\infty$. Hence, we have proven the first relation of the lemma for $t>s$. For $t=s$, since the matrix $A(s)$ is column-stochastic and $\phi(s)$ is a stochastic vector, we obviously have $|[A^{\prime}(s)]_{ij}-\phi_{j}(s)|\leq 2$ for all $i,j$, showing that the relation also holds for $t=s$.

As for the second statement, when the graphs $G(t)$ are regular, each of the matrices $A(t)$ is doubly stochastic. Then, the results of imply that: if

where $\bar{x}_{s}$ is the average of the entries of $x(s)$. From the preceding inequality, we can see that for all $t>s\geq 0$,

Since $\sqrt{1-\beta/2}\leq 1-\beta/4$ for all $\beta\in(0,1)$, it follows that we may choose $C=\sqrt{2}$ and $\lambda=(1-1/(4n^{3}))^{1/B}$. The same line of argument can be used to show that we may choose $C=1$ and $\lambda=\max_{t\geq 0}{{\sigma_{2}(A(t))}}$. ∎

We will end up applying this lemma to a sequence of graphs which is only uniformly strongly connected after throwing out a few graphs at the start. To that end, we have the following corollary whose proof is straightforward.

Suppose that the graph sequence $G(t)$ has the following property: there exists an integer $T>0$ such that given any integer $s\geq 0$, there is a time $0\leq t_{s}\leq T$ for which the graph sequence $G(s+t_{s}),G(s+t_{s}+1),G(s+t_{s}+2),\ldots$ is uniformly strongly connected. Then, for each integer $s\geq 0$ there is a stochastic vector $\phi(s)$ such that for all $i,j$ and $t\geq s$,

where $C,\lambda$ may be taken as in Lemma 3.

In the proof of Lemma 1, we will make use of a lower bound on the entries of the vectors ${\bf 1}^{\prime}A(t)\cdots A(0)$. To provide such a bound, we employ the following intermediate result which provides a uniform lower bound on the entries of the vectors ${\bf 1}^{\prime}A^{\prime}(t)\cdots A^{\prime}(0)$.

Given a graph sequence $\{G(t)\}$, define

If the graph sequence $\{G(t)\}$ is uniformly strongly connected, then $\delta^{\prime}\geq\frac{1}{n^{nB}}.$ If each $G(t)$ is regular, then $\delta^{\prime}=1$.

By the definition of matrices $A(t)$ in Eq. (10), we have $[A(t)]_{ii}=1/d_{i}(t)$. Since $d_{i}(t)\leq n$, it follows that $[A(t)]_{ii}{{\geq}}1/n$ for all $t$ and $i$. Therefore, for all $i$,

Thus, we certainly have $[{\bf 1}^{\prime}A^{\prime}(t)\cdots A^{\prime}(1)]_{i}\geq 1/n^{nB}$ for all $i$ and all $t$ in the range $1\leq t\leq n^{nB}$. However, it was shown in that for $t>(n-1)B$, every entry of $A^{\prime}(t)\cdots A^{\prime}(1)$ is positive and has value at least $1/n^{nB}$. Since $n^{nB}>(n-1)B$, this proves the bound $\delta^{\prime}\geq 1/n^{nB}$. The final claim that $\delta^{\prime}=1$ for a sequence of regular graphs is trivial. ∎

As an immediate consequence of the definition of $\delta^{\prime}$, we have $\phi_{j}(s)\geq\delta^{\prime}/n$ for all $j$.

By taking transposes and applying Lemmas 2, 3, 4, we immediately obtain the following result on the products $A(t)\cdots A(s)$. For convenience, let us adopt the notation of denoting these products by $A(t:s)$, i.e.,

Let the graph sequence $\{G(t)\}$ be uniformly strongly connected. Then, the following statements are valid:

If in addition each $G(t)$ is regular, we may choose

whenever $\sup_{t\geq 0}{{\sigma_{2}(A(t))}}<1$.

Moreover, if the graphs $G(t)$ are regular, we have $\delta=1$.

The stochastic vectors $\phi(t)$ satisfy for all $j,$

The results follow by employing Corollary 1, Lemmas 3, 4, and Remark 2, where we take the transposes of the matrices. The only aspect that needs to be verified is that these lemmas can be applied, which is routine, with the exception of the issue of the uniform strong connectivity requiring elaboration.

When we transpose $A(t:s)$ to obtain the product $A^{\prime}(s)\cdots A^{\prime}(t-1)A^{\prime}(t)$, we have reversed the order in which the matrices appear. Moreover, by taking the transposes of each matrix, we have effectively reversed the direction of every edge in each graph $G(t)$. In the case when $\sup_{t\geq 0}\sigma_{2}(A(t))<1$, it is easy to see that $B=1$ and the resulting “reversed” sequence is connected at every step, so the bound of Lemma 3 applies verbatim. In the other cases, the resulting sequence is still strongly connected once we throw out at most $B$ graphs from the start of the sequence. The bounds of Corollary 1 thus apply with $t-s$ replaced by $t-s-B$, which we take care of by instead doubling the constant $C$. ∎

The parameter $\delta$ as defined in Lemma 4 may be thought of as a measure of imbalance of the influence among the nodes. Indeed, $\delta$ is defined as the best lower bound on the row sums of the matrices $A(t:s)$. In the case when each of the graphs $G(t)$ is regular, the matrices $A(t)$ will be doubly stochastic and we will have $\delta=1$ as previously remarked (i.e., no imbalance of influence). By contrast, when $\delta\approx 0$, there is a node $i$ such that the $i$’th row in some $A(t:s)$ will have entries which are all nearly zero; in short, it is almost as if node $i$ has no in-neighbors at all.

We proceed with our sequence of intermediate lemmas for the proof of Lemma 1. We will now need some auxiliary results on the convolution of two scalar sequences, as in the following statement.

( Lemma 3.1) Let $\{\gamma_{k}\}$ be a scalar sequence.

With these pieces in places, we can now proceed to the proof of Lemma 1. Our argument will rely on Corollary 2 on the products $A(t:s)$ and the just-stated Lemma 5 on the convolution sequences.

(a) By inspecting Eq. (18) it is easy to see that for all $t\geq 0$,

Moreover, since each $A(t)$ is column-stochastic, we have that ${\bf 1}^{\prime}A(t)={\bf 1}^{\prime}$ and Eq. (25) further implies that

Now, from Eq. (26) and Eq. (27) we obtain for all $t\geq 0$,

According to Corollary 2, if we define $D(t,s)$ to be

where the constants $C>0$ and $\lambda\in(0,1)$ have the properties listed in Corollary 2.

Therefore, from relation (28) it follows for $t\geq 0$,

We may derive a similar expression for $y(t+1)$:

which holds for all $t\geq 0$. From (32) and (34) we obtain for every $t\geq 1$ and all $i$,

By bringing the fractions to a common denominator, after the cancellation of some terms, we find that

Observe that the denominator of the above fraction is $n$ times the $i$’th row sum of $A(t:0)$. By definition of $\delta$, this row sum is at least $\delta$, and consequently

Factoring $n$ out in the last term, and using estimates for $|[D(t:s)]_{ij}|$ as given in (31), we obtain

Now we look at the term ${\bf 1}^{\prime}x(t)$. From Eq. (27), we have

From the preceding two relations it follows that for all $i$ and $t\geq 1$,

Since we were able to choose $C\leq 4$ in all the cases considered in Lemma 2, we may choose $C=4$ to obtain the result in part (a).

(b) By letting $t\to\infty$ in the preceding relation, since $\lambda\in(0,1)$, we find that for all $i,$

When $\epsilon_{i}(t)\to 0$ for all $i$, then $\|\epsilon(t)\|_{1}\to 0$ and, by Lemma 5(a), we conclude that

and the result follows from the preceding two relations.

$($c) Since $\{\alpha(t\}$ is positive and non-increasing sequence, we have $\alpha(t+1)\leq\alpha(1)$ for all $t\geq 0$ and $\alpha(t+1)\leq\alpha(s)$ for all $t\geq s\geq 0$. Using these relations, we obtain

Lemma 1, which we have just proved, is the central result of this section. It states that each of the sequences $z_{i}(t+1)$ tracks the average $\bar{x}(t)={\bf 1}^{\prime}x(t)/n$ increasingly well as time goes on. We will later require a corollary of this lemma showing that a weighted time-average of each $z_{i}(t+1)$ tracks a weighted time-average of $\bar{x}(t)$. The next corollary gives a precise statement of this result. In the derivation, we also use the following inequality

and the relation $2\left(\sqrt{t+2}-1\right)\geq\sqrt{t+1}$ for all $t\geq 1$.

Suppose that all the assumptions of Lemma 1 are satisfied. Moreover, let $\alpha(t)=1/\sqrt{t}$ for all $t\geq 1$ and the perturbations $\epsilon_{i}(t)$ are bounded as follows:

we have for every $i=1,\ldots,n$ and $t\geq 1$,

where $\delta\in(0,1]$ and $\lambda\in(0,1)$ are as given in Lemma 1.

From Lemma 1(a) we have for all $i$ and all $t\geq 1$,

From the definition of the perturbed push-sum method, we have

where the last equality follows from $y_{i}(0)=1$ for all $i$. Thus, $z_{i}(1)$ is a weighted average of the entries in $x(0)$, while $\bar{x}(0)$ is the average of these entries. Hence,

By summing the relations in Eqs. (44) and (46), we obtain the estimate in Eq. (40). The relation in Eq. (42) follows from $\alpha(t)=1/\sqrt{t}$ and Eqs. (39)–(40). ∎

We conclude this section by noting that Lemma 1 and Corollary 3 can be extended to the case when $x_{i}(t)$ (and, by extension, $z_{i}(t)$) is a $d$-dimensional vector, by applying the results to each coordinate component of the space.

IV Convergence Results for Subgradient-Push Method

We turn now to the proofs of our main results, namely Theorems 1 and 2. Our arguments will crucially rely on the convergence results for the perturbed push-sum method we have established in Section III.

We give a brief, informal summary of the main ideas behind our argument. The convergence result for the perturbed push-sum method of Section III implies that, under the appropriate assumptions, the entries of ${\mathbf{z}}_{i}(t)$ get close to each other over time, and consequently ${\mathbf{z}}_{i}(t)$ approaches a multiple of the all-ones vector. Thus, every node takes a subgradient of its own function $f_{i}$ at nearly the same point. Over time, these subgradients are averaged over the nodes by the push-sum-like updates of our method. Consequently, the subgradient-push method approximates the ordinary (centralized) subgradient algorithm applied to the average function $\frac{1}{n}\sum_{j=1}^{n}f_{j}$.

We now begin the formal process of proving Theorems 1 and 2. In what follows, we will use a deterministic counterpart of the well-known (almost) supermartingale convergence result (; see also , Lemma 11, Chapter 2.2). The result is given in the following lemma.

Let $\{v_{t}\}$ be a non-negative scalar sequence such that

where $b_{t}\geq 0,$ $u_{t}\geq 0$ and $c_{t}\geq 0$ for all $t\geq 0$ with $\sum_{t=0}^{\infty}b_{t}<\infty$, and $\sum_{t=0}^{\infty}c_{t}<\infty$. Then, the sequence $\{v_{t}\}$ converges to some $v\geq 0$ and $\sum_{t=0}^{\infty}u_{t}<\infty$.

Our first step is to establish a lemma pertinent to the convergence of a sequence satisfying a subgradient-like recursion. In the proof of this lemma, we make use of Lemma 6.

where $b_{t}\geq 0,$ $\alpha_{t}\geq 0$ and $c_{t}\geq 0$ for all $t\geq 0$, with $\sum_{t=0}^{\infty}b_{t}<\infty$, $\sum_{t=0}^{\infty}\alpha_{t}=\infty$ and $\sum_{t=0}^{\infty}c_{t}<\infty$. Then, the sequence $\{x_{t}\}$ converges to some solution $x^{*}\in X^{*}$.

Thus, all the conditions of Lemma 6 are satisfied, and by this lemma we obtain the following statements:

Since $\sum_{t=0}^{\infty}\alpha_{t}=\infty$, it follows from (48) that

A key relations in the proofs of Theorems 1 and 2 will be obtained by applying Lemma 7 to the average process

where ${\mathbf{x}}_{i}(t)$ is the sequence generated by the subgradient-push method. We will need to argue that $\bar{\mathbf{x}}(t)$ satisfies the assumptions of Lemma 7, for which the following result will be instrumental.

Since the subgradient norms of each $f_{i}$ are uniformly bounded by $L_{i}$, it further follows that for all $t\geq 0$,

We next consider a cross-term ${\mathbf{g}}^{\prime}_{i}(t+1)(\bar{\mathbf{x}}(t)-{\mathbf{v}})$ in (51). For this term, we write

Using the subgradient boundedness and the Cauchy-Schwarz inequality, we can lower bound the first term ${\mathbf{g}}_{i}^{\prime}(t+1)(\bar{\mathbf{x}}(t)-{\mathbf{z}}_{i}(t+1))$ as

As for the second term ${\mathbf{g}}_{i}^{\prime}(t+1)({\mathbf{z}}_{i}(t+1)-{\mathbf{v}})$, we can use the fact that ${\mathbf{g}}_{i}^{\prime}(t+1)$ is the subgradient of $f_{i}(\theta)$ at $\theta={\mathbf{z}}_{i}(t+1)$ to obtain:

from which, by adding and subtracting $f_{i}\left(\bar{\mathbf{x}}(t)\right)$ and using the Lipschitz continuity of $f_{i}$ (implied by the subgradient boundedness), we further obtain

By substituting the estimates of Eqs. (56)–(58) back in relation (54), and using $F({\mathbf{x}})=\sum_{i=1}^{n}f_{i}({\mathbf{x}})$ we obtain

With all the pieces in place, we are finally ready to prove Theorem 1. The proof idea is to show that the averages $\bar{\mathbf{x}}(t)$, as defined in Lemma 8, converge to some solution $x^{*}\in Z^{*}$ and then show that ${\mathbf{z}}_{i}(t+1)-\bar{\mathbf{x}}(t)$ converges to 0 for all $i$, as $t\to\infty$. The last step will be accomplished by invoking Lemma 1 on the perturbed push-sum protocol.

Therefore, since by assumption $\sum_{t=1}^{\infty}\alpha(t)^{2}<\infty$ and $\|{\mathbf{g}}_{i}(t)\|\leq L_{i}$, we obtain for all $i=1,\ldots,n$,

Now, observe that we can apply Lemma 8 with ${\mathbf{v}}=z^{*}$ for any solution $z^{*}\in Z^{*}$ to obtain

where $F^{*}$ is the optimal value (i.e., $F^{*}=F(z^{*})$ for any $z^{*}\in Z^{*}$). In view of Eq. (62), it follows that

Having proven Theorem 1, we now turn to the proof of the convergence rate results of Theorem 2. The first step will be a slight modification of the result we have just proved - whereas the proof of Theorem 1 relies on $F(\bar{\mathbf{x}}(t))\rightarrow F^{*}$, in order to obtain a convergence rate we will now need to argue that we can replace $F(\bar{\mathbf{x}}(t))$ by $F$ evaluated at a running average of the vectors ${\mathbf{z}}_{i}(t)$ for any $i$. This is stated precisely in the following lemma.

If all the assumptions of Theorem 1 are satisfied and $\alpha(t)=1/\sqrt{t}$, then for all $t\geq 1$ and any $z^{*}\in Z^{*}$,

From Lemma 8 we have for any ${\mathbf{v}}$ and $t\geq 0$,

Now setting ${\mathbf{v}}={\mathbf{z}}^{*}$ for some ${\mathbf{z}}^{*}\in Z^{*}$ and using the convexity of $F$, we obtain

(cf. Eq. (40) of Corollary 3), which after some elementary algebra implies that

Now, using the fact that the Euclidean norm of a vector is not larger that its $1$-norm, we substitute Eq. (71) into Eq. (67). Then, using the definition of $S(t)$ and Eq. (39) we bound the denominator in Eq. (67) as follows

We are now finally in position to prove Theorem 2. At this point, the proof is a simple combination of Lemma 9, which tells us that $F\left(\frac{\sum_{k=0}^{t}\alpha(k+1)\bar{\mathbf{x}}(k)}{\sum_{k=0}^{t}\alpha(k+1)}\right)$ approaches $F(z^{*})$, along with Corollary 3, which tells us that $\frac{\sum_{k=0}^{t}\alpha(k+1)\bar{\mathbf{x}}(k)}{\sum_{k=0}^{t}\alpha(k+1)}$ and $\frac{\sum_{k=0}^{t}\alpha(k+1){\mathbf{z}}_{i}(k+1)}{\sum_{k=0}^{t}\alpha(k+1)}$ get close to each other over time for all $i=1,\ldots,n$.

It is easy to see by induction that the vectors $\widetilde{\mathbf{z}}_{i}(t)$ defined in the statement of Theorem 2 are weighted time-averages of ${\mathbf{z}}_{i}(t)$, given by: for all $i$,

By the boundedness of subgradients we obtain for all $i$ and $t\geq 0$,

Applying Corollary 3 to each of the coordinates of the vectors ${\mathbf{z}}_{i}(k+1)$ and $\bar{\mathbf{x}}(t)$, we obtain for all $i$ and $t\geq 1$,

By summing the preceding relation and that of Lemma 9, we have

V Simulations

We report some simulations of the subgradient-push method which experimentally demonstrate that its performance is often quite scalable. We will be optimizing the scalar function $F(\theta)=\sum_{i=1}^{n}p_{i}(\theta-u_{i})^{2}$ where $u_{i}$ is a variable that is known only to node $i$. This is a canonical problem in distributed estimation: the nodes are attempting to measure a parameter $\widehat{\theta}$, and node $i$ measures $u_{i}=\widehat{\theta}+w_{i}$ where $w_{i}$ are jointly Gaussian and zero mean. Letting $p_{i}$ be the inverse of the variance of $w_{i}$, the maximum likelihood estimate is the minimizer $\theta^{*}$ of $F(\theta)$ ($\theta^{*}$ is unique provided that $p_{i}>0$ for at least one $i$). We set a half of the $p_{i}$’s to zero (corresponding to nodes not taking any measurements) and the other half of the $p_{i}$’s to uniformly random values between and $1$. The initial points $x_{i}(0)$ are independent random variables, each with a standard Gaussian distribution. Figure 1 shows the results for simple random graphs where every node has two out-neighbors, one belonging to a fixed cycle and the other one chosen uniformly at random at each step. The plot to the left shows how $\|z(t)-\theta^{*}{\bf 1}\|$ decays on a graph instance with $n=1000$ nodes, while one to the right shows the (average) time until $\|z(t)-\theta^{*}{\bf 1}\|\leq 0.1$ as the size $n$ of the graph varies from 10 to 90. Figure 2 illustrates the same quantities for the sequence of graphs which alternate between two (undirected) star graphs.

We see that in both cases the initial decay is quite fast and it takes only a small number of iterations to bring all the nodes reasonably close to the optimal value. Nevertheless, the decay is clearly sub-geometric, consistently with the bounds we have proven. However, in both cases, the time to get the error below the threshold of $0.1$ starting from random initial points appears to scale linearly in the number of nodes.

VI Conclusions

We have introduced the subgradient-push, a broadcast-based distributed protocol for distributed optimization of a sum of convex functions over directed graphs. We have shown that, as long as the communication graph sequence $\{G(t)\}$ is uniformly strongly connected, the subgradient-push succeeds in driving all the nodes to the same point in the set of optimal solutions. Moreover, the objective function converges at a rate of $O(\ln t/\sqrt{t})$, where the constant depends on the initial vectors, bounds on subgradient norms, consensus speed $\lambda$ of the graph sequence $\{G(t)\}$, as well as a measure of the imbalance of influence $\delta$ among the nodes.

Our results motivate the open problems associated with understanding how the consensus speed $\lambda$ depends on properties of the sequence $\{G(t)\}$. Similarly, it is also natural to ask how the measure of imbalance of influence $\delta$ depends on the combinatorial properties of the graphs $G(t)$, namely how it depends on the diameters, the size of the smallest cuts, and other pertinent features of the graphs in the sequence $\{G(t)\}$.

References