Missing and spurious interactions and the reconstruction of complex networks

General reliability formalism

Consider an observed network with adjacency matrix $A^{O}$; $A^{O}_{ij}=1$ if nodes $i$ and $j$ are connected and $0$ otherwise. We assume that this observed network is a realization of an underlying probabilistic model, either because the network itself is the result of a stochastic process, because the measurement has uncertainty, or both For simplicity, in this manuscript we use language that is consistent with a situation in which a true network exists but is obscured by the inaccuracies of the observation process. Thus, we talk about the “true” network, which has no “errors,” and about “observed” networks, which have “errors.” However, the formalism is valid even if the network is itself the outcome of a stochastic process.. Let us call $\mathcal{M}$ the set of generative models that could conceivably give rise to the observed network, and $p(M|A^{O})$ the probability that $M\in\mathcal{M}$ is the model that gave rise to the observation $A^{O}$. If we could get a new observation of the network, the outcome would in general be different from $A^{O}$; our best estimate for the probability $p(X=x)$ for an arbitrary network property $X$ is

where $p(X=x|M)$ is the probability that $X=x$ in a network generated with model $M$. Using Bayes theorem, we can rewrite Eq. (1) as

where $p(A^{O}|M)$ is the probability that model $M$ gives rise to $A^{O}$ among all possible adjacency matrices, and $p(M)$ is the a priori probability that model $M$ is the correct one. We call $p(X=x|A^{O})$ the reliability of the $X=x$ measurement.

Stochastic block models

Given the generality of these arguments, the key to good estimates of reliability is to identify sets of models that are general, empirically grounded, and analytically or computationally tractable. Here, we focus on the family $\mathcal{M}_{\rm BM}$ of stochastic block models . In a stochastic block model, nodes are partitioned into groups and the probability that two nodes are connected depends only on the groups to which they belong (Fig. 1).

Stochastic block models are empirically grounded in that they capture two ubiquitous and fundamental properties of real complex networks. First, nodes in real networks are often organized into modules or communities , which may overlap or be hierarchically nested , so that connections are relatively more abundant within modules than between modules. In most real-world networks, this modularity is significantly larger than expected from chance . Second, nodes in real networks fulfill distinct roles and connect to each other depending on these roles . Role-to-role connections are not necessarily assortative , that is, nodes with a certain role may or may not tend to connect with other nodes with the same role. In general, stochastic block models are particularly appropriate when nodes belong to groups and interact with each other depending on their group membership (regardless of whether interactions occur mostly within groups or between groups).

Stochastic block models are also appropriate in that they can capture other more general connectivity correlations in the network. For example, if people establish social connections with others according to age, then a block model that partitions individuals into age groups will capture some of the correlations in the network.

In general, complex networks result from a combination of mechanisms, including modularity, role structure, and maybe other factors. Although partitions into modules, roles, and age groups, for example, can be very different from each other, some block model in the $\mathcal{M}_{\rm BM}$ family is likely to capture each of them separately; by sampling over all models $M\in\mathcal{M}_{\rm BM}$ we capture a variety of correlations, ideally to the exact degree that they are relevant.

Additionally, stochastic block models are analytically tractable because in a stochastic block model the probability that nodes $i$ and $j$ are connected depends only on the groups to which they belong . Therefore, we can calculate the reliability of individual links and the reliability of entire networks.

Link reliability: missing and spurious interactions

The reliability of an individual link is $R^{L}_{ij}\equiv p_{\rm BM}(A_{ij}=1|A^{O})$, that is, the probability that the link “truly” exists given our observation of the whole network (and our choice of the family of stochastic block models). Assuming no prior knowledge about the suitability of the models, we obtain (Methods)

where the sum is over partitions $P$ in the space $\mathcal{P}$ of all possible partitions of the network into groups, $\sigma_{i}$ is node $i$’s group (in partition $P$), $l^{O}_{\alpha\beta}$ is the number of links in the observed network between groups $\alpha$ and $\beta$, and $r_{\alpha\beta}$ is the maximum possible number of links between $\alpha$ and $\beta$ (Fig. 1). The function $\mathcal{H}(P)$ is a function of the partition

and $Z=\sum_{P\in\mathcal{P}}\exp[-\mathcal{H}(P)]$.

In practice, it is not possible to sum over all partitions even for small networks The number of distinct partitions of $N$ elements into groups is $\sum_{k=1}^{N}\frac{1}{k!}\sum_{l=1}^{k}\binom{k}{l}\,(-1)^{k-l}\,l^{N}$, which grows faster than any finite power of $N$.. However, since Eq. (3) has the same mathematical form as an ensemble average in statistical mechanics , one can use the Metropolis algorithm to correctly sample relevant partitions (that is, partitions that significantly contribute to the sum) and obtain estimates for the link reliability (Methods).

We use the link reliability to identify missing and spurious interactions in network observations. We evaluate the performance of our approach using five high-quality networks: the social network of interactions between people in a karate club , the social network of frequent associations between 62 dolphins , the air transportation network of Eastern Europe , the neural network of the nematode C. elegans , and the metabolic network of E. coli . All of these networks have been manually curated and are widely used in the literature as model systems. Therefore, in what follows we assume that each of these networks is the “true” network $A^{T}$ and is error-free. We then generate hypothetical observations $A^{O}$ by adding or removing random connections from $A^{T}$, and evaluate the ability of our approach to recover the features of the true network By adding and removing connections in this way, we are implicitly focusing on random errors; we discuss at the end how our approach can also deal with systematic (or, in general, correlated) errors..

To quantitatively study missing interactions, we generate observed networks $A^{O}$ by removing random links from the true network $A^{T}$. We then estimate the link reliability $R_{ij}^{L}$ for each of these false negatives ($A^{O}_{ij}=0$ and $A^{T}_{ij}=1$), as well as for the true negatives ($A^{O}_{ij}=0$ and $A^{T}_{ij}=0$). We measure the algorithm’s ability to identify missing interactions by ranking the reliabilities (in decreasing order) and calculating the probability that a false negative has a higher ranking than a true negative . Similarly, we quantify the ability to identify spurious interactions by adding random links to the true network, obtaining and ranking the link reliabilities (again, in decreasing order), and calculating the probability that a false positive ($A^{O}_{ij}=1$ and $A^{T}_{ij}=0$) is ranked lower than a true positive ($A^{O}_{ij}=1$ and $A^{T}_{ij}=1$).

In Fig. 2, we compare our approach to the hierarchical random graph (HRG) approach of Clauset et al. and to a local algorithm based on the number of common neighbors between each pair of nodes (Methods; see Supporting Information, Fig. S1, for a comparison to other local algorithms). We find that, except for one network, our approach consistently outperforms all others at identifying both missing interactions and spurious interactions. Our approach is also the only one that performs consistently well for all networks (unlike local algorithms, which work well for some networks but very poorly for others) and for both missing and spurious interactions (unlike the HRG algorithm, which performs comparatively worse at detecting spurious interactions A plausible explanation for this behavior is that, because in the HRG model most parameters are used to “fit” low-level features of the network (pairs of nodes, triplets of nodes, and so on), the HRG approach may overfit spurious links.). Our algorithm is also consistently the most accurate when applied to a number of model networks, including networks with hierarchically nested modules, networks with a strongly disassortative role structure, and non-modular scale-free networks (Supporting Information, Fig. S2). We find that only when the network is strictly a hierarchical random graph, is the HRG approach slightly more accurate at predicting missing interactions than the BM approach (Supporting Information Secs. 2 and 3). Remarkably, even for strict hierarchical random graphs, the BM approach is more accurate at identifying spurious interactions.

Network reliability and network reconstruction

The success at detecting both missing and spurious interactions confirms that our approach is able to uncover the structural features of the true network $A^{T}$. The natural question is thus whether it is possible to “reconstruct” the observation $A^{O}$ to gain greater insight into the global structure of $A^{T}$. This is difficult because, in general, adding a few candidate missing interactions and removing a few candidate spurious interactions does not give satisfactory network reconstructions (one of the main problems being that one does not know, a priori, how many missing and spurious interactions there are).

Therefore, the first step toward network reconstruction is to obtain the network reliability $R^{N}_{A}\equiv p_{\rm BM}(A|A^{O})$, that is, the probability that $A$ is the true network given our observation $A^{O}$ (and our choice of the family of stochastic block models). We obtain (Methods)

$l_{\alpha\beta}$ is the number of links in $A$ between groups $\alpha$ and $\beta$, and $\mathcal{H}(P)$ and $Z$ are the same as in Eq. (3). Once more, we use the Metropolis algorithm to estimate $R^{N}_{A}$.

Given the network reliability $R^{N}_{A}=p_{\rm BM}(A|A^{O})$, the expected value of a property $X$

over all possible networks $A$ is a better estimate of $X(A^{T})$ than $X(A^{O})$. We find that in many situations $R^{N}_{A^{T}}\gg R^{N}_{A^{O}}$ (Supporting Information, Fig. S13), which means that, presented only with an inaccurate observation $A^{O}$ (and with the knowledge about complex networks embodied in the stochastic block model family), our approach is remarkably able to identify that $A^{T}$ is a more likely network than the observation $A^{O}$ itself. This confirms that, even without knowing $A^{T}$, it is possible to estimate a property $X(A^{T})$ better than just by measuring that property on $A^{O}$ (that is, better than assuming $X(A^{T})=X(A^{O})$).

Since summing over all possible networks in Eq. (7) is prohibitive, we use the approximation $\langle X\rangle\approx X(A^{R})$, where $A^{R}$ is the network that maximizes $R^{N}_{A}$ (in other words, $A^{R}$ is the maximum a posteriori estimate of $A$). The network $A^{R}$ is what we call a network reconstruction, and we claim that $X(A^{R})$ is, in general, a better estimate of $X(A^{T})$ than $X(A^{O})$. In practice, we build reconstructions by heuristically maximizing $R^{N}_{A}$, starting from $A^{O}$ (Methods).

We test our network reconstruction approach by generating hypothetical observed networks $A^{O}$ from the true test networks $A^{T}$ described above. Each observation has a fraction of the true interactions removed (we call this fraction the observation error rate), and an identical number of random interactions added. In Fig. 3 we show the true air transportation network of Eastern Europe, as well as a hypothetical observation of this network (with an observation error rate of 20%) and the corresponding reconstruction. The reconstruction has 13% fewer missing and spurious interactions than the observation and, qualitatively, it appears that individual node properties (specifically, degree and betweenness centrality) are also better captured by the reconstruction.

However, from a systems perspective global network properties are more relevant than local node-level features. Therefore, the ultimate goal is to generate network reconstructions whose global properties are closer to those of the true network than those of the observations. To quantitatively investigate whether our approach accomplishes this aim, we calculate six network properties (static and dynamic) for observations and for the corresponding reconstructions of the air transportation network of Eastern Europe, and compute the relative error with respect to the true value. As we show in Fig. 4, the reconstruction consistently improves the estimates of these properties. Only when the observed network contains less than 10% of errors it is better, for a few of the properties, to use the observed network rather than the reconstruction. We obtain similar results for other networks and other network properties (Supporting Information, Figs. S8-S12).

Application to a protein interaction network

As we have discussed before, protein interaction networks are among the networks that may benefit the most from our approach. Ultimately, only experiments can prove our results useful; such experiments are, however, beyond the scope of this work. Nevertheless, here we show, however, how our approach can help in directing the effort to refine protein interaction data.

We consider the protein interaction network of yeast that Gavin et al. obtained using affinity purification and mass spectrometry (AP/MS) . In AP/MS essays, a “bait” protein is used to detect “prey” proteins that interact with the bait directly or indirectly. Since bait and prey play different roles, we limit ourselves to the set of 991 proteins that are both viable baits and viable prey (for example, we discard proteins that only appear as prey because prey-prey interactions cannot possibly be observed). We build a protein interaction network by connecting all pairs of proteins that are reported as a bait-prey pair at least once Note that we do not advocate that this is the most appropriate procedure to analyze the structure of a protein interaction network (see for a detailed discussion). Rather, we use this procedure because it enables us to test whether our algorithm can separate the least reliable and most reliable interactions.. From this network, we obtain the reliability for all pairs of proteins.

We evaluate how successful our algorithm is by considering those proteins among the 991 in the network that have been used once, and only once, as bait (some proteins are used as bait in several independent essays). For a pair of these proteins $A$ and $B$, a link in the network can represent two distinct situations: (i) the interaction was observed once (with $A$ as bait and $B$ as prey but not the other way around, or vice versa); (ii) the interaction was observed twice (both with $A$ as bait and with $B$ as bait). Since these experimentally “non-reproducible” and “reproducible” interactions (3113 and 867 interactions in our network, respectively) are encoded identically in the network, it is interesting to see if our algorithm assigns lower reliability to the first and higher to the latter.

Remarkably, among the 100 interactions with the lowest link reliability according to our algorithm, only 5 are experimentally reproducible. Conversely, among the 100 interactions with the highest link reliability, as many as 65 are experimentally reproducible. The probabilities of observing by chance such a small number in the first case and such a large number in the latter case are $p_{\leq}=3\times 10^{-6}$ and $p_{\geq}=2\times 10^{-20}$, respectively. Our approach is therefore successfully separating interactions that are likely to be spurious from those that are likely to be correct, without using any biophysical or biochemical information.

Discussion

We have shown that our network reconstruction method allows for a better characterization of network data sets, which will be particularly useful in data sets that we know contain many inaccuracies, such as protein interactomes. We have also shown that our approach reliably identifies missing and spurious interactions in complex networks, so that we can identify suspect interactions for further experimental probing.

Interestingly, our method can also guide new discoveries. If a given interaction between $i$ and $j$ truly exists but our approach predicts a very low reliability for the interaction (or vice versa), that means that the function of the interaction is very specific (since the interaction is rare among nodes that are otherwise similar to $i$ and $j$) and, therefore, functionally or evolutionarily important.

Finally, our approach is flexible enough to allow generalizations in several directions. Arguably the most important of these is the extension to arbitrarily sophisticated families of models. In particular, one could use models that are the “product” of a network model $M_{n}$ (probably a block model) and an error model $M_{e}$ that incorporates the relevant error structure (maybe another block model with non-uniform priors). The flexibility of our approach, along with its generality and its performance, will make it applicable to many areas where network data reliability is a source of concern.

Outline of the reliability calculations

Formally, a block model $M=(P,\mathbf{Q})$ is completely determined by the partition $P$ of nodes into groups and the matrix $\mathbf{Q}$ of probabilities of linkage between groups, so that Eq. (2) in the main text can be rewritten as

where $\mathcal{P}$ is the space of all possible partitions of the network into groups, $G$ is the number of distinct group pairs, and $Z$ is a normalizing constant.

Within the family of stochastic block models, one can evaluate the likelihood of each model $M$ because the probability of any two nodes $i$ and $j$ being connected depends only on the groups to which they belong. We have that

where $l^{O}_{\alpha\beta}$ is the number of links in $A^{O}$ between nodes in groups $\alpha$ and $\beta$ of $P$, and $r_{\alpha\beta}$ is the maximum number of such links (that is, the number of pairs of nodes such that one node is in $\alpha$ and the other is in $\beta$).

Note that, among all possible block models, there is at least one whose likelihood is 1, namely, the block model in which each node is in a different block and each $Q_{\alpha\beta}$ is 1 or 0 depending on whether the corresponding nodes are connected or not. This model contributes to $p(X=x|A^{O})$ much more than most other models. However, there is only one such model (or very few), whereas there are many models with, for example, four blocks. This “entropic” term prevents overfitting of the network by very detailed (and ultimately uninformative) block models.

Using that $p(A_{ij}=1|P,\mathbf{Q})=Q_{\sigma_{i}\sigma_{j}}$ (where $\sigma_{i}$ is the module of node $i$ in partition $P$) and assuming no prior knowledge about the models (that is, $p(P,\mathbf{Q})={\rm const.}$), one can use Eqs. (8) and (9) to obtain Eqs. (3)-(6) in the main text.

Metropolis estimation of link and network reliability

To estimate the link and network reliabilities given by Eqs. (3) and (5) we use the following procedure. We start by placing each of the $N$ nodes in a group, which we choose with uniform probability from a set of $N$ possible groups (that is, there are as many groups as nodes). In general, some of these $N$ groups will be empty after the initial node assignment.

At each step we select a random node and attempt to move it to a randomly selected group. This update scheme is appropriate because: (i) it results in an ergodic exploration of the space of possible partitions, and (ii) it satisfies detailed balance (since the probability of choosing a move and its reverse are identical). To decide whether we accept the move, we calculate the change $\Delta\mathcal{H}$ (Eq. (4)): if $\Delta\mathcal{H}\leq 0$, the change is automatically accepted; otherwise, the change is accepted with probability $\exp(-\Delta\mathcal{H})$.

The sampling procedure starts after an equilibration period, during which $\mathcal{H}$ decreases from an initial value to its equilibrium value. We sample the partition space by considering $S=10^{4}$ partitions, each one separated from the previous one by a number of steps that is large enough for the two partitions to be reasonably uncorrelated (as measured by the mutual information between partitions).

Because the link and network reliabilities are ensemble averages over independent partitions, it is straightforward to parallelize the algorithm so that the partitions are obtained concurrently. Therefore, given enough computational resources, the reliabilities can be calculated even for large networks (probably up to millions of nodes) in relatively short times.

Benchmark algorithms for the identification of missing and spurious interactions

The hierarchical random graph approach is described in detail in . We use the implementation provided by the authors (available at http://www.santafe.edu/$\sim$aaronc/hierarchy/hrg_20080819_predictHRG_v1.0.3.tgz), which we modified slightly to be able to study spurious as well as missing interactions.

We analyze three local algorithms: common neighbors, degree product, and Jaccard index (the last two in Supporting Information only). For each of these algorithms, the link “reliability” $R^{L}_{ij}$ is defined as follows (note that, for these approaches, the “reliability” is not a probability, but just a score that enables us to rank node pairs):

Common neighbors: $R^{L}_{ij}=\lVert\Gamma_{i}\cap\Gamma_{j}\rVert$, where $\Gamma_{i}$ is the set of neighbors of node $i$, and $\lVert\dots\rVert$ indicates the number of nodes in a set.

Degree product: $R^{L}_{ij}=\lVert\Gamma_{i}\rVert\times\lVert\Gamma_{j}\rVert$.

Jaccard index: $R^{L}_{ij}=\lVert\Gamma_{i}\cap\Gamma_{j}\rVert/\lVert\Gamma_{i}\cup\Gamma_{j}\rVert$.

Heuristic network reconstruction

The goal of the heuristic network reconstruction algorithm is to find $A^{R}=\operatorname{arg\,max}_{A}R^{N}_{A}$, where $R^{N}_{A}=p(A|A^{O})$ is the reliability of network $A$ given observation $A^{O}$. Since exhaustive maximization of $R^{N}_{A}$ is not possible, we use the following heuristic method. Start by evaluating the link reliabilities $R^{L}_{ij}$ for all pairs of nodes in $A^{O}$; sort observed links ($A^{O}_{ij}=1$) by increasing reliability, and observed non-links ($A^{O}_{ij}=0$) by decreasing reliability. Then choose pairs of link/non-link in order: remove the link (which has a low reliability) and add the non-link (which has a high reliability), and accept the change if, and only if, $R^{N}_{A}$ increases. Repeat this procedure, going down the lists, until we reject five consecutive attempts to swap a link/non-link pair. At this point, reevaluate $R^{L}_{ij}$ and repeat the process. The algorithm stops when no link swaps are accepted.

References