Predicting missing links in complex networks based on common neighbors and distance

Abstract

The algorithms based on common neighbors metric to predict missing links in complex networks are very popular, but most of these algorithms do not account for missing links between nodes with no common neighbors. It is not accurate enough to reconstruct networks by using these methods in some cases especially when between nodes have less common neighbors. We proposed in this paper a new algorithm based on common neighbors and distance to improve accuracy of link prediction. Our proposed algorithm makes remarkable effect in predicting the missing links between nodes with no common neighbors and performs better than most existing currently used methods for a variety of real-world networks without increasing complexity.

Introduction

As an important branch of network data analysis, predicting missing links in complex network has attracted many researchers’ widespread attentions not only because data collected from network platforms is incomplete^1,2, but also it is helpful to understand evolution of networks^3,4,5, and predict future conflict and individual preferences^6,7. In principle many evolution models correspond to a link prediction approach. Thus, link prediction can be used in revealing hidden information and evaluating the performance of distinct models. Moreover, link prediction has been applied to health care and communication field to identify abnormal cases^8,9.

The main work in link prediction is to estimate the missing link between two nodes based on current links and interactions in networks¹⁰. Link prediction discusses missing links and spurious links. In this paper, we focus on predicting missing links. Generally, between nodes with very high likelihood scores are considered to be highly likely to have missing links. In the past few years, many prediction methods based on topological structure of networks have been proposed related to local paths, common neighbors and random walk^10,11,12,13. In social networks two individuals who have more common friends are very likely to be friends in future. Furthermore, community methods, hierarchical models and probabilistic methods are also used for link prediction^{14,15,16,17,18,19}. Recently, information theory and spectral method of adjacency matrix have been adopted to find missing links^20,21,22.

The prediction methods based on common neighbors metric¹⁰ are very popular due to its simplicity. But with single common neighbors metric is not accurate enough to reveal the similarities between nodes and reconstruct properly networks, especially there are less common neighbors between nodes in sparse networks. A part of missing links could not be predicted because there are no common neighbors between them, but they often play a key role to connect different communities, and affect network properties, such as betweenness centrality, average distance, congestion and spreading ability. Therefore, it is important to propose an algorithm to predict missing links between nodes with no common neighbors.

A lot of real-world networks indicate high clustering properties. There are a large number of short loops. It is a good idea to exploit this properties to improve accuracy of link prediction. In this paper, we separate link prediction into two parts: predicted links that generate loops of length 3 and predicted links that generate short loops of length more than 3. So common neighbors and distance metric are adopted to predict these loops. The key question is to estimate the proportion of two parts. A new method is proposed to achieve it. By estimating the proportion of missing links between nodes with no common neighbors in total missing links, our algorithm makes remarkable effect to predict the missing links between nodes with no common neighbors, and improve the accuracy of link prediction. The experimental results show that it can obtain significantly better prediction accuracy for a variety of real-world networks than other methods.

Results

Data sets Description

The test data sets of real-world networks in this paper are:

• Karate: The test data of Karate club network was collected by Zachary, which indicates the interactions of 34 members of a university Karate club²³.

• Dolphins: It is an animal relationship network studied by Lusseau et al. with 62 bottlenose dolphins living in Doubtful Sound of New Zealand²⁴.

• Polbook: This is a network of books about US politics published around the time of the 2004 presidential election and sold by the online bookseller Amazon.com. The network was compiled by Krebs and is unpublished, but can be found on Krebs’ website (see http://www.orgnet.com).

• Word: The data is a network of common adjective and noun adjacencies for the novel “David Copperfield” by Charles Dickens, as described by Newman²⁵.

• Neural: This data represents the neural network of C. Elegans. The nodes in the original data are not consecutively numbered, so they have been renumbered to be consecutive²⁶.

• Circuit: Electronic circuits can be viewed as networks in which nodes are electronic components (like capacitors, diodes, etc.) and connections are wires. Our network maps one of the benchmark circuits of the so-called ISCAS'89 set (see data set from http://www.weizmann.ac.il/mcb/UriAlon/)²⁷.

• Email: This is a network of e-mail interchanges between members of the Universitat Rovira i Virgili (Tarragona) (see data set from http://deim.urv.cat/~alexandre.arenas/data/welcome.htm)²⁸.

• Power: This is an undirected, unweighted network representing the topology of the Western States Power Grid of the United States²⁶.

The data sets (1–5) and (8) can be downloaded from Mark Newman’s network data sets: http://www-personal.umich.edu/~mejn/netdata/. The parameters of networks about the number of nodes N, the number of links m, average degree 〈k〉, average distance 〈d〉, assortativity coefficient r and degree heterogeneity H are listed in Table 1.

Table 1 Illustration of properties of networks.

Link prediction method

Two metrics are used to evaluate the accuracy of link prediction methods: AUC (areas under the receiver operating characteristic curve)²⁹ and Precision³⁰. Given an unweighted and undirected network G = (V, E) with vertex set V = {v₁, v₂, …, v_N} and the observed link set E, where the size of E is m. The self-loops and multiple links are not allowed. E are randomly divided into two disjoint subsets: the probe set E^P and the training set E^T. E^P is used for testing and is viewed as unknown information. E^T is viewed as known information. A good prediction method should have high AUC value according to the definition of AUC, i.e. the links in probe set have higher scores than non-existing links. Precision is computed as the fraction of correct predicted links in the top-L ranking lists, where L is the total number of missing links (L = |E^P|) (see Methods section).

Now G′ = (V, E^T) is known, so basic idea in reconstructing network is to add top-L predicted links to G′ to obtain G^* = (V, E^*) so that G^* is as close as possible to G. Therefore, a good predicted method can provide trusted prediction in the evolution of networks.

In this paper we separate the probe set E^P into two subsets: and , which denote link set between nodes with common neighbors and link set between nodes with no common neighbors in G′, respectively, that is,

where Γ(i) denotes the set of neighbors of node i. Let is the proportion of in E^P. The test will be performed with E^P accounting for 10% of the observed link set E, and randomly selects E^P to remove every time. The results of c_r are the average of 20 realizations for each network (see Table 1).

But for a practical observed network G, link prediction methods are used to predict the possible links in the future (network evolution), we only knew roughly the total number of missing links L = |E^P|, which is consistent with other methods in literatures. Therefore, it is key to estimate the proportion of in E^P in order to improve prediction accuracy under the Precision metric. We define to evaluate strength of links between nodes with common neighbors in G′ = (V, E^T) (The corresponding definition is c_n for G, and c_n is labeled as “CN coefficient” in this paper), which is defined as the fraction of links between nodes with common neighbors in E^T, that is,

where δ_ij = 1, if Γ(i) ∩ Γ(j) ≠ ∅, 0 otherwise, and i ~ j denotes node i and j to be adjacent. The results of and c_n are listed in Table 1. Between and c_r has low RMSE (root-mean-square error) and high positive correlation measured by Pearson correlation coefficient (CC). Furthermore, between c_n, c_r and clustering coefficient c (defined in Methods section) also indicate high positive correlation (see Table 2). Therefore, it is feasible to use instead of c_r to estimate the proportion of in E^P. Thus, in G′.

Table 2 Root mean square error (EMSE) and Pearson correlation coefficient (CC) between c_n, , c_r and clustering coefficient c in the 8 networks.

There are large number of short loops in real-world networks. In order to illustrate the fact, the distribution of “pseudo-distance” will be given. Generally, the distance d_ij of two nodes i, j is defined as the length of the shortest paths between them. d_ij is infinite if no such path exist. “pseudo-distance” is defined to be the length of the shortest paths between i and j in network G − e for a link e = (i, j), that is, . Let

which denotes the fraction of links where pseudo-distance is d in total links. In a way it can reveal distance distribution of missing links in G′. Figure 1 describes the distribution p_d for d = 2, 3, 4 and the rest in 8 real-world networks, where the distribution of most networks are concentrated in d = 2, 3, 4. It is obvious that p₂ = c_n.

Figure 1

The distributions p_d for d = 2, 3, 4 and the rest in 8 real-world networks.

On the other hand, these missing links for d ≠ 2 (links between nodes with no common neighbors) play a pivotal role in determining the structure properties of networks. But they are neglected by mostly current existing link prediction methods. A single method based on common neighbors could not predict these important missing links. So we propose in this paper a new prediction method based on common neighbors and distance. The score of likelihood is defined as follows:

where CN_ij = |Γ(i) ∩ Γ(j)| represents the number of common neighbors for node i and j. The above equation equals to:

Since between two nodes is more likely to possess a link if they have more common neighbors, it is not difficult to find that it is equivalent to the CN method¹⁰ for these pairs of nodes with common neighbors, but distance plays an important role in predicting missing links between nodes with no common neighbors because high score is obtained by short distance. Our method is summarized in Methods section.

Experiment

In Table 3, the predicted results of different methods under the AUC metric are listed in 8 real-world networks. Our method is compared with prediction methods based on common neighbors¹⁰: CN method, Sal (Salton Index), Jaccard Index, Sen (Sϕrensen Index), HPI (Hub Promoted Index), HDI (Hub Depressed Index), LHN (Leicht-Holme-Newman Index), AA (Adamic-Adar Index) and RA method (Resource Allocation Index). For computation in these methods can be seen in Table 4. The results are the average of 20 realizations for each network under 10% and 20% probe set. Probe set will be randomly removed every time. The highest value of AUC for each network is labeled in boldface. The accuracy of our method outperforms other methods except Neural network, because this network possesses high c_n, high degree heterogeneity and negative assortativity. RA index and AA index have similar form, and thus they have nearly same scores. Circuit and Power network have low c_n, and thus for most methods assign low AUC scores which are approximately 0.5 for this two networks. Conversely, our method shows better results due to distance.

Table 3 The AUC of different methods under 10% and 20% probe set in 8 networks.

Table 4 The computation of link prediction methods.

In most algorithms, AUC shows slightly downward trend when the proportion of E^P in E increases from 10% to 20% (see Fig. 2). The main reason is that the decrease of training set E^T will result in the number of pairs of nodes with common neighbors becoming small, which increases the difficulty of link prediction.

Figure 2

The changes of AUC when increases from 10% to 20% in 8 real-world networks (a–h).

On the other hand, the prediction results under the Precision metric are given in Table 5. Similarly, our method noticeably outperforms other methods except Polbook and Power network in  = 20%, because Power network has low (or c_n), high m and large average distance 〈d〉 = 18.989. According to Eq. (12) in Power network our method need predict too much missing links with large distance (i.e. the presence of few short loops with length more than 3), which result in low prediction accuracy than CN method. It should be mentioned that Precision indicates the opposite changing trend compared with AUC except Circuit and Power network (They are low ) with the increase of E^P (see Fig. 3).

Table 5 The Precision of different methods under 10% and 20% probe set in 8 networks.

Figure 3

The changes of Precision when increases from 10% to 20% in 8 real-world networks (a–h).

The results of most algorithms are better for high ratio of E^P than low one for Precision metric. The main reason is that the decrease of training set E^T will result in weak n′ and strong n′′ according to the definition of AUC (see Methods section), which make negative contributions to AUC. But the increase of probe set E^P, the probability to obtain relevant items will increase, and it is easier to find the missing links, which is a good explanation for this phenomenon. Therefore, in practical application it is necessary to combine two metrics to evaluate accuracy of a link prediction method.

Moreover, CN coefficient c_n of original network may also affect prediction accuracy. In Fig. 4 AUC metric and c_n have high positive correlation for almost all methods, but for Precision metric there are only RA, AA, CN and our method keeping high positive correlation. The change in probe set has little effect on all methods according to AUC metric, but makes great effect on Jaccard, HDI and Sen method according to Precision metric.

Figure 4

Pearson correlation coefficient of different methods for prediction accuracy metrics vs. c_n under 10% and 20% probe set in 8 networks.

(a) The correlation coefficient of different methods for AUC metric vs. c_n. (b) The correlation coefficient of different methods for Precision metric vs. c_n.

Next, we compared our method with other two classic methods PA (Preferential Attachment Index)³¹ and LP (Local Path Index)¹¹, which do not directly relate to common neighbors, but based on local information. Table 6 indicates the prediction accuracy of LP, PA and our method under the Precision metric with 10% and 20% probe set, where our method has the best performance.

Table 6 The Precision of LP, PA and our method under 10% and 20% probe set in 8 networks.

The most striking feature of our method is to make remarkable effect to predict missing links between nodes with no common neighbors (i.e. the accuracy to find ), compared with LP and PA method according to Precision metric (see Table 7). The above mentioned methods based on common neighbors cannot find any missing links between nodes with no common neighbors, and thus we do not list them here. The results indicate that LP cannot find any missing links with respect to , and PA method could find a small amount of for certain networks because the proportion of in E^P is low except Circuit network and Power network. The proportion can be calculated by Eq. (12).

Table 7 The Precision of different methods to predict missing links between nodes with no common neighbors under 10% and 20% probe set in 8 networks.

The time-consuming of computing distance between all pairs of nodes is at most O(mN) by using BFS (Breadth First Search). For a sparse network (m = O(N)) the complexity of our method is equivalent to the complexity of CN method.

Discussion and Conclusion

In the past few years, many link prediction algorithms have been proposed, but most of the algorithms do not account for missing links between two nodes with no common neighbors. Generally, to predict these missing links between nodes with no common neighbors are very difficult because they account for low proportion in missing links, but they have significance in determining network structure and network properties. We proposed in this paper a new algorithm based on common neighbors and distance to improve prediction accuracy, which separates link prediction into two parts: predicted links that generate loops of length 3 (missing links between nodes with common neighbors) and predicted links that generate short loops of length more than 3 (missing links between nodes with no common neighbors). By estimating the proportion of missing links between nodes with no common neighbors in total missing links, our algorithm makes remarkable effect to predict missing links between nodes with no common neighbors. For other methods based on common neighbors cannot find any missing links between nodes with no common neighbors. A series of experimental results indicate that prediction accuracy of our proposed method is better than most existing currently used methods for a variety of real-world networks. The complexity of this method is almost same as that of CN method. Moreover, there are two rules: (i) With the increase of probe set, experimental results indicate that the changing trend of scores according to AUC metric is not consistent with Precision metric for most algorithms and networks. Thus, it is necessary to combine two metrics to evaluate the accuracy of a link prediction method. (ii) The between AUC metric and c_n has high positive correlation.

On the other hand, the questions of link prediction have not been solved completely. For example, how to evaluate superiority of a link prediction method except current metrics. Because a good prediction method should not only take into account prediction accuracy, but also pay attention to network properties. Furthermore, it is important how to choose a suitable link prediction method according to the feature of network as there is no absolutely good method for all networks. Link prediction has been extended to weighted and directed version^32,33,34,35. It can also predict signed links with positive and negative relationships in social networks, and predict spurious interactions^15,36,37. Whether it is possible to modify our method to deal with them. All are a long-standing challenge work.

Methods

Metrics

AUC is defined as:

A link prediction method provides an ordered list of scores of all links in U − E^T (scores represent the likelihood of missing links), where U is a universal set for links. At each time, we will randomly select a link in U − E and a link in probe set E^P to compare their scores. After comparison of n times, there are n′ times the links in E^P having higher scores and n′′ times they have same scores. The degree to which the score exceeds 0.5 represents how better the method performs than pure chance.

Another metric to measure accuracy is Precision, which is computed as follows:

where L_r is relevant links (i.e. generally, we take the top-L links as the predicted links according to scores, and there are L_r links in the probe set E^P (L = |E^P|)). Thus, the higher Precision value means the higher accuracy.

Our link prediction method

Given an undirected and unweighted network G = (V, E) with vertex set V = {v₁, v₂, …, v_N} and the observed link set E, where the size of E is m. The self-loops and multiple links are not allowed. In order to evaluate the performance of an algorithm, a certain proportion links in G will be randomly selected to constitute probe set E^P, and the rest links constitute training set E^T (E^T ∪ E^P = E, E^T ∩ E^P = ∅). We separate E^P into two subsets: and denote respectively link set between nodes with common neighbors and link set between nodes with no common neighbors . and are calculated using Eqs (11 and 12) because E^P is used for testing and is viewed as unknown information. According to computation of s_ij (Eq. (13)) to obtain the scores for all non-exist links U − E^T, sort the list of scores in non-increasing order. Then select top- links between nodes with common neighbors and top- links between nodes with no common neighbors from U − E^T to constitute predicted links.

where Γ(i) denotes the set of neighbors of node i. δ_ij = 1, if Γ(i) ∩ Γ(j) ≠ ∅, 0 otherwise, and i ~ j denotes node i and j to be adjacent.

where CN_ij = |Γ(i) ∩ Γ(j)| is the number of common neighbors of node i and j. d_ij is the distance between i and j. The Precision metric in Eq. (9) can be written as follows:

where , denote relevant links between nodes with common neighbors and relevant links between nodes with no common neighbors, respectively. Similarly, to predict future missing links for a current observed network G, is replaced by c_n as follows:

which is the proportion of links between nodes with common neighbors in link set E.

Parameters

The local clustering coefficient c(i) of a node i is defined as the probability that two distinct neighbors of i are connected³⁸.

where |E_i| denotes the number of links that actually exist between k_i nodes, and c(i) = 0 if k_i = 0, 1. The clustering coefficient c of a network is the average of all nodes:

Assortativity of network is called as assortative mixing, which refers to the tendency of network nodes to joint other nodes preferentially with similar or opposite properties³⁹:

where .