Playing the role of weak clique property in link prediction: A friend recommendation model

An important fact in studying link prediction is that the structural properties of networks have significant impacts on the performance of algorithms. Therefore, how to improve the performance of link prediction with the aid of structural properties of networks is an essential problem. By analyzing many real networks, we find a typical structural property: nodes are preferentially linked to the nodes with the weak clique structure (abbreviated as PWCS to simplify descriptions). Based on this PWCS phenomenon, we propose a local friend recommendation (FR) index to facilitate link prediction. Our experiments show that the performance of FR index is better than some famous local similarity indices, such as Common Neighbor (CN) index, Adamic-Adar (AA) index and Resource Allocation (RA) index. We then explain why PWCS can give rise to the better performance of FR index in link prediction. Finally, a mixed friend recommendation index (labelled MFR) is proposed by utilizing the PWCS phenomenon, which further improves the accuracy of link prediction.


I. INTRODUCTION
The research of link prediction mainly focuses on forecasting potential relations between nonadjacent nodes, including the prediction of the unknown links or the further nodes [1].Since the wide range of applications of link prediction, such as recommending friends in online social networks [2], exploring protein-to-protein interactions [3], reconstructing airline network [4], and boosting e-commerce scales, which has attracted much attention recently [5][6][7].The probabilistic model and machine learning were mainly introduced in link prediction.The notion of probabilistic link prediction and path analysis using Markov chains method were first proposed and evaluated in Ref. [8], and then Markov chains method was further studied in adaptive web sites [9]; In Ref. [10], Popescul et al. studied the application of statistical relational learning to link prediction in the domain of scientific literature citations.
However, the mentioned methods for link prediction were mainly based on attributes of nodes.It is known that the structure of the network is easier to be obtained than the attributes of nodes, as a result, the network-structure-based link prediction has attracted increasing attention.Along this line, Liben-Nowell et al. developed approaches to link prediction based on measures for analyzing the "proximity" of nodes in a network [11].Since hierarchical structure commonly exists in the food webs, biochemical networks, social networks and so forth, a link prediction method based on the knowledge of hierarchical structure was investigated in Ref. [12], and they found that such a method can provide an accurate performance.Zhou et al. proposed a local similarity index-Resource Allocation (RA) index to predict the missing links, and their findings indicate that RA index has the best performance of link prediction [13].Given that many networks are sparse and very huge, so Liu et al. presented a local random walk method to solve the problem of missing link prediction, and which can give competitively good prediction or even better prediction than other random-walk-based methods while has a lower computational complexity [14].In view of the local community features in many networks, Cannistraci et al. proposed an efficient computational framework called local community paradigm to calculate the link similarity between pairs of nodes [3].Liu et al. designed a parameter-free local blocking predictor to detect missing links in given networks via local link density calculations, which performs better than the traditional local indices with the same time complexity [15].
Since the structural properties of networks have significant effects on the performance of algorithms in link predictions, there are some literatures have proposed some methods by making use of the structural properties of networks.Such as the algorithms by playing the roles of hierarchical structure [12], clustering [16], weak ties [5] and local community paradigm [3].However, current advances in including structural properties into link prediction are still not enough.In this paper, by investigating the local structural properties in many real networks, we find a common phenomenon: nodes are preferentially linked to the nodes with weak clique structure (PWCS).Then based on the observed phenomenon, a friend recommendation (FR) index is proposed.In this method, when a node j introduces one of his friends to a node i, he will not introduce the nodes who are also the neighbors of node i.Our results show that the performance of FR index is significantly better than CN, AA and RA indices since FR index can make good use of PWCS in networks.At last, to further play the role of PWCS, we define a mixed friend recommendation (MFR) method, which can better improve the accuracy in link prediction.

II. PRELIMINARIES
Considering an undirected and unweighed network G(V, E), where V is the set of nodes and E is the set of links.The multiple links and self-connections are not allowed.For a network with size N , the universal set of all possible links, denoted by U , containing of N (N −1) 2 pairs of links.For each pair of nodes, x, y ∈ V , we assign a score, S xy , according to a defined similarity measure.Higher score means higher similarity between x and y, and vice versa.Since G is undirected, the score is supposed to be symmetry, that is S xy = S yx .All the nonexistent links are sorted in a descending order according to their scores, and the links at the top are most likely to exist [13,14].To test the prediction accuracy of each index, we adopt the approach used in Ref. [13].The link set E is randomly divided into two sets Where set E T is the training set and is supposed to be known information, and E P is the testing set for the purpose of testing and no information therein is allowed to be used for prediction.As in previous literatures, the training set E T always contains 90% of links in this work, and the remaining 10% of links constitute the testing set.Two standard metrics are used to quantify the accuracy of prediction algorithms: area under the receiver operating characteristic curve (AUC) and Precision [5].
Area under curve (AUC) can be interpreted as the probability that a randomly chosen missing link (a link in E P ) is given a higher score than a randomly chosen nonexistent link (a link in U − E P ).When implementing, among n independent comparisons, if there are n ′ times the missing link has a higher score and n ′′ times they are the same score, AUC can be read as follow [5]: If all the scores generated from independent and identical distribution, the accuracy should be about 0.5.Therefore, the degree to which the accuracy exceeds 0.5 indicates how much the algorithm performs better than pure chance.
Precision is the ratio of the number of missing links predicted correctly within those top-L ranked links to L, and L = 100 in this paper.If m links are correctly predicted, then Precision can be calculated as [5]: We mainly compare three local similarity indices for link prediction, including (1) Common Neighbors(CN) [17]; (2) Adamic-Adar (AA) index [18]; (3) Resource Allocation (RA) index [13].Among which, CN index is the simplest index.AA index and RA index have the similar form, and they both depress the contribution of the high-degree common neighbors, however, Zhou et al. have shown that the performance of RA index is generally better than AA index.
Let Γ(i) be the neighbor set of node i, |.| be the cardinality of the set, and k(i) be the degree of node i.Then CN index, AA index and RA index are defined as AA index RA index

III. DATA SET
In this paper, we choose twelve representative networks drawn from disparate fields: including: (1) C. elegans-The neural network of the nematode worm C. elegans [19]; (2) NS-A coauthorship network of scientists working on network theory and experiment [20]; (3) FWEW-A 66 component budget of the carbon exchanges occurring during the wet and dry seasons in the graminoid ecosystem of South Florid [21]; (4) FWFW-A food web in Florida Bay during the rainy season [21]; (5) USAir-The US Air transportation system [22]; (6) Jazz-A collaboration network of jazz musicians [23]; (7) TAP-yeast protein-protein binding network generated by tandem affinity purification experiments [24]; (8) Power-An electrical power grid of the western US [19]; (9) Metabolic-A metabolic network of C.elegans [25]; (10) Yeast-A protein-protein interaction network in budding yeast [26]; (11) Router-A symmetrized snapshot of the structure of the Internet at the level of autonomous systems [27]; (12) PB-A network of the US political blogs [28].Topological features of these networks are summarized in Tab.I.

IV. UNIVERSALITY OF PWCS PHENOMENON
To check whether the PWCS phenomenon commonly exists in real networks, we divide all links into common links or strong-tie links by judging whether the number of common neighbors between the two endpoints is larger than a threshold β.Taking Fig. 1 as an example, when we choose β = 3, the links {A, B} and {A, C} in Figs.1(a), (b) and (c) can be correspondingly degenerated to the sketches in Fig 1 . (d)-(f), where common links and strong-tie links are marked by fine links and bold links, respectively.
In this paper, the threshold β is chosen such that the number of common links and the number of strong-tie links are approximately equal in each network.Once the value of β is fixed, there are seven possible configurations for the connected subgraphs with 3 nodes (i.e., triples [29] ), all the seven configurations are plotted in Fig. 2, where the bold links and (each CS represents a configuration in Fig. 2) in networks.If both of {A, B} and {A, C} are strong-tie links, then the probability of node B connecting node C is defined as [30]: If only one of links {A, B} or {A, C} is strong-tie link, then the probability of node B connecting node C is defined as: If neither of them is strong-tie link, then the probability of node B connecting node C is: First, we define a subgraph with n nodes be a weak clique the number of links among the n nodes is rather dense, which is an extended definition of n-clique where all pairs of nodes are connected.Next, by calculating the probability of node B connecting C, we can judge whether the phenomenon that nodes are preferentially linked to the nodes with weak clique structure (i.e., PWCS phenomenon) commonly exists in a network.We say that the PWCS phenomenon exists in the network if P 1 > P 2 and P 1 > P 3 .Moreover, we say that the PWCS phenomenon is significant if P 1 > P 2 > P 3 , otherwise, the PWCS phenomenon is weak as P 1 > P 3 ≥ P 2 .
Table II reports the values of P 1 , P 2 and P 3 in the twelve real networks (labelled as RN) and the values on the corresponding null networks (labelled NN) are also comparatively shown.One can find that P 1 > P 2 and P 1 > P 3 in eleven networks except for Metabolic network (P 1 < P 3 , marked by red color).However, in the corresponding null networks, P 1 ≈ P 2 ≈ P 3 .Also, for C.celegans, FWEW, FWFW, Power, Router and PB networks, where P 1 > P 2 > P 3 .As a result, we can state that PWCS phenomenon is more significant in these six networks.

V. FRIEND RECOMMENDATION MODEL
Given that PWCS phenomenon commonly exists in real networks, whether can we design an effective link prediction method based on this phenomenon.Considering the cases in Fig. 3, where node 3 ask its neighbor node 2 to introduce a friend to it.Since the number of common neighbors between node 2 and node 3 in Fig. 3(c) is larger than that of in Fig. 3 (b) and is further larger than that of in Fig. 3(a), in other words, the strength of link {2, 3} in Fig. 3(c) is the strongest.According to PWCS phenomenon, the probability (labelled by f 123 ) of node 1 (call nominee, green color) being introduced to node 3 (call acceptor, red color) by node 2 (call introducer, blue color) in Fig. 3(c) should be larger than that of in Fig. 3  (b), and then further larger than that of in Fig. 3(a).To reflect the mentioned fact, we define f ilj be the probability of i being introduced to j by their common neighbor l, which is given as: Based on the definition in Eq. ( 9), the values of f 123 in Fig. 3(a), (b) and (c) are 1/3, 1/2 and 1, respectively.That is to say, the probability f ilj can reflect the PWCS phenomenon in real networks.More importantly, Eq. ( 9) addresses two important facts: first, since node l will not introduce node j to j, as a result, 1 is subtracted in denominator of Eq. ( 9); second, in social communication, when a friend introduce one of his friends to me, he should introduce his friends but excluding the common friends.Therefore, the common neighbors set between j and l (i.e., Γ(l) ∩ Γ(j)) should be subtracted in denominator of Eq. ( 9).For instance, in Fig. 3(c), node 2 will not introduce node 3 to node 3, and nodes 4 and 5 should not be introduced to node 3.
Let f ij be the weight of node i being introduced to node j (we use weight rather than probability since f ij may larger than 1), which is written as: Here the value of f ij increases with the number of common

neighbors.
With the above preparations, the similarity index S F R ij for a pair of nodes i and j is defined as which guarantees S F R ij = S F R ji .The sketches in Fig. 4 is given to show how to calculate the similarity between node 1 and node 2 based on the FR index.Also, the red, blue and green nodes denote the acceptors, introducers and nominees, respectively.Node 2 can be introduced to node 1 by node 3 (see Fig. 4(a)) or node 4 (see Fig. 4(b)).When node 3 is an introducer (see Fig. 4(a)), who will introduce nodes 2, 5 and 7 (green color) to node 1 with equal probability, but excludes node 4, i.e., f 231 = 1/3.Similarly, when node 4 is an introducer (see Fig. 1(b)), who just introduces nodes 2 and 6 (green color) to node 1 with equal probability, but excludes node 3, i.e., f 241 = 1/2.Therefore, the probability f 21 = 1/3 + 1/2 = 5/6.Likely, from Figs. 5(c) and (d), the value of f 12 = 1/2 + 1/2 = 1.Therefore, the FR similarity index is S F R 12 = S F R 21 = 11/12.Combing Eqs. ( 9), ( 10) and (11), the advantages of FR index can be summarized as: 1) similar to many local similarity indices, the similarity between a pair of nodes increases with the number of common neighbors; 2) like AA index and RA index, FR index depresses the contribution of the high-degree common neighbors; 3) most importantly, FR index can make use of the PWCS phenomenon in many real networks; 4) FR index has higher resolution than other local similarity indices.For instance, the similarities S CN  13 , S AA 13 and S RA 13 are the same in Figs.3(a), (b) and (c).Yet, the value of S F R 13 in Fig. 3(c) is larger than Fig. 3(b), and is further larger than Fig. 3(a).

VI. RESULTS
In this section, the comparison of FR index with CN, AA and RA indices in twelve networks is summarized in Tab.III.As shown in Tab.III, FR index in general outperforms the other three indices in link prediction, regardless of AUC or Precision.The highest accuracy in each line is emphasized in bold.Moreover, the correlation of ranking values between FR index and RA index is given in Fig. 5, where the percentage values in x or y axis is the top percentage of ranking values based on Precision.As a result, a small percentage value means a higher ranking value.Fig. 5 describes that a high RA ranking value of links gives rise to a high FR ranking value.However, a high FR ranking value of links may induce a low RA ranking value of links.Take Tap and Yeast networks as examples, based on FR index, some links have higher ranking values, however their corresponding ranking values based on RA index may very small (see the regions marked by pink dash boundary in Figs.5(g) and (j)).
By analyzing a typical case in the Yeast network (see Fig. 6), where two nodes A and B are the neighbors of introducer C (in fact, there has a link connecting A and B in the Yeast network).Since links {A, C} and {B, C} are strongtie links.When using FR index, the similarity S F R AB is rather large, which can predict the existence of link {A, B}.However, for RA index, since the large degree value of introducer C, the similarity S RA AB is very small, such an existing link {A, B} cannot be accurately predicted by RA index.

VII. ROLE OF PWCS
We have validated that the FR index based on PWCS phenomenon can improve the performance of link prediction, and the reasons were also analyzed.Here we want to know how the strength of PWCS affects the performance of RA index and FR index.For this purpose, we propose a generalized friend recommendation (GFR) index, which is given as: where parameter 0 ≤ α ≤ 1 is used to uncover the role of PWCS.As α = 0, Eq. ( 12) returns to RA index, that is, S GF R ij = S RA ij .When α = 1, the difference between FR method and GFR method is the absence of 1 in the denominators of Eq. ( 12), therefore, we can simply view GFR index is the same as FR index when α = 1.As a result, with the increasing of α from zero to one, S SF R index can comprehensively investigate the role of PWCS on the RA index and FR index.
The effect of α on the Precision in all twelve networks is plotted in Fig. 7.As illustrated in Fig. 7, several interesting phenomena and meaningful conclusions can be summarized: First, except for Metabolic network, the Precision for the case of α > 0 is far larger than the case of α = 0 (i.e., RA index) in all other 11 networks.Since P 1 > P 2 and P 1 > P 3 in these 11 networks, which indicates that PWCS phenomenon in networks can ensure the higher accuracy of FR index (i.e.,α = 1) in link prediction ; Second, Metabolic network has non-PWCS phenomenon since P 1 > P 2 and P 2 < P 3 , and Fig. 7(i) suggests that Precision decreases with the value of α.In other words, FR index is invalid in network with non-PWCS phenomenon, which again emphasizes the importance of PWCS in link prediction; At last, by systematically comparing the subfigures in Fig. 7, one can see that, when the networks with weak PWCS P 1 > P 3 ≥ P 2 (i.e., the insets are light red background, see Fig. 7(b), (e), (f), (g) and (j)), Precision increases with α at first and then decreases when α is further increased (except Fig. 7(e)).However, when P 1 > P 2 > P 3 (i.e., networks with significant PWCS, the insets are white background, see Figs.In view of this observation, we can conjecture the role of PWCS can be further explored when the PWCS phenomenon is significant.Unfortunately, the maximal value α in Eq. ( 12) is one.the denominator may be negative if α > 1.So we design a new index to further explore the role of significant PWCS.
Since Eq. ( 12) can be rewritten as when α = 1.To further play the role of PWCS, another similarity index, called strong friend recommendation (labelled as

SFR) index, is given in following
Combing Eq. ( 13) with Eq. ( 14), we can find that two subtrahends S CN il and S CN jl in the numerator of Eq. ( 13) are removed.So Eq. ( 14) can better play the role of PWCS.
We conjecture that the performance of SFR index is better than GFR index when P 1 > P 2 > P 3 (i.e., significant PWCS), and worse than that of GFR index when P 1 < P 2 and P 1 < P 3 (i.e., non-PWCS).However, it is difficult to distinguish which one has better performance when P 1 > P 3 ≥ P 2 (i.e., weak PWCS).As presented in Tab.IV, Precision in 12 networks validates our conjecture.
Synthesizing the above results, we can find that the ranking of P 1 , P 2 and P 3 has determinant effect on the performance of the proposed index.Inspired by this clue, we may design a universal indicator to do link prediction based on the values of P 1 , P 2 and P 3 in different networks.To this end, we design a mixed friend recommendation (labelled MFR) index: S F R ij , P 1 > P 2 , P 1 > P 3 , P 2 < P 3 ; Table 5 lists the results of MFR index and FR index in 7 networks (since MFR index is the same to FR index when P 1 > P 2 , P 1 > P 3 and P 2 < P 3 , in this case, it is unnecessary to compare the two indices).The results in Tab.V indicate that, compared with FR index, MFR index can further improve the accuracy of link prediction.

VIII. CONCLUSIONS
In summary, by analyzing the structural properties in real networks, we have found that there exists a common phenomenon: nodes are preferentially linked to the nodes with weak clique structure.Then we have proposed a friend recommendation model to better predict the missing links based on the significant phenomenon.Through the detailed analysis and experimental results, we have shown that FR index has several typical characteristics: First, FR index is based on the information of common neighbors, which is a local similarity index.Thus, the algorithm is simple and has low complexity; Second, the common neighbors with small degrees has greater contributions than the common neighbors with larger degrees; Third, FR index can take full advantage of the PWCS phenomenon, and so forth.
Furthermore, we also proposed an SFR index to further improve the accuracy of link prediction when networks have significant PWCS phenomenon.At last, by judging whether the networks have significant PWCS, weak PWCS or non-PWCS phenomenon, we have also proposed a mixed friend recommendation index which can increase the accuracy of link prediction in different networks.In this work, we mainly applied FR index to unweighed and undirected networks, and how to generalize our FR index to weighted [31,32] or directed networks [33] is our further purpose.

FIG. 1 :
FIG. 1: (Color online) Degenerating the upper sketches into the lower cases by judging whether two links {B, A} and {A, C} are strong-tie link or common link.Here we assume that if the number of common neighbors between A and B (or A and C) is larger than β = 3, then the link is strong-tie link; otherwise, the link is common link in the opposite case.Fine lines and the bold lines in (d)-(f) are the common links and the strong-tie links, respectively.

FIG. 4 :
FIG. 4: (Color online) Calculation of the similarity S F R12 between node 1 and node 2. Nodes 1 and 2 can be introduced by their common neighbors 3 and 4. (a) node 3 introduces his friends to node 1 .Only neighbor nodes 2, 5, 7 can be introduced to node 1 but excludes node 4, since node 4 has been a friend of node 1.Thus, the probability of node 3 introducing node 2 to node 1 is: f231 = 1/3; (b) node 2 is introduced to node 1 by node 4, here only nodes 2 and 6 can be introduced to node 1.As a result, the probability f241 = 1/2; (c) node 1 is introduced to node 2 by node 3, here only nodes 1 and 5 can be introduced to node 1.As a result, the probability f132 = 1/2; (d) node 1 is introduced to node 2 by node 4, here only nodes 1 and 6 can be introduced to node 1.As a result, the probability f141 = 1/2.We have f21 = 1/3 + 1/2 by combing (a) and (b), and f12 = 1/2 + 1/2 by combing (c) and (d).So the FR similarity index is S F R 12 = S F R 21 = 11/12.

FIG. 5 :
FIG. 5: (Color online) The correlation of ranking values between FR index and RA index based on Precision.The percentage values in x-axis and y-axis are the top percentage ranking values of FR index and RA index, respectively.The regions marked by pink dash boundary in subfigures (g) and (j) correspond to the cases in which some links have higher FR ranking values but have low RA ranking values.

FIG. 6 :
FIG. 6: (Color online) A typical case in the Yeast network is considered to emphasize the difference between FR index and RA index, where nodes A, B and C are the node 1175, 421 and 205 in the Yeast network.Two links {A, C} and {B, C} share a common endpoint C, and both of them are strong-tie links.Therefore, the similarity S F R AB is rather large.However, when using RA index, the ranking number of S RA AB is very low owing to the large degree value of node C, causing the failure of RA index in predicting such an existing link.Red nodes, green nodes and blue nodes are the neighbors of A, B and C (including themselves), respectively.Purple nodes are the common neighbors of A and C; white nodes are the common neighbors of A, B and C.

TABLE I :
[29]basic topological features of twelve example networks.N and M are the total numbers of nodes and links, respectively.C and r are clustering coefficient and assortative coefficient, respectively.H is the degree heterogeneity, defined as H = k 2 k 2 , where k denotes the average degree[29].
fine lines denote strong-tie links and common links, respectively.Let N

TABLE II :
The values of P1, P2 and P3 in 12 real networks (RN) and the corresponding null networks (NN) are reported.Results in NN are marked in Italic.Results in networks with significant PWCS, i.e., P1 > P2 > P3 are shown in blue color, and results in Metabolic are marked by red color due to its specificity.

TABLE III :
Comparison of S F R with S CN , S AA and S RA in 12 networks, including AUC and Precision.The highest value in each row is marked in bold.

TABLE IV :
The comparison of Precision between SFR index and GFR index (α = 1) in 12 networks.The results suggest that the accuracy of link prediction can be further improved by SFR index when the networks have significant PWCS.The highest value in each case is marked as bold.