Abstract
Representation of large data sets became a key question of many scientific disciplines in the last decade. Several approaches for network visualization, data ordering and coarsegraining accomplished this goal. However, there was no underlying theoretical framework linking these problems. Here we show an elegant, information theoretic data representation approach as a unified solution of network visualization, data ordering and coarsegraining. The optimal representation is the hardest to distinguish from the original data matrix, measured by the relative entropy. The representation of network nodes as probability distributions provides an efficient visualization method and, in one dimension, an ordering of network nodes and edges. Coarsegrained representations of the input network enable both efficient data compression and hierarchical visualization to achieve high quality representations of larger data sets. Our unified data representation theory will help the analysis of extensive data sets, by revealing the largescale structure of complex networks in a comprehensible form.
Introduction
Complex network^{1,2} representations are widely used in physical, biological and social systems, and are usually given by huge data matrices. Network data size grew to the extent, which is too large for direct comprehension and requires carefully chosen representations. One option to gain insight into the structure of complex systems is to order the matrix elements to reveal the concealed patterns, such as degreecorrelations^{3,4} or community structure^{5,6,7,8,9,10,11}. Currently, there is a diversity of matrix ordering schemes of different backgrounds, such as graph theoretic methods^{12}, sparse matrix techniques^{13} and spectral decomposition algorithms^{14}. Coarsegraining or renormalization of networks^{15,16,17,18,19,20} also gained significant attention recently as an efficient tool to zoom out from the network, by averaging out shortscale details to reduce the size of the network to a tolerable extent and reveal the largescale patterns. A variety of heuristic coarsegraining techniques – also known as multiscale approaches – emerged, leading to significant advances of networkrelated optimization problems^{21,22} and the understanding of network structure^{19,20,23}. As we discuss in the Supplementary Information in more details, coarsegraining is also closely related to some blockmodels useful for clustering and benchmark graph generation^{24,25,26}.
The most essential tool of network comprehension is a faithful visualization of the network^{27}. Preceding more elaborate quantitative studies, it is capable of yielding an intuitive, direct qualitative understanding of complex systems. Although being of a primary importance, there is no general theory for network layout, leading to a multitude of graph drawing techniques. Among these, forcedirected^{28} methods are probably the most popular visualization tools, which rely on physical metaphors. Graph layout aims to produce aesthetically appealing outputs, with many subjective aims to quantify, such as minimal overlaps between not related parts (e.g. minimal edge crossings in d = 2), while preserving the symmetries of the network. Altogether, the field of graph drawing became a meeting point of art, physics and computer science^{29}.
Since the known approaches for the above problems generally lead to computationally expensive NPhard problems^{30}, the practical implementations were necessarily restricted to advanced approximative heuristic algorithms. Moreover, there was no successful attempt to incorporate network visualization, data ordering and coarsegraining into a common theoretical framework. Since information theory provides ideal tools to quantify the hidden structure in probabilistic data^{31,32}, its application to complex networks^{25,26,33,34,35,36,37} is a highly promising field. In this paper, our primary goal is to show an elegant, information theoretic representation theory for the unified solution of network visualization, data ordering and coarsegraining, establishing a common ground for the first time for these separated fields.
Usually, in graph theory, the complex system is at the level of abstraction, where each node is a dimensionless object, connected by lines representing their relations, given by the input data. Instead, we study the case in which both the input matrix and the approximative representation is given in the form of a probability distribution. This is the routinely considered case of edge weights reflecting the existence, frequency or strength of the interaction, such as in social and technological networks of communication, collaboration and traveling or in biological networks of interacting molecules or species. As discussed in details in the Supplementary Information, the probabilistic framework has long traditions in the theory of complex networks, including general random graph models, all Bayesian methods, community detection benchmarks^{24}, blockmodels^{25,26} and graphons^{38}.
The major tenet of our unified framework is that the best representation is selected by the criteria, that it is the hardest to be distinguished from the input data. In information theory this is readily obtained by minimizing the relative entropy – also known as the KullbackLeibler divergence^{39} – as a quality function. In the following we show that the visualization, ordering and coarsegraining of networks are intimately related to each other, being organic parts of a straightforward, unified representation theory. We also show that in some special cases our unified framework becomes identical with some of the known stateoftheart solutions for both visualization^{40,41,42} and coarsegraining^{25,26}, obtained independently in the literature.
Results
General network representation theory
For simplicity, here we consider a symmetric adjacency matrix, A, having probabilistic entries a_{ij} ≥ 0 and we try to find the optimal representation in terms of another matrix, B, having the same size. For more general inputs, such as hypergraphs given by an H incidence matrix, see the Methods section. The intuitive idea behind our framework is that we try to find the representation which is hardest to be distinguished from the input matrix. Within the frames of information theory, there is a natural way to quantify the closeness or quality of the representation, given by the relative entropy. The relative entropy, , measures the extra description length, when B is used to encode the data described by the original matrix, A, expressed by
where and ensure the proper normalizations of the probability distributions. As a shorthand notation here and in the following an asterisk indicates in index, for which the summation was carried out, as in the cases of and . Although is not a metric and not symmetric in A and B, it is an appropriate and widely applied measure of statistical remoteness^{43}, quantifying the distinguishability of B form A. The highest quality representation is achieved, when the relative entropy approaches 0, and our goal is to obtain a B* representation satisfying
Although can be in principle arbitrarily large, there is always a trivial upper bound available by the uncorrelated, product state representation, B^{0}, given by the matrix elements . For an illustration see Fig. 1a. It follows simply from the definition of the S(A), total information content and I(A), mutual information, given in the Methods section, that . Consequently, the optimized value of can be always normalized with I(A), or alternatively as
Here η is the ratio of the needed extra description length to the optimal description length of the system. In the following applications we use η to compare the optimality of the found representations. As an important property, the optimization of relative entropy is local in the sense, that the global optimum of a network comprising independent subnetworks is also locally optimal for each subnetwork. The finiteness of D_{0} also ensures, that if i and j are connected in the original network (a_{ij} > 0), then they are guaranteed to be connected in a meaningful representation as well, enforcing b_{ij} > 0, otherwise D would diverge. In the opposite case, when we have a connection in the representation, without a corresponding edge in the original graph (b_{ij} > 0 while a_{ij} = 0), b_{ij} does not appear directly in D, only globally, as a part of the b_{**} normalization. This densitypreserving property leads to a significant computational improvement for sparse networks, since there is no need to build a denser representation matrix, than the input matrix if we keep track of the b_{**} normalization. Nevertheless, the B matrix of the optimal representation (where D is small) is close to A, since due to Pinsker's inequality the total variation of the normalized distributions is bounded by D^{44}
Thus, in the optimal representation of a network all the connected network elements are connected, while having only a strongly suppressed amount of false positive connections. Here we note, that our representation theory can be straightforwardly extended for input networks given by an H incidence matrix instead of an adjacency matrix, for details of this case see the Methods section.
Network visualization and data ordering
Since forcedirected layout schemes^{28} have an energy or quality function, optimized by efficient techniques borrowed from manybody physics^{45} and computer science^{46}, graph layout could be in principle serve as a quantitative tool. However some of, these popular approaches inherently struggle with an information shortage problem, since the edge weights only provide half the needed data to initialize these techniques. For instance, for the initialization of the widely applied FruchtermanReingold^{47} (or for the KamadaKawai^{48}) method we need to set both the strength of an attractive force (optimal distance) and a repulsive force (spring constant) between the nodes in order to have a balanced system. Due to the lack of sufficient information, such graph layout techniques become somewhat illdefined and additional subjective considerations are needed to double the information encoded in the input data, traditionally by a nonlinear transformation of the attractive force parameters onto the parameters of the repulsive force^{47}. Global optimization techniques, such as information theoretic methods^{40,41,42,49} can, in principle, solve this problem by deriving the needed forces from one single information theoretic quality function.
In strong contrast to usual graph layout schemes, where the nodes are represented by points (without spatial extension) in a ddimensional background space, connected by (straight, curved or more elaborated) lines, in our approach network nodes are extended objects, namely probability distributions (ρ(x)) over the background space. The ddimensional background space is parametrized by the ddimensional coordinate vector, x. Importantly, in our representation the shape of nodes encodes just that additional set of information, which has been lost and then arbitrarily reintroduced in the above mentioned forcedirected visualization methods. In the following we consider the simple case of Gaussian distributions – having a width of σ, and norm , see Eq. (6) of the Methods section –, but we have also tested the nondifferentiable case of a homogeneous distribution in a spherical region of radius σ leading to similar results. For a given graphical representation the b_{ij} edge weights are defined as the overlaps of the distributions ρ_{i} and ρ_{j}, given in the Methods section. For a schematic illustration see Fig. 1b.
The trivial data representation of B^{0} can be obtained by an initialization, where all the nodes are at the same position, with the same distribution function (apart from a varying h_{i} ∝ a_{i*} normalization to ensure the proper statistical weight of the nodes). This way, initially D_{0} = I(A) is the mutual information of the input matrix, irrespectively from the chosen distribution function. The numerical optimization can be straightforwardly carried out by a fast and inefficient greedy optimization or a relatively slow, but more efficient simulated annealing scheme starting with an initialization of B^{0}. As a reasonable compromise, in the differentiable case of Gaussian distributions we use a NewtonRaphson iteration as in the KamadaKawai method^{48} (for details see the Supplementary Information), having a runtime of for N nodes. Here we note, that similarly to the related tSNE method^{41,50}, discussed in the Supplementary Information, the optimization can be in principle carried out in time by applying the BarnesHut approximation^{45}.
Independently from the chosen optimization protocol, the finiteness of D_{0} ensures that the connected nodes overlap in the layout as well, even for distributions having a finite support. Moreover, independent parts of the network (nodes or sets of nodes without connections between them) tend to be apart from each other in the layout. The densitypreserving property of the representation leads to the fact, that even if all the nodes overlap with all other nodes in the layout, the B matrix can be kept exactly as sparse as the A matrix, while keeping track only of the sum of the b_{**} normalization including the rest of the potential b_{ij} matrix elements. Additionally, if two rows (and columns) of the input matrix are proportional to each other, then it is optimal to represent them with the same distribution function in the layout, as though the two rows were merged together.
In the differentiable case, e.g. with Gaussian distributions, our visualization method can be conveniently interpreted as a forcedirected method. If the normalized overlap, b_{ij}/b_{**}, is smaller at a given edge than the normalized edge weight, a_{ij}/a_{**}, then it leads to an attractive force, while the opposite case induces a repulsive force. For details see the Supplementary Information. For Gaussian distributions all nodes overlap in the representations, leading typically to D > 0 in the optimal representation. However, for distributions with a finite support, such as the above mentioned homogeneous spheres, perfect layouts with D = 0 can be easily achieved even for sparse graphs. In d = 2 dimensions this concept is reminiscent to the celebrated concept of planarity^{51}. However, our concept can be applied in any dimensions. Furthermore, it goes much beyond planarity, since any network of (e.g. a fully connected graph) is perfectly represented in any dimensions by B^{0}, that is by simply putting all the nodes at the same position.
Our method is illustrated in Fig. 2. on the Zachary karate club network^{52}, which became a cornerstone of graph algorithm testing. It is a weighted social network of friendships between N_{0} = 34 members of a karate club at a US university, which fell apart after a debate into two communities. While usually the size of the nodes can be chosen arbitrarily, e.g. to illustrate their degree or other relevant characteristics, here the size of the nodes is part of the visualization optimization by reflecting the width of the distribution, indicating relevant information about the layout itself. In fact, the size of a node represents the uncertainty of its position, serving also as a readily available local quality indicator. For illustration of the applicability of our network visualization method to larger collaboration^{53} and information sharing^{54} networks, having more than 10,000 nodes, see the Supplementary Information.
Our network layout technique works in any dimensions, as illustrated in d = 1, 2 and 3 in Fig. 2. In each case the communities are clearly recovered and, as expected, the quality of layout becomes better (indicated by a decreasing η value) as the dimensionality of the embedding space increases. Nevertheless, the one dimensional case deserves special attention, since it serves as an ordering of the elements as well (after resolving possible degenerations with small perturbations), as illustrated in Fig. 1e.
Since , H(A, B) is the (unnormalized) crossentropy, we can equivalently minimize the crossentropy for B. For a comparison to the known crossentropy methods^{55,56,57} see the Supplementary Information. However, as a consequence, the visualization and ordering is perfectly robust against noise in the input matrix elements. This means, that even if the input A matrix is just the average of a matrix ensemble, where the a_{ij} elements have an (arbitrarily) broad distribution, the optimal representation is the same as it were by optimizing for the whole ensemble simultaneously. This extreme robustness follows straightforwardly from the linearity of the H(A, B) crossentropy in the a_{ij} matrix elements. Note, however, that the optimal value of the distinguishability is generally shifted by the noise.
When applying a local scheme for the optimization of the representations, we generally run into local minima, in which the layout can not be improved by single node updates, since whole parts of the network should be updated (rescaled, rotated or moved over each other), instead. Being a general difficulty in many optimization problems, it was expected to be insurmountable also in our approach. In the following we show, that the relative entropy based coarsegraining scheme – given in the next section – can, in practice, efficiently help us trough these difficulties in polynomial time.
Coarsegraining of networks
In the process of coarsegraining we identify groups or clusters of nodes, and try to find the best representation, while averaging out for the details inside the groups. Inside a group, the nodes are replaced by their normalized average, while keeping their degrees fixed. As the simplest example, the coarsegraining of two rows means, that instead of the original k and l rows, we use two new rows, being proportional to each other, while the b_{k*} = a_{k*} and b_{l*} = a_{l*} probabilities are kept fixed
In other words, we first simply sum up the corresponding rows and obtain a smaller matrix, then inflate this fused matrix back to the original size while keeping the statistical weights of the nodes (degrees) fixed. For an illustration of the smaller, fused data matrices see the lower panels of Fig. 3a–d. For a symmetric adjacency matrix, the coarsegraining step can be also carried out simultaneously and identically for the rows and columns, known as a biclustering. The optimal biclustering is illustrated in Fig. 2f for the Zachary karate club network. The heights in the shown dendrogram indicate the D values of the representations when the fusion step happens.
For the general, technical formulation of our coarsegraining approach and details of the numerical optimization, see the Supplementary Information. As it turns out, for coarsegraining, is nothing but the amount of lost mutual information between the rows and columns of the input matrix. In other words, is the amount of lost structural signal during coarsegraining and finally we arrive at a complete loss of structural information, . Prevailingly, this final state coincides with the above proposed initialization step of our network layout approach. As a further connection with the graph layout, if two rows (or columns) are proportional to each other, they can be fused together without losing any information, since their coarsegraining leads to no change in the KullbackLeibler divergence, D.
Since it is generally expected to be an NPhard problem to find the optimal simplified, coarsegrained description of a network at a given scale, we have to rely on approximate heuristics having a reasonable runtime. In the following we use a local coarsegraining approach, where in each step a pair of rows (and columns) is replaced by coarsegrained ones, giving the best approximative new network in terms of the obtained pairwise Dvalue. This way the optimization can be generally carried out in time for N nodes. As a common practice, for larger networks we could use the approximation of fusing together a finite amount (eg. 1%) of the nodes in each step instead of a single pair, leading to an improved runtime.
As illustrated in Fig. 2f the coarsegraining process creates a hierarchical dendrogram in a bottomup way, representing the structure of the network at all scales. Here we note, that a direct optimization is also possible for our quality function at a fixed number of groups, creating a clustering. As described in the Supplementary Information in details, our coarsegraining scheme comprises also the case of the overlapping clustering, since it is straightforward to assign a given node to multiple groups as well. As noted there, when considering nonoverlapping partitionings with a given number of clusters, our method gives back the degreecorrected stochastic blockmodel of Karrer and Newman^{25} due to the degreepreservation. Consequently, our coarsegraining approach can be viewed as an overlapping and hierarchical reformulation and generalization of this successful stateoftheart technique.
Hierarchical layout
Although the introduced coarsegraining scheme may be of significant interest whenever probabilistic matrices appear, here we focus on its application for network layout, to obtain a hierarchical visualization^{58,59,60,61,62,63}. Our bottomup coarsegraining results can be readily incorporated into the network layout scheme in a topdown way by initially starting with one node (comprising the whole system), and successively undoing the fusion steps until the original system is recovered. Between each such extension step the layout can be optimized as usual.
We have found, that this hierarchical layout scheme produces significantly improved layouts – in terms of the final D value – compared to a local optimization, such as a simple simulated annealing or NewtonRaphson iteration. By incorporating the coarsegraining in a topdown approach, we first arrange the position of the largescale parts of the network, and refine the picture in later steps only. The refinement steps happen, when the position and extension of the largescale parts have already been sufficiently optimized. After such a refinement step, the nodes – moved together so far – are treated separately. At a given scale (having N ≤ N_{0} nodes), the D value of the coarsegraining provides a lower bound for the D value of the obtainable layout. Our hierarchical visualization approach is illustrated in Fig. 3. with snapshots of the layout and the coarsegrained representation matrices of the Zachary karate club network^{52} at N = 5, 15, 25 and 34. As an illustration on a larger and more challenging network, in Fig. 4. we show the result of the hierarchical visualization on the giant component of the weighted human diseasome network^{64}. In this network we have N_{0} = 516 nodes, representing diseases, connected by mutually associated genes. The colors indicate the known disease groups, which are found to be well colocalized in the visualization.
Discussion
In this paper, we have introduced a unified, information theoretic solution for the longstanding problems of matrix ordering, network visualization and data coarsegraining. While establishing a connection between these separated fields for the first time, our unified framework also incorporates some of the known stateofthe art efficient techniques as special cases. In our framework, the steps of the applied algorithms were derived in an ab inito way from the same first principles, in strong contrast to the large variety of existing algorithms, lacking such an underlying theory, providing also a clear interpretation of the obtained results.
After establishing the general representation theory, in our paper we first demonstrated that the minimization of relative information yields a novel visualization technique, while representing the A input matrix by the B cooccurrence matrix of extended distributions, embedded in a ddimensional space. As another application of the same approach, we obtained a hierarchical coarsegraining scheme, when the input matrix is represented by its subsequently coarsegrained versions. Since these applications are two sides of the same representation theory, they turned out to be superbly compatible, leading to an even more powerful hierarchical visualization technique, illustrated on the realworld example of the human diseasome network. Although we have focused on the visualization in ddimensional flat, continuous space, the representation theory can be applied more generally, incorporating also the case of curved or discrete embedding spaces. As a possible future application, we mention the optimal embedding of a (sub)graph into another graph.
We have also shown that our relative entropybased visualization with e.g. Gaussian node distributions can be naturally interpreted as a forcedirected method. Traditional force directed methods prompted huge efforts on the computational side to achieve scalable algorithms applicable for the large data sets in real life. Here we can not and do not wish to compete with such advanced techniques, but we believe that our approach can be a good starting point for further scalable implementations. As a first step towards this goal, we have outlined the possible future directions of computational improvement. Moreover, in the Supplementary Information we illustrated the applicability of our approach on larger scale networks as well. We have also demonstrated, that network visualization is already interesting in one dimension yielding an ordering for the elements of the network. Our efficient coarsegraining scheme can also serve as an unbiased, resolutionlimitfree, starting point for the infamously challenging problem of community detection by selecting the best cut of the dendrogram based on appropriately chosen criteria.
Our data representation framework has a broad applicability, starting form either the nodenode or edgeedge adjacency matrices or the edgenode incidence matrix of weighted networks, incorporating also the cases of bipartite graphs and hypergraphs. We believe, that our unified representation theory is a powerful tool to gain a deeper understanding of the huge data matrices in science, beyond the limits of existing heuristic algorithms. Since in this paper our primary intention was merely to demonstrate a proof of concept study of our theoretical framework, more detailed analyses of interesting complex networks will be the subject of forthcoming articles.
Methods
We use the most general form of the input matrices, without assuming their normalization. In accordance, there is no need to normalize the information theoretic measures over , such as the information content or the mutual information between the rows and columns of A given by , where and . If we start with the H edgenode cooccurrence (incidence) matrix instead, suited to describe hypergraphs as well, then A ~ H^{T} H is simply given by the elements, , where . This way we generally have nonzero diagonal entries (a_{ii}). See the Supplementary Information for a discussion on the case with strictly zero diagonals.
The parametrization of the Gaussian distributions used in the visualization is the following in ddimensions
For a given graphical representation the B cooccurrence matrix is built up from the overlaps of the distributions ρ_{i} and ρ_{j} – analogously to the construction of A from H – as , where is an (irrelevant) global normalization factor. Although our network layout works only for symmetric adjacency matrices, the ordering can be extended for hypergraphs with asymmetric H matrices as well, since the orderings of the two adjacency matrices H H^{T} and H^{T} H readily yield orderings for both the rows and columns of the matrix, H.
For details of the numerical optimization for visualization and coarsegraining see the Supplementary Information. The codes written in C++ using OpenGL are freely available  as commandline programs  upon request.
Additional Information
How to cite this article: Kovács, I. A. et al. A unified data representation theory for network visualization, ordering and coarsegraining. Sci. Rep. 5, 13786; doi: 10.1038/srep13786 (2015).
References
 1.
Newman, M. E. J. Networks: An Introduction. (Oxford Univ. Press, 2010).
 2.
Albert, R. & Barabási, A.L. Statistical mechanics of complex networks. Reviews of Modern Physics 74, 47–97 (2002).
 3.
Newman, M. E. J. Assortative mixing in networks. Phys. Rev. Lett. 89, 208701 (2002).
 4.
Reshef, D. N. et al. Detecting novel associations in large data sets. Science 334, 1518–1524 (2011).
 5.
Girvan, M. & Newman, M. E. J. Community structure in social and biological networks. Proc. Natl Acad. Sci. USA 99, 7821–7826 (2002).
 6.
Newman, M. E. J. Communities, modules and largescale structure in networks. Nature Physics 8, 25–31 (2012).
 7.
Fortunato, S. Community detection in graphs. Phys. Rep. 486, 75–174 (2010).
 8.
Kovács, I. A., Palotai, R., Szalay, M. S. & Csermely, P. Community landscapes: an integrative approach to determine overlapping network module hierarchy, identify key nodes and predict network dynamics. PLoS ONE 5, e12528 (2010).
 9.
Olhede, S. C. & Wolfe, P. J. Network histograms and universality of blockmodel approximation. Proc. Natl Acad. Sci. USA 111, 14722–14727 (2014).
 10.
Bickel P. J., Chen A. A nonparametric view of network models and NewmanGirvan and other modularities. Proc. Natl. Acad. Sci. USA 106 (50), 21068–21073 (2009).
 11.
Bickel P. J. & Sarkar P. Hypothesis testing for automated community detection in networks. arXiv: 1311.2694. (2013) (Date of access: 15/02/2015).
 12.
King, I. P. An automatic reordering scheme for simultaneous equations derived from network analysis. Int. J. Numer. Methods 2, 523–533 (1970).
 13.
George A. & Liu, J. W.H. Computer solution of large sparse positive definite systems. (PrenticeHall Inc, 1981).
 14.
West, D. B. Introduction to graph theory 2nd edn. (PrenticeHall Inc, 2001).
 15.
Song, C., Havlin, S. & Makse, H. A. Selfsimilarity of complex networks. Nature 433, 392–395 (2005).
 16.
Gfeller, D. & De Los Rios, P. Spectral coarse graining of complex networks. Phys. Rev. Lett. 99, 038701 (2007).
 17.
SalesPardo, M., Guimera, R., Moreira, A. A. & Amaral L. A. N. Extracting the hierarchical organization of complex systems. Proc. Natl. Acad. Sci. USA 104, 15224–15229 (2007).
 18.
Ravasz, E., Somera, A. L., Mongru, D. A., Oltvai, Z. N. & Barabási, A.L. Hierarchical organization of modularity in metabolic networks. Science 297, 1551–1555 (2002).
 19.
Radicchi, F., Ramasco, J. J., Barrat, A. & Fortunato, S. Complex networks renormalization: flows and fixed points. Phys. Rev. Lett. 101, 148701 (2008).
 20.
Rozenfeld, H. D., Song, C. & Makse, H. A. Smallworld to fractal transition in complex networks: a renormalization group approach. Phys. Rev. Lett. 104, 025701 (2010).
 21.
Walshaw, C. A multilevel approach to the travelling salesman problem. Oper. Res. 50, 862–877 (2002).
 22.
Walshaw, C. Multilevel refinement for combinatorial optimisation problems. Annals of Operations Research 131, 325–372 (2004).
 23.
Ahn, Y.Y., Bagrow J. P. & Lehmann S. Link communities reveal multiscale complexity in networks Nature 1038, 1–5 (2010).
 24.
Lancichinetti, A., Fortunate, S. & Radicchi, F. Benchmark graphs for testing community detection algorithms, Phys. Rev. E 78, 046110 (2008).
 25.
Karrer, B. & Newman, M. E. J. Stochastic blockmodels and community structure in networks. Phys. Rev. E 83, 016107 (2011).
 26.
Larremore, D. B., Clauset, A. & Jacobs, A. Z. Efficiently inferring community structure in bipartite networks. Phys. Rev. E 90, 012805 (2014).
 27.
Di Battista, G., Eades, P., Tamassia, R. & Tollis, I. G. Graph Drawing: Algorithms for the Visualization of Graphs. (PrenticeHall Inc, 1998).
 28.
Kobourov, S. G. Spring embedders and forcedirected graph drawing algorithms. arXiv: 1201.3011 (2012) (Date of access: 15/02/2015).
 29.
Graph Drawing, Symposium on Graph Drawing GD'96 (ed North, S. ), (SpringerVerlag, Berlin, 1997).
 30.
Garey, M. R. & Johnson, D. S. Computers and Intractability: A Guide to the Theory of NPCompleteness. (W.H. Freeman and Co., 1979).
 31.
Kinney, J. B. & Atwal, G. S. Equitability, mutual information, and the maximal information coefficient. Proc. Natl. Acad. Sci. USA 111, 3354–3359 (2014).
 32.
Lee, D. D. & Seung, H. S. Learning the parts of objects by nonnegative matrix factorization. Nature 401, 788–791 (1999).
 33.
Slonim, N., Atwal, G. S., Tkačik, G. & Bialek, W. Informationbased clustering. Proc. Natl. Acad. Sci. USA 102, 18297–18302 (2005).
 34.
Rosvall, M. & Bergstrom, C. T. An informationtheoretic framework for resolving community structure in complex networks. Proc. Natl. Acad. Sci. USA 104, 7327–7331 (2007).
 35.
Rosvall, M., Axelsson, D. & Bergstrom, C. T. The map equation. Eur. Phys. J. Special Topics 178, 13–23 (2009).
 36.
Zanin, M., Sousa, P. A. & Menasalvas, E. Information content: assessing mesoscale structures in complex networks. Europhys. Lett. 106, 30001 (2014).
 37.
Allen, B., Stacey, B. C. & BarYam, Y. An informationtheoretic formalism for multiscale structure in complex systems. arXiv: 1409.4708 (2014) (Date of access: 15/02/2015).
 38.
Lovász, L. Large networks and graph limits, volume 60 of American Mathematical Society Colloquium Publications. American Mathematical Society, Providence, RI, (2012).
 39.
Kullback, S. & Leibler, R. A. On information and sufficiency. Annals of Mathematical Statistics 22, 79–86 (1951).
 40.
Hinton, G. & Roweis, S. Stochastic Neighbor Embedding, in Advances in Neural Information Processing Systems, Vol. 15, 833840 (The MIT Press, Cambridge, 2002).
 41.
van der Maaten, L. & Hinton, G. Visualizing Data using tSNE, Journal of Machine Learning Research 9, 2579–2605 (2008).
 42.
Yamada, T., Saito, K. & Ueda, N. Crossentropy directed embedding of network data, Proceedings of the 20th International Conference on Machine Learning (ICML2003), 832–839 (2003).
 43.
Grünwald, P. D. The Minimum Description Length Principle, (MIT Press, 2007).
 44.
Cover, Th. M. & Thomas, J. A. Elements of Information Theory 1st edn, Lemma 12.6.1, 300–301 (John Wiley & Sons, 1991).
 45.
Barnes, J. & Hut, P. A hierarchical O(NlogN) forcecalculation algorithm. Nature 324, 446–449 (1986).
 46.
Gansner, E. R., Koren, Y. & North, S. in Graph drawing by stress majorization, Vol. 3383 (ed Pach, J. ), 239–250 (SpringerVerlag, 2004).
 47.
Fruchterman, T. M. & Reingold, E. M. Graph Drawing by ForceDirected Placement, Software: Practice & Experience 21, 1129–1164 (1991).
 48.
Kamada, T. & Kawai, S. An algorithm for drawing general undirected graphs. Information Processing Letters (Elsevier) 31, 7–15 (1989).
 49.
Estévez, P. A., Figueroa, C. J. & Saito, K. Crossentropy embedding of highdimensional data using the neural gas model. Neural Networks 18, 727–737 (2005).
 50.
van der Maaten, L. J. P. Accelerating tSNE using TreeBased Algorithms. Journal of Machine Learning Research 15, 3221–3245 (2014).
 51.
Hopcroft, J. & Tarjan, R. E. Efficient planarity testing. Journal of the Association for Computing Machinery 21, 549–568 (1974).
 52.
Zachary, W. W. An information flow model for conflict and fission in small groups. Journal of Anthropological Research 33, 452–473 (1977).
 53.
Leskovec, J., Kleinberg, J. & Faloutsos, C. Graph Evolution: Densification and Shrinking Diameters. ACM Transactions on Knowledge Discovery from Data (ACM TKDD), 1(1), (2007). Data is available at: http://snap.stanford.edu/data/caHepPh.html.
 54.
Boguña, M., PastorSatorras, R., DiazGuilera, A. & Arenas, A. Models of social networks based on social distance attachment. Phys. Rev. E 70, 056122 (2004). Data is available at: http://deim.urv.cat/alexandre.arenas/data/welcome.htm.
 55.
Kullback, S. Information Theory and Statistics, (John Wiley: New York, NY, USA, 1959).
 56.
Kapur, J. N. & Kesavan, H. K. The inverse MaxEnt and MinxEnt principles and their applications, in Maximum Entropy and Bayesian Methods, Fundamental Theories in Physics, Springer: Netherlands, , 39, 433–450 (1990).
 57.
Rubinstein, R. Y. The crossentropy method for combinatorial and continuous optimization. Method. Comput. Appl. Probab. 1, 127–190 (1999).
 58.
Gajer, P., Goodrich, M. T. & Kobourov, S. G. A multidimensional approach to forcedirected layouts of large graphs, Computational Geometry: Theory and Applications 29, 3–18 (2004).
 59.
Harel, D. & Koren, Y. A fast multiscale method for drawing large graphs. J. Graph Algorithms and Applications 6, 179–202 (2002).
 60.
Walshaw, C. A multilevel algorithm for forcedirected graph drawing. J. Graph Algorithms Appl. 7, 253–285 (2003).
 61.
Hu, Y. F. Efficient and high quality forcedirected graph drawing. The Mathematica Journal 10, 37–71 (2006).
 62.
SzalayBekö, M., Palotai, R., Szappanos, B., Kovács, I. A., Papp, B. & Csermely P., ModuLand plugin for Cytoscape: determination of hierarchical layers of overlapping network modules and community centrality. Bioinformatics 28, 2202–2204 (2012).
 63.
Six, J. M. & Tollis, I. G. in Software Visualization, Vol. 734, (ed Zhang, K. ) Ch. 14, 413–437 (Springer: US, , 2003).
 64.
Goh, K.I. et al. The human disease network. Proc. Natl. Acad. Sci. USA 104, 8685–8690 (2007).
Acknowledgements
We are grateful to the members of the LINKgroup (www.linkgroup.hu) and E. Güney for useful discussions. This work was supported by the Hungarian National Research Fund under grant Nos. OTKA K109577, K115378 and K83314. The research of IAK was supported by the European Union and the State of Hungary, cofinanced by the European Social Fund in the framework of TÁMOP 4.2.4. A/211120120001 'National Excellence Program'.
Author information
Affiliations
Wigner Research Centre, Institute for Solid State Physics and Optics, H1525 Budapest, P.O.Box 49, Hungary
 István A. Kovács
Institute of Theoretical Physics, Szeged University, H6720 Szeged, Hungary
 István A. Kovács
Center for Complex Networks Research and Department of Physics, Northeastern University, 177 Huntington Avenue, Boston, MA 02115, USA
 István A. Kovács
Institute of Organic Chemistry, Research Centre for Natural Sciences, Hungarian Academy of Sciences, Pusztaszeri út 5967, H1025 Budapest, Hungary
 Réka Mizsei
Department of Medical Chemistry, Semmelweis University, H1444 Budapest, P.O.Box 266, Hungary
 Péter Csermely
Authors
Search for István A. Kovács in:
Search for Réka Mizsei in:
Search for Péter Csermely in:
Contributions
I.A.K. and R.M. conceived the research and ran the numerical simulations. I.A.K. devised and implemented the applied algorithms. I.A.K. and P.Cs. wrote the main manuscript text. All authors reviewed the manuscript.
Competing interests
The authors declare no competing financial interests.
Corresponding author
Correspondence to István A. Kovács.
Supplementary information
PDF files
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Further reading

The EntOptLayout Cytoscape plugin for the efficient visualization of major protein complexes in protein–protein interaction and signalling networks
Bioinformatics (2019)

Sensory Neuropathy Affects Cardiac miRNA Expression Network Targeting IGF1, SLC2a12, EIF4e, and ULK2 mRNAs
International Journal of Molecular Sciences (2019)

MicroRNA interactome analysis predicts posttranscriptional regulation of ADRB2 and PPP3R1 in the hypercholesterolemic myocardium
Scientific Reports (2018)

Spectral Entropies as InformationTheoretic Tools for Complex Network Comparison
Physical Review X (2016)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.