E-mail leaves a robust trace of the interactions between two people.© NSU

Want to know how your organization really works - who speaks to whom, who holds the power? Then study the flow of internal e-mail, say scientists at global technology firm Hewlett-Packard.

The researchers have developed a way to use e-mail exchanges to build a map of the structure of an organization. The map shows the teams in which people actually work, as opposed to those they are assigned to.

The technique can also reveal who is at the heart of each sub-group. These people often correspond with company-designated leaders such as project managers. But unofficial de facto leaders can also emerge. The approach might even help to pinpoint the heads of criminal or terrorist networks.

Communities of practice

It has long been recognized that big institutions tend to divide organically into informal collaborative networks, called communities of practice.

For example, colleagues in one department might all tend to consult one particularly useful person in another department, linking the group into a community of practice. Such collaborations are very common in scientific research. Groups in different universities share information, skills and expertise to solve problems.

But communities of practice are difficult to identify - the process typically involves laborious interviews and surveys.

E-mail, however, leaves a robust trace of the interactions between two people. If you want to know what they said, privacy issues could pose obstacles. But simply to know that they communicate, all you need are the names of the sender and the recipient, say Joshua Tyler and colleagues at Hewlett-Packard's labs in Palo Alto, California.

Tyler's group uses this information to construct a communications graph in which lines - each denoting a direct e-mail exchange - link nodes that correspond to individuals. Next the researchers use a computer to search for links with high 'betweenness'. These are the few connections between groups of highly connected nodes. Removing them decomposes the graph into a collection of isolated clusters of nodes, which correspond to the communities.

There are several tricks to this reduction process. For example, taking out one link alters the others' betweenness, which must be recalculated at each step. And the process has to stop before the communities themselves get fragmented.

Tyler and colleagues tested their community-finding algorithm on a set of nearly 200,000 e-mails exchanged between 485 employees of the Hewlett-Packard labs over three months.

The graph created from this data is a 'small world' - any node can be reached from any other by just a few steps.

The researchers found 66 communities. They asked 16 employees how well the method had identified their community of practice. Most responded along the lines of "yes, that's pretty much our project team", even though the communities often crossed the formal departmental boundaries defined by the company.

Similar techniques for extracting information about web community structure from emails have been suggested by other groups too. Yukio Ohsawa of the University of Tsukuba in Japan has developed a method called KeyGraph that looks for connectivity patterns similar to the 'betweenness' of the Hewlett-Packard team, which has been used by businesses for at least three years². And a team at the Universitat Rovira i Virgili in Tarragona, Spain, have looked at the email network of about 1,700 researchers at their university to deduce its community structure¹. 

• References

  1. Tyler, J. R., Wilkinson, D. M. & Huberman, B. A. Email as spectroscopy: automated discovery of community structure within organizations. Preprint http://xxx.lanl.gov/arXiv:cond-mat/0303264, (2003).
  2. Ohsawa, Y. et al. Featuring web communities based on word co-occurrence structure of communications. published online, url: http://www2002.org/CDROM/refereed/105/index.html (2002).
  3. Guimera, R., Danon, L., Diaz-Guilera, A., Giralt, F. & Arenas, A. Self-similar community structure in organisations. Preprint, http://xxx.arxiv.org/abs/cond-mat/0211498. (2002).