A social interaction field model accurately identifies static and dynamic social groupings


Identifying whether people are part of a group is essential for humans to understand social interactions in social activities. Previous studies have focused mainly on the perceptual grouping of low-level visual features. However, very little attention has been paid to grouping in social scenes. Here we implemented virtual reality technology to manipulate characteristics of avatars in virtual scenes. We found that closer interpersonal distances, more direct interpersonal angles and more open avatar postures led to a higher probability of a group being judged as interactive. We developed a social interaction field model that describes a front−back asymmetric social interaction field. This model accurately predicts participants’ perceptual judgements of social grouping in real static and dynamic social scenes. Our findings indicate that the social interaction field model is an efficient computational framework for analysing social interactions and provides insight into how human observers perceive the interactions of others, enabling the identification of social groups.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Social interaction field deduced from behavioural data.

Body of the avatar in a, Unity Technologies

Fig. 2: The impact of postural openness on interaction judgement.

Body of the avatar in a, Unity Technologies

Fig. 3: The extension of the SIFM to multiple people.

Panel c, University of Reading

Fig. 4: SIFM predictions for social grouping in static and dynamic scenes.

Images in a adapted from University of Reading (left); ref. 54 and ref. 59, Springer (middle); and ref. 55, IEEE, The Technologies of Vision (TeV) team of Fondazione Bruno Kessler (right)

Fig. 5: SIFM predictions for actual human interaction behaviours.

Data availability

All data that support our findings are publicly available at https://figshare.com/s/e860b1ee96956dbd0a5c.

Code availability

The code for data analysis and modelling is available at https://github.com/ECNU-VRLAB/Social-Interaction-Field-Model.


  1. 1.

    Gallup, A. C. et al. Visual attention and the acquisition of information in human crowds. Proc. Natl Acad. Sci. USA 109, 7245–7250 (2012).

  2. 2.

    Henderson, J. M. & Hollingworth, A. High-level scene perception. Annu. Rev. Psychol. 50, 243–271 (1999).

  3. 3.

    Fletcher-Watson, S., Findlay, J. M., Leekam, S. R. & Benson, V. Rapid detection of person information in a naturalistic scene. Perception 37, 571–583 (2008).

  4. 4.

    De Jaegher, H., Di Paolo, E. & Gallagher, S. Can social interaction constitute social cognition? Trends Cogn. Sci. 14, 441–447 (2010).

  5. 5.

    Kendon, A. Conducting Interaction: Patterns of Behavior in Focused Encounters (Cambridge Univ. Press, 1990).

  6. 6.

    Wagemans, J. et al. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychol. Bull. 138, 1172–1217 (2012).

  7. 7.

    Wagemans, J. et al. A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations. Psychol. Bull. 138, 1218–1252 (2012).

  8. 8.

    Field, D. J., Hayes, A. & Hess, R. F. Contour integration by the human visual system: evidence for a local “association field”.Vis. Res. 33, 173–193 (1993).

  9. 9.

    Kubovy, M. & Wagemans, J. Grouping by proximity and multistability in dot lattices. Psychol. Sci. 6, 225–234 (1995).

  10. 10.

    Grossberg, S. & Pessoa, L. Texture segregation, surface representation and figure−ground separation. Vis. Res. 38, 2657–2684 (1998).

  11. 11.

    Wagemans, J. in The Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience Vol. 2 (ed. Wixted, J. T.) 803–872 (Wiley, 2018).

  12. 12.

    Hall, E. T. in The Hidden Dimension (eds. McNickle, R. et al.) 113–125 (Doubleday, 1966).

  13. 13.

    Neri, P., Luu, J. Y. & Levi, D. M. Meaningful interactions can enhance visual discrimination of human agents. Nat. Neurosci. 9, 1186–1192 (2006).

  14. 14.

    Ding, X., Gao, Z. & Shen, M. Two equals one: two human actions during social interaction are grouped as one unit in working memory. Psychol. Sci. 28, 1311–1320 (2017).

  15. 15.

    Hayduk, L. A. Personal space: an evaluative and orienting overview. Psychol. Bull. 85, 117–134 (1978).

  16. 16.

    Sanchez-Vives, M. V. & Slater, M. From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6, 332–339 (2005).

  17. 17.

    Parsons, T. D., Gaggioli, A. & Riva, G. Virtual reality for research in social neuroscience. Brain Sci. 7, 1–21 (2017).

  18. 18.

    Banakou, D., Groten, R. & Slater, M. Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc. Natl Acad. Sci. USA 110, 12846–12851 (2013).

  19. 19.

    Claessens, P. M. & Wagemans, J. Perceptual grouping in Gabor lattices: proximity and alignment. Percept. Psychophys. 67, 1446–1459 (2005).

  20. 20.

    Apfelbaum, E. P., Sommers, S. R. & Norton, M. I. Seeing race and seeming racist? Evaluating strategic colorblindness in social interaction. J. Pers. Soc. Psychol. 95, 918–932 (2008).

  21. 21.

    Glick, P. & Fiske, S. T. in Revisioning Gender (eds Ferree, M. M. et al.) 365–398 (AltaMira, 1999).

  22. 22.

    Reis, H. T., Maniaci, M. R., Caprariello, P. A., Eastwick, P. W. & Finkel, E. J. Familiarity does indeed promote attraction in live interaction. J. Pers. Soc. Psychol. 101, 557–570 (2011).

  23. 23.

    Reis, H. T., Nezlek, J. & Wheeler, L. Physical attractiveness in social interaction. J. Pers. Soc. Psychol. 38, 604–617 (1980).

  24. 24.

    Hayduk, L. A. The shape of personal space: an experimental investigation. Can. J. Behav. Sci. 13, 87–93 (1981).

  25. 25.

    Argyle, M. Bodily Communication (Routledge, 2013).

  26. 26.

    Hall, J. A., Coats, E. J. & LeBeau, L. S. Nonverbal behavior and the vertical dimension of social relations: a meta-analysis. Psychol. Bull. 131, 898–924 (2005).

  27. 27.

    Vacharkulksemsuk, T. et al. Dominant, open nonverbal displays are attractive at zero-acquaintance. Proc. Natl Acad. Sci. USA 113, 4009–4014 (2016).

  28. 28.

    Hall, E. T. A system for the notation of proxemic behavior. Am. Anthropol. 65, 1003–1026 (1963).

  29. 29.

    Ellyson, S. L. & Dovidio, J. F. in Power, Dominance, and Nonverbal Behavior (eds. Ellyson, S. L. & Dovidio, J. F.) 1–27 (Springer, 1985).

  30. 30.

    Hartuv, E. & Shamir, R. A clustering algorithm based on graph connectivity. Inform. Process. Lett. 76, 175–181 (2000).

  31. 31.

    Chen, K., Loy, C. C., Gong, S. & Xiang, T. Feature mining for localised crowd counting. In Proc. British Machine Vision Conference 2012 1–11 (BMVC, 2012).

  32. 32.

    Zen, G., Lepri, B., Ricci, E. & Lanz, O. Space speaks—towards socially and personality aware visual surveillance. In Proceedings of the 1st ACM International Workshop on Multimodal Pervasive Video Analysis 37–42 (ACM, 2010).

  33. 33.

    Sekuler, A. B. Motion segregation from speed differences: evidence for nonlinear processing. Vis. Res. 30, 785–795 (1990).

  34. 34.

    Feng, L. & Bhanu, B. Understanding dynamic social grouping behaviors of pedestrians. IEEE J. Sel. Top. Signa. 9, 317–329 (2015).

  35. 35.

    Palomares, N. A. Toward a theory of goal detection in social interaction. Commun. Res. 35, 109–148 (2008).

  36. 36.

    Watson, A. C., Nixon, C. L., Wilson, A. & Capage, L. Social interaction skills and theory of mind in young children. Dev. Psychol. 35, 386–391 (1999).

  37. 37.

    Kelly, H. H. & Stahelski, A. J. Social interaction basis of cooperators’ and competitors’ beliefs about others. J. Pers. Soc. Psychol. 16, 66–91 (1970).

  38. 38.

    Dovidio, J. F., Kawakami, K. & Gaertner, S. L. Implicit and explicit prejudice and interracial interaction. J. Pers. Soc. Psychol. 82, 62–68 (2002).

  39. 39.

    Mondloch, C. J. et al. Face perception during early infancy. Psychol. Sci. 10, 419–422 (1999).

  40. 40.

    Rosenthal, T. L. & Zimmerman, B. J. Social Learning and Cognition (Academic, 2014).

  41. 41.

    Wellman, H. M. Making Minds: How Theory of Mind Develops (Oxford Univ. Press, 2014).

  42. 42.

    Mehrabian, A. Inference of attitudes from the posture, orientation, and distance of a communicator. J. Consult. Clin. Psychol. 32, 296–308 (1968).

  43. 43.

    Kersten, D., Mamassian, P. & Yuille, A. Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004).

  44. 44.

    Wellman, H. M., Cross, D. & Watson, J. Meta-analysis of theory-of-mind development: the truth about false belief. Child Dev. 72, 655–684 (2001).

  45. 45.

    Adolphs, R. The social brain: neural basis of social knowledge. Annu. Rev. Psychol. 60, 693–716 (2009).

  46. 46.

    Helbing, D. & Molnár, P. Social force model for pedestrian dynamics. Phys. Rev. E 51, 4282–4286 (1995).

  47. 47.

    Lewin, K. Field Theory in Social Science: Selected Theoretical Papers. (Harpers, 1951).

  48. 48.

    Gessaroli, E., Santelli, E., di Pellegrino, G. & Frassinetti, F. Personal space regulation in childhood autism spectrum disorders. PLoS ONE 8, 1–8 (2013).

  49. 49.

    Mehran, R., Oyama, A. & Shah, M. Abnormal crowd behavior detection using social force model. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops 935–942 (IEEE, 2009).

  50. 50.

    Qin, Z. & Shelton, C. R. Social grouping for multi-target tracking and head pose estimation in video. IEEE T. Pattern Anal. 38, 2082–2095 (2016).

  51. 51.

    Chen, X., Qin, Z., An, L. & Bhanu, B. Multi-person tracking by online learned grouping model with non-linear motion context. IEEE T. Circ. Syst. Vid. 26, 2226–2239 (2016).

  52. 52.

    Jung, E. et al. The influence of human body orientation on distance judgments. Front. Psychol. 7, 1–9 (2016).

  53. 53.

    Vestner, T., Tipper, S. P., Hartley, T., Over, H. & Rueschemeyer, S.-A. Bound together: social binding leads to faster processing, spatial distortion, and enhanced memory of interacting partners. J. Exp. Psychol. Gen. https://doi.org/10.1037/xge0000545 (2019).

  54. 54.

    Loy, C. C., Ke, C., Shaogang, G. & Xiang, T. in Modeling, Simulation and Visual Analysis of Crowds (eds. Ali, S. et al.) 347–382 (Springer, 2013).

  55. 55.

    Setti, F., Lanz, O., Ferrario, R., Murino, V. & Cristani, M. Multi-scale f-formation discovery for group detection. In 2013 IEEE International Conference on Image Processing 3547–3551 (IEEE, 2013).

  56. 56.

    Brainard, D. H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

  57. 57.

    Kleiner, M. et al. What’s new in Psychtoolbox-3? Perception 36, ECVP Abstract Supplement (2007).

  58. 58.

    Weibull., W. A statistical distribution function of wide applicability. J. Appl. Mech. 18, 293–297 (1951).

  59. 59.

    Kumagai S., Hotta K. (2014) HLAC between cells of HOG feature for crowd counting. in Advances in Visual Computing ISVC 2014. Lecture Notes in Computer Science, vol 8887 (eds. Bebis, G. et al.) (Springer, Cham, 2014).

Download references


This research was supported by the National Natural Science Foundation of China (grant no. 31771209). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We are grateful for the helpful comments on our manuscript from Y. Zhou and L. Li and for technical support from X.-M. Wang, M.-C. Miao and H.-N. Wu.

Author information

S.-G.K., C.Z., M.H. and Q.L. designed the research. Q.L., M.H., C.Z. and Y.-F.H. collected experimental data. M.H., Q.L., C.Z. and Y.-F.H. derived the models and analysed data under the supervision of S.-G.K. S.-G.K., C.Z., Q.L. and M.H. wrote the paper.

Correspondence to Shu-Guang Kuai.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary Information

Supplementary Methods 1 and 2, Supplementary Figs. 1−4, Supplementary Table 1, and Supplementary References.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Further reading