Introduction

These are times of ubiquitous pattern recognition, even if what counts as a pattern has multiple answers. Some patterns are complex analytical instruments involved with data mining, not without relation to military and security applications. There are patterns of stimulus-response or behavioral repetition, indexing a psychological or socio-technical reality; patterns like fractal mountains and whorls in sea shells generated by forces of nature. Finally, there are perceptual patterns people design and produce and learn to “recognize” in a different way (such as the ornamental patterns of textiles voluntarily produced for an aesthetic function). It is easy for the homonym to wreak havoc: nature and culture, machine and human, form and information, repetition and singularity, space and time, objective reality and subjective artifact of a mental kind all comprehended in one term with messily entangled meanings.

Pattern recognition has not always been as tightly associated with machines as it is today. Even currently, humans are often credited with the ability to recognize patterns, though it is not immediately clear how this ability relates to recognition by machines. The comparison encompasses not only the abilities of humans and machines but also the underlying relationships that go into recognition, such as perception, language, and memory in humans or their functional equivalents in machines. Ray Kurzweil, a computer scientist and futurist known for promoting the singularity hypothesis, suggests that “human beings have only a weak ability to process logic but a very deep core capability of recognizing patterns” (Kurzweil 2012, p. 38). This assertion contrasts with machinic or algorithmic cognitive processes, which excel in logical and symbolic manipulation but have relatively weaker “core capabilities” in pattern recognition. Likewise, Robert Goldstone, in the context of an introductory course, states, “humans’ ability to recognize patterns is what separates us most from machines” (Goldstone, p. 1), thus considering pattern recognition as a uniquely human capacity while placing pattern recognition by machine as a separate domain.Footnote 1

On the other hand, cognitive psychology, rooted in cybernetic modeling, often takes a more equivocal approach, providing definitions applicable to both humans and machines. According to cognitive psychology, “Pattern recognition is the ability to pick out and organize some stimuli which can then be identified from long-term memory” (Lund 2002, p. 72) and “the ability to abstract and integrate certain elements of a stimulus into an organized scheme for memory storage and retrieval” (Solso as cited in Lund 4). Despite not challenging their information-theoretical premises, these formulations operate on a cognitive level that accommodates human aesthetics in terms of time, scale, and form. This openness allows for investigations informed by aesthetic concerns, epistemological reflections, design discourse, and media theory.

The complexity and the intellectual fertility of the human-machine encounter in the case of patterns and pattern recognition can be traced back to the time of the rise of computation as a cultural and technical phenomenon, and more specifically, its claims to model human perception and intelligence. In the American context, in fields like architecture, media theory, art education, and social sciences, a more and more informationally defined pattern starts to mediate reflection across the board starting around the 1950s. A list of influential works with stakes in a concept of informational pattern may better indicate its widespread adoption and its centrality to important intellectual developments of the second half of the last century in American culture and beyond: György Kepes’s The New Landscape in Art and Science (1956), Edward Hall’s The Silent Language (1959), Christopher Alexander’s Notes on the Synthesis of Form (1964) and A Pattern Language (1977), Marshall McLuhan’s Understanding Media (1964), as well as Gregory Bateson’s Steps to an Ecology of Mind (1972) all depend in various degrees on the promises of insight they attach to the category of patterns for their respective arguments. Similar claims have been made for the natural science research of the same period, as in Lorraine Daston and Peter Galison’s observation about a “celebration of the human ability to see patterns” in the 1950s–1960s scientific context. In this period, conceptualizations of an inherently human “ability to see patterns” and pattern recognition as a computational goal develop alongside one another and do not neutralize each other.

The cultural moment defined by these works and their social conditions has received many illuminating treatments; but for this partial account, works by Orit Halpern, Reinhold Martin, Larry Busbea, and Seb Franklin deserve mention in that each has a sense of how the concerns of the late fifties to early seventies discourse in cybernetics, media, and architectural theory find echoes in the present moment, concerns aptly embodied by the pattern concept. In different versions of what looks like a single entangled story, a primarily visual reception of pattern constitutes a key player in the stories and critical accounts of the 50s to 70s culture offered by these authors, whether it is about patterns as abstractions in new methodologies of social science (the work of Edward Hall in Busbea’s case), patterns as driving ambitious but flawed alliances between science and visuality (Martin’s Kepes), or the unpacking of a generalized and hegemonic model of cybernetic cognition (Halpern and Franklin). Of the four scholars, Busbea offers a strong version of continuity between then and now, whereby, as he claims, “there is virtually no theme, practice, or technological advance being addressed today that was not discussed at length at the end of the 1960s and the beginning of the next decade” (Busbea 2020, p. xvi). Similarly, Martin singles out the organicist features of the architectural elements developed around this period-- “patterned modularities”-- as “haunt(ing) all debates in today’s digital age, which is to say today’s globalized age, an age that began in both senses in the wartime cauldron of technological innovation and market realignment” (Martin 2005, p. 13). If this period that still haunts the present in significant ways made such a diverse and pervasive use of patterns, it is perhaps the notion of pattern that constitutes one of its prime legacies. Despite the potential vagueness of its “metadisciplinary shifter” function (Busbea, p. 46), the key elements that allowed it to respond to different cultural needs may also be serving similar ends today: The debates it sparks regarding the differences and operational similarities between machines and humans still strike a chord. Its role as a tool for managing information overload remains unchanged since Kepes and McLuhan’s time. The peculiar morphologies it conjures continue to evoke a sense of totality. Its scalability, adept at shifting between vastly different scales, remains relevant in a world grappling with ever-larger and increasingly impactful systemic effects in the environment, geopolitics, technology, and economics.Footnote 2 The computation that attended its emergence has come a long way after exponential development, but the impulse to find digital or data-sorting answers to all kinds of problems endures. In the complex move from form to information, analog to digital, and Gestalt to system it constitutes, the story of pattern has as good a claim as any to offer a prehistory of the present.

The following account offers interlinked snapshots conveying the tensions that attended the emergence of pattern recognition as a general framework for describing wholes in multiple domains, ranging from the human organism through the grasp of social structure to the psychology of perception. The cases it takes up to this end are the cybernetician Norbert Wiener’s definition of a “pattern,” cultural anthropological visions of the whole, and finally, the arrival of a pattern recognition concept defined by an alliance between machine learning and cognitive psychology. On the one hand, the discussion traces a process that starts by drawing models from the human to transfer to the machine, only to end up establishing machinic pattern recognition as a primary model to make sense of human cognition: the shift from the “unmarked” human to the unmarked machine. In charting this movement, the article attributes major importance to intermediary conceptual formations, as will be seen in Bateson’s tripartite classification of digital, analog, and Gestalt coding. On the other hand, the discussion also outlines the complementary cybernetic ontology that grounds its notion of biological individuation on information. Exploring the context of reception of the pattern concept thanks to a series of figures ranging from science fiction authors, social scientists like Margaret Mead, to media theorists like Marshall McLuhan, there comes into view a whole cultural and epistemic landscape that tries to negotiate its relation to the newly emerging informational paradigm with its computational tools and respond to the shift from qualitative to quantitative modes of knowing.

A translation ploy

The pattern concept gained renewed credibility following advances in information theory and cybernetics. Based on a mathematical order and organization beyond the particularities of material and situation, the cybernetic concept of pattern often supported a vision of human communication that presupposed transferability across media and material support.Footnote 3 This vision found its best representation in the thought experiment Norbert Wiener offered of traveling by telegraph, a “phantasy at the service of philosophy” (Wiener 1989, 95). The thrust of Wiener’s argument, a momentous one, is that “the distinction between material transportation and message transportation is not in any theoretical sense permanent and unbridgeable” (Wiener 1989, 98), heralding so many sci-fi scenarios of teleportation. Extended further, the argument finds form in the speculative question of what would happen if we were “to transmit the whole pattern” of the human body and the brain. It bears noting that the whole intervention cybernetics makes in the problem of individuation becomes visible in the thought experiment and its associated concept of pattern: patterns from then on become the native abstractions of a reality transformed into information.Footnote 4

Wiener’s early concept of informational pattern assumes the priority of systemic relationships over the elements that compose a given whole. As he claims, “One of the most interesting aspects of the world is that it can be considered to be made of patterns. A pattern is essentially an arrangement. It is characterized by the order of the elements of which it is made rather than the intrinsic nature of these elements” (Wiener 1950, p. 3). The travel by telegraph thought experiment is present in all the editions of The Human Use of Human Beings, but this speculative definition seems to have been dropped from the second edition onward, probably because of its looming generality. A shift of emphasis in later editions modifies this generality and limits Wiener’s “pattern” to the dealings with biological organisms from an informational perspective.Footnote 5 Pattern becomes an index and a potential extract of “organization,” while the latter inscribes itself as a message: “A pattern is a message, and may be transmitted as a message.” Thus, it becomes possible for Wiener to write, “We are not stuff that abides, but patterns that perpetuate themselves,” with the whole question of personal identity and individuality registered therein (Wiener 1989, p. 96).

This type of organizational pattern plays a significant role in the "saturation" of culture by information theory and cybernetics, with lasting consequences (Geroulanos and Weatherby, 2020, p. 1). What’s more, Wiener offers his definition of pattern as linked with a self-definition of cybernetics and in a public rather than specialist context, which means he introduces it to a milieu where there are already other conceptions of the primacy of wholes over their elements. A prominent one among such conceptions is Gestalt theory. Cybernetics and information theory, along with the notion of pattern they produced, carved their proper cultural niche concomitantly with the withdrawal of the Gestalt paradigm that they helped displace in a relation of succession and overlap.

Steven Helmling once made the interesting suggestion that pattern recognition is an “anglicization” of Gestalt and, considering how cybernetics and Gestalt theory each negotiated the difference between their primary domain and the domains they explored analogically, the echoes are hard to deny.Footnote 6 What cybernetics and information theory did was give a formal language to problems of form and purposiveness already at play in debates across psychology, biology, and last but not least, anthropology, areas which Gestalt theory had either adopted as first-degree concern or with which it had also made contact. Perhaps the most decisive difference between Gestalt theory, on the one hand, and cybernetics-information theory alliance, on the other, is their relation to technology, whereby cybernetics often comes down on the side of the mechanizability of functions taken to be human; a strong confidence in the capacity of mathematical formalism to result in operational reduction and obtain engineering solutions is an essential part of cybernetics. On a closely related level, the proper conceptualization of feedback relations in cybernetics gave weight to the grammar of this mode of establishing relationships, wholes, and systems, in a way not true for Gestalt theory, where the concept of “field” seemed to be the organizing concern. In rivalries of explanatory power, proofs of engineering carry tremendous cultural weight, such that even Gestalt wholes can be formalized. In the brief but critical exchange between Gestalt and cybernetics, the latter would “explain” on a new basis (or at least make a pass at doing so) the conclusions of the former, and this has momentous implications for pattern recognition and its genealogy.

The participants of the Macy conferences constitute the center of influence in this overall tendency, and it is possible to trace an arc from Warren McCulloch and Walter Pitts’s “How We Know Universals,” through Wiener’s Cybernetics, to ultimately reach Bateson’s Mind and Nature.Footnote 7 In each of these works, formal cybernetic explanations are brought to bear on phenomena that constituted crucial cases for Gestalt theory like outlines and the identification of geometric shapes.Footnote 8 It is not a coincidence that scholars like Wolf Kittler attribute special significance to the Macy milieu: “I think it is safe to assume that this was the place were Norbert Wiener picked up the term gestalt for his own purposes, which, in the age of electronic data processing, would soon lead to automated systems of pattern recognition” (Kittler, 2008, p. 83). In this sense, “the abiding interest” in patterns at the Macy conferences that Bernard Geoghegan also observes, seems to have resulted in fruitful developments (Geoghegan, p. 44).

While cybernetics had its share of mixed relationships (with psychoanalysis, for instance), when it comes to Gestalt, there is a unique and strong sense of attraction and a striking blend of similarities and differences. I have noted Wiener’s tentative definition of pattern above. Similarly, As Geoff Bowker noted, certain cyberneticians considered theirs to be a “science of form,” not unlike the most classical definition of Gestalt theory (Bowker 1993, p. 111). The psychology of perception found in the work of Gestalt theorists of the Berlin school is a fundamental statement on organized wholes; it constitutes an early theorization of patterns in perception, memory, and conceptuality. Figure-ground distinction and effects of wholeness that simple addition among the constituents cannot account for are central concerns of Gestalt theory. Instead of positing a two-tier process of the unformed and meaningless sensation being processed into sense-making units by structures higher in intellectual hierarchy, these scientists held perception itself had inborn formative or organizational capacities that made a “two-step” process unnecessary (Van Campen 1997). Totalities are already given on the level of perception, and facing “the problem of perceptual organization” only “at the next higher story” would just be “too late” (Arnheim 1969, p. 81). This rather technical point should accompany the common idea of Gestalt theory as an advocacy of wholes irreducible to their parts, to give a stronger basis to appreciate the later survivals of, and divergences from it in approaches to pattern recognition as a human capacity.

Problems of social form

Before cybernetics had to make its own deliberations on the reach of its concepts, analogical impulses and extensions of Gestalt theory in the direction of social research frequently took it out of the home field of experimental psychology, and it became vulnerable to being identified with looser forms of holism. If the versatility of pattern and pattern recognition can often border on overdetermination–as in Busbea’s “shifter” designation for the pattern concept– this was also true of the extensions of Gestalt theory. In a prefiguration of the areas of influence and “legitimacy exchanges” of cybernetic models (Bowker 1993), its signature conception of wholes irreducible to their components went on to inform debates in philosophy and the social sciences as well, cropping up in places wherever an understanding of emergent totalities was urgent, leading to an important episode in the trial of morphological thinking with historicity. Although leading proponents like Köhler were unwilling to “throw in a quick extension to social life” (cited in Ash, 2007, p. 257), the reasonable objection that patterns in perception are scales away and perhaps different from patterns in culture and society would often be bypassed in practice.Footnote 9 People like Ernst Jünger were quick to seize on the social implications of Gestalt concepts, such that at some point it became difficult to hang onto a “social form” and deny that it also meant “total mobilization”.Footnote 10 In a reflexive use, Gestalt theory could be directed toward the problems of a turbulent Europe and its holistic visions appropriated by conservative agendas, a situation that led historians like Norton Wise to urge for discernment: “we should avoid coloring holism with a monochromatic Nazi tint” (Wise 2004, p. 228).Footnote 11

A notable example of holism, distinct from a narrow viewpoint, emerges from American cultural anthropologists’ adoption of Gestalt concepts. In their work, they creatively envision complete and interconnected structures that describe both individuals and societies. The concept of culture, as seen in the context of “culturalism,” aligns with a “functionalist” approach that emphasizes integrating cultural elements into a cohesive whole (Hegeman 2001, p. 38). According to Susan Hegeman, these patterns function as components of a “transhistorical social configuration” or the collective personality of a group of people (38). Even before World War II, cultural patterns had captivated numerous social scientists, elevating cultural anthropology’s prominence. In this regard, Gestalt theory provided a valuable framework for understanding foreign cultures. For instance, Ruth Benedict’s seminal work, Patterns of Culture benefited from a Gestalt-inspired structural integration to comprehend different cultures. Benedict believed that “cultures are individual psychology magnified onto a large screen, given immense proportions and a vast time span” (Benedict 1932). Drawing on Gestalt theory, she approached individual cultures as holistic entities. Likewise, Gregory Bateson employed patterns in his anthropological work, predating the emergence of cybernetics. Overall, the incorporation of Gestalt concepts by American cultural anthropologists not only showcases an alternative form of holism but also lays the groundwork for employing “patterns” to enrich descriptive knowledge within the social sciences and humanities.

This concept of patterns as a means of understanding finds a further example when we consider research programs like Edward Hall’s, designed to identify abstract governing patterns for various cultural formations. In both cases, the aim is to provide intelligible overviews: “Hall refused to relinquish the belief that such systems of construction could be observed and analyzed as if from a distance” (Busbea 2020, p. 58). In exploring these overviews, therefore, one encounters the fundamental question of distance. A distinct and contradictory interplay of nearness and distance characterizes the epistemology of wholes and patterns within the social sciences setting of the time: distance could be both enabling and hindering. The context of the Macy conferences where cybernetics emerged evidently contributed to these issues, with the significant example of a discussion of processes of recall, in which Norbert Wiener raises a question that encapsulates the whole context: “Is the observer at a greater disadvantage as he observes something like himself?” (in Pias 2016, p. 125). Wiener believed it must be so. Margaret Mead, on her part, thought that it was “a lot easier to study the culture where you can’t marry people, where there’s such a gulf that that kind of overidentification does not occur,” vividly illustrating the virtue of distance (Mead in Brand 1976, p. 12). Finally, Gregory Bateson had a systems-theoretical sense that the “obvious” was particularly difficult to see, with all that this implies in the way of “diagnosing the machinery of our society” (Bateson 2008: p. 429, p. 436).Footnote 12

In short, even if patterns here are mainly conceived as qualitative cultural totalities, they presented a necessary foil to cybernetics and information theory’s grasp of systemic relations; the way a Gestalt aesthetic serves the conceptual constitution of cultural totalities through calibrations of distance offers a blueprint for an information theoretical datafication of society. As Geoghegan rightly points out, “an ethnographic theory of cultural patterning, developed in the study of colonial subjects and ostensibly dysfunctional American families, provided the model for the integration of information theories and social theories” (Geoghegan, 2023, p. 44).

Before moving on to the pitfalls of patterns in social science research, a dramatic illustration of absolute distance could be useful. The goal of comprehension of cultural totality sought by such an anthropology proved attractive to various areas of artistic production such as sci-fi, where questions of cultural contact and foreignness often played out in different galactic settings. Chad Oliver, who has the distinction of having been an anthropologist and a sci-fi author, imagined a contact scenario in “Rite of Passage” (1951) very similar to the ones found in the kind given by culturalist accounts: in this story, the goal is to rise above individual or particular details and to grasp the pattern of the extraterrestrial culture of the Nerns--an allusion to another classic of American anthropology-- as a whole. Oliver’s protagonist Martin Ashley might be a stand-in for the author himself: “It’s easy to identify various items in a culture…a totem pole here, a spear there, a feather cape somewhere else….Unfortunately, however, all that isn’t too important. It doesn’t tell you much that you need to know if you’re going to understand a culture. What counts is how these things are put together. Cultures are not just collections of random ideas and spear points, you see. They are dynamic, integrated systems—blueprints for living” (Oliver 1951). Ashley all but pronounces the term and it falls to another character to come up with it: “You mean like patterns?” In a way reminiscent of Benedict’s cultural patterns, but this time taken to space, in Oliver’s story sci-fi had a sense of culture as “something all of a piece” (Benedict 1960, p. 34).

Cultural anthropology’s vision of social unity and relativism of values based on it, as well as its use of aesthetic categories, received strong criticism from multiple quarters. According to Johannes Fabian, this kind of research “proposed to study culture with the help of aesthetic concepts such as pattern, style, and configuration” (Fabian 2014, p. 46) with the ultimate effect of undervaluing the proper historicity and temporality of the cultures submitted to such a visual abstraction. Other critics pointed out the paradoxical nature of the relativism obtained from such totalization: in the large scheme of instrumentalization of science in the Cold War, the result is making known the superiority of the relativizer—the American anthropologist that is-- who arranges and compares the cultural sets or totalities with each other, rising above the complexities of indigenous daily existence and historical experience in a given foreign society to a position beyond comparison. Even in a structuralist form, this kind of formalist relativism “gets carried away,” trying to reach a position “which includes all the others and itself” (Merleau-Ponty 2010, p. 14). Notwithstanding these crucial points, it is important to bear in mind that this kind of social science and its particular understanding of pattern kept alive the claims that can be made for a historical morphology, endowing its formulation with the highest stakes. It is not an accident that it was also one of the first formations to bear the brunt of the first waves of quantification in social sciences, having had to tackle the implications of newly burgeoning computational tools of analysis before things like forensic architecture or quantitative history existed.Footnote 13

Apart from these questions of objectifying distance and an insufficient acknowledgment of the temporal coevalness of the target culture, there were a different set of objections to expanding and making socio-historical use of aesthetic conceptions of totality as one found in Gestalt theory, and some of these objections were articulated in the language of ideology critique: As Steven Helmling highlights, Theodor Adorno thought that Gestalt and the totalities it championed functioned as “unconsciously synthetic” pacifications, “thus effecting (false, familiarized, familiarizing) reconciliations or integrations of experiential fragmentariness” in social life (Helmling 2003). For Adorno, inasmuch as it served to cover over contradictions with spurious effects of reconciliation, Gestalt became a formal proxy for political conformism: “Gestalt is an instance or model, indeed an epitome, of ideology as such: reflex and reinforcer of the habitual familiarizations, the ideological conditionings, the false reconciliations or ‘imaginary solutions to real contradictions’ of the historically and culturally given” (Helmling 2003). In an extension of such a negative assessment, later Reinhold Martin would take a similar line against discourses of organized unity that served as blueprints for artistic design and figures of environmental coherence, referring to “the experience of the ahistorical, a-contextual subject of Gestalt psychology” (Martin 2004, p. 84).

This part of the story of Gestalt and what it contributed to the later career of patterns and pattern recognition concerns formalism at large. In Adorno’s critique of Gestalt, we have an instance of an anti-formalism that is inspired by Marxist reflection theory, a homological model of the relation between abstraction and social reality: a false integration or reconciliation in social reality--as concealment of hard economic contradictions and inequalities-- makes the structural integration and effects of unity characteristic of Gestalt perception suspect; the common operation of synthesis works to bind the social and formal/perceptual homologues to each other. As relatively recent works in literary criticism like Caroline Levine’s argue, homologically established forms of ideology-critique directed toward totalities are at best blunt tools, and abandoning the agency or “affordance” of forms is not a necessary condition of doing justice to historical realities. One can even “introduce more wholes” to check the power of “harmfully totalizing and unifying wholes” (Levine 2017). This does not mean that it is possible or desirable to bring back a kind of relativist anthropological analysis that deals with cultural wholes as somewhat aesthetically conceived unities in the vein of cultural anthropology; steering a way between a renewed formalism and the pitfalls identified by critics like Fabian and Adorno might be possible.

1960: The rise of pattern recognition

Starting from the 1950s and advancing into the 1960s, we can observe the often conflicting paradigms of Gestalt psychology and cybernetics coexisting within the same cultural context, resulting in a linguistic overlap that signifies a transition. This transition involves a gradual decline in the influence of Gestalt psychology and a shift towards pattern recognition defined in terms of information processing. The shift ranges from growing optimism in powers of computation to the arrival of cognitive psychology with its information-processing vision of the mind. According to Steve Heims’s pithy diagnosis of this moment, “Gestalten go to bits” (Heims 1991, p. 201). While this transition was acknowledged, it was not universally embraced.

One of the best illustrations for the coexistence and reinscription involved is an article by Gregory Bateson from 1951, written from the high point of the Macy conferences, on the heels of the appearance of Wiener’s The Human Use of Human Beings. The article entitled “Information and Codification: A Philosophical Approach” has a good claim to being one of the missing links between the contrasting approaches to organization constituted by cybernetics and Gestalt theory. It also represents a more balanced assessment of the relationship between mechanism and Gestalt qualities than Wiener’s translation ploy in Cybernetics. The context is Bateson’s attempt to introduce a tripartite classification of information-coding: digital, analogic, and Gestalten.

Bateson assigns the first category of digital codification to counting mechanisms that work on a discrete basis, mentioning calculating machines made up of cogs. The second form of codification, the analogic, is concerned with the capability for producing iconic, indexical, or functional equivalence: this time, Bateson takes the example of the model of an environmental interaction constituted by a wind tunnel, where the two sets of relationships in the real world and that in the model are mapped onto each other functionally and semiotically without discrete counting. The third form is a hybrid or intermediary between the first two, defined by an identification of analog forms by digitally based processes. The newly invented optical character recognition machines provide Bateson’s key example: “there are a few machines which are capable of codifying information in units comparable to what the psychologists call Gestalten” (Bateson 1951, p. 172). Although it works through discrete bits, this kind of machine, Bateson writes, “is doing something very closely comparable to that recognition of Gestalten whereby a human being knows that a square is a square even though it may be of almost any size and presented at any angle”--an example directly responding to Wiener’s Gestalt and universals discussion. As we will see shortly, Bateson’s phrase “something very closely comparable” is perhaps more carefully worded than others made at the time. Bateson combines this illustration with what seems to be a common trope of the mechanization trend of its time and the Macy milieu, that of human perception as a scanning apparatus: “when the eye scans an object, the shape of the object is certainly transformed into a temporal sequence of impulses in the optic nerve” (p. 170).Footnote 14 Despite the way Gestalten provide analogue underpinnings to scanning, in the same instance, the kind of irreducible wholes they denote find themselves incorporated into a new artificial medium that eclipses their explanatory power.Footnote 15 The issue of the third type of codification raised in Bateson’s discussion would go on to be a major point of tension between the early machine learning approaches and Gestalt theory, as in the work of a Gestalt theorist like Rudolf Arnheim, whose Visual Thinking (1969) insisted on the fundamental difference between “the spontaneous grasp of pattern” proper to humans and pattern recognition by machine (Arnheim p. 43).

In the 1950s, which Herbert Schantz (1982) calls the “OCR’s fabulous fifties,” another significant milestone was reached in pattern recognition research.Footnote 16 This was a famous paper (1959) on frogs’ visual systems, authored by a group that included McCulloch, Pitts, Jerome Lettvin, and Humberto Maturana. By modeling the visual system of frogs, this paper extended McCulloch and Pitts’ exploration of geometric universals, which had been significant for Wiener. The paper was remarkable for its revisiting of territory associated with Gestalt theory. It postulated the existence of elements--certainly, not the “fields” of Gestalt-- within frogs’ visual systems that respond to variables such as edges, contrast, movement, and illumination. It referred to the modulation of the figure-background relationship in various contexts, with its central focus being on the problem of perception. Additionally, the authors acknowledged their debt to a notable figure in machine learning and pattern recognition, Oliver Selfridge, and his early “experiments with mechanical recognizers of pattern” (Lettvin et al. 1959).

Bringing these developments into a more popular context, Oliver Selfridge and Ulric Neisser’s paper (1960) titled “Pattern Recognition by Machine” appeared in the widely circulated Scientific American, offering a computationally charged perspective on the subject. While Neisser was among the early advocates of cognitive psychology, as noted, Selfridge was a pioneer of machine learning. Their article inquired into the possibility of computerized perception, providing a broad definition of pattern recognition as an intellectual skill that encompasses cognition, abstraction, and perception: “Understanding speech and reading print are examples of a basic intellectual skill that can variously be called cognition, abstraction, or perception; perhaps the best general term for it is pattern recognition” (Selfridge and Neisser 1960). This definition established pattern recognition as a human intellectual skill that could potentially be reimagined and computationally explained within a mechanical framework. The authors acknowledged that machines, at that point, lacked perception of patterns but believed that achieving pattern recognition machines would be pivotal for further integration of machine learning developments: “We suspect that until programs to perceive patterns can be developed, achievements in mechanical problem solving will remain isolated technical triumphs” (Selfridge and Neisser 1960).

Recent research by scholars like Aaron Mendon-Plasek reveals that early pattern recognition work was invested in “mechanized significance,” bracketing the distinction between human sense-making and machines, in the very process of replacing human form recognition by machinic kinds in key branches of industry. The bracketing is evident in an earlier paper by Selfridge (1955): “The whole process of Pattern Recognition is inevitably tied up with ways of determining significance. I suggest—this is my own fancy—that this is the distinction usually made between machines and men. That men can learn by experience to extract and deal with the significant things and machines cannot… I do not, however, believe it is a valid distinction” (Selfridge 1955, p. 92, as cited in Plasek 2020, p. 62). The statement is made in the context of early research—using military funding—that aims to get computers to recognize determinate shapes and letters through probabilistic reasoning on random sequences; a context where Selfridge is still trying to ensure the quality of the “isolated technical triumphs” in their own right. In his words, this is the attempt to get computers “to hunt for good sequences” (Selfridge 1955, p. 93). In establishing a statistical criterion on the basis of which he questions the exclusive attribution of “significance” to humans, Selfridge’s early intervention and his emphasis on experience in particular, aligns with Wiener’s project in Cybernetics’ “Gestalt and Universals” chapter, his predecessor’s attempt to mechanistically ground empiricist procedures like association and classification. To these, Selfridge adds the fundamental Gestalt quality of the figure and ground organization: “Pattern recognition is the extraction of the significant features from a background of irrelevant detail” (Selfridge 1955, p. 91).Footnote 17

As Plasek notes, Selfridge’s fundamental insight finds expression in the question: “If humans could learn significance (and indeed, everything else that they can learn) solely through experiencing the world…why couldn’t machines learn an image recognition task solely from labeled images?” (Mendon-Plasek 38). However, everything depends on the criteria by which key categories such as learning, “significance” and context are defined. Whether context is determined by the data provided or includes the perspective of the researcher is a crucial consideration not treated in Selfridge’s early account; similarly, to come up with a unitary concept of significance for the machine and the human, it would have been necessary to address how differences in the learning processes between humans and machines may inform the concept of significance itself.

The value here is perhaps more in the challenge of the statements than in the resolution of these problems. A charitable interpretation of Selfridge’s wager would emphasize the liberating novelty of a framing that blurs the ontological separation between human experience and machine learning, presenting compelling pragmatic evidence in favor of this perspective. On the other hand, it is also possible to argue that Selfridge’s position aligns with the broader translation tendencies observed in cybernetics, within the larger context of the ongoing tension between Gestalt theory and the realm of “bits.”

Selfridge had legitimate interests in the nature of learning and the mind, but in the succeeding decades after his groundbreaking work, pattern recognition research was oriented toward applications related to “medical diagnosis, industrial inspection, personal identification and man-machine interaction” (Duin and Pekalska 2005) in addition to the longstanding interest in character recognition technologies. Furthermore, some of the most influential adopters of Selfridge’s work were scientists working on operations research like Herbert Simon and Alan Newell (1958), automating problem solving and decision making, and extending machine learning heuristics toward industry uses. The drive to build expert systems sometimes resulted in the temptation to overstate their capabilities. Such a direction was correlated with a search for automation and the replacement of human functions in various domains of labor, the equation of robustness with statistical procedures, and what we may call a tendency to abstract from the “whole circuit,” and work with relatively narrow definitions of context. An epistemology in the image of cost saving, resource allocation uses, and time sensitivity of an economic sort often becomes prevalent in many examples of pattern recognition research during the 70 s and 80 s.Footnote 18 The dominance of such tendencies seem to overshadow the broader and speculatively rich aspirations that underlie Selfridge’s mechanized concept of significance.

As mentioned earlier, writers like Arnheim took serious exception to the generalization of pattern recognition on the mechanical model to the working of the human perception and the mind, appealing to the concept of fields and spontaneity in the very moment of their seeming eclipsing (Arnheim 1969). However, there were more receptive players in the cultural arena, as well. The media theorist Marshall McLuhan’s use of “pattern recognition” in Understanding Media (1964) indicates a more liberal and receptive uptake with wide-ranging cultural impact. McLuhan takes on board the Selfridge and Neisser definition of the term and specifically seizes on the association they make between the heuristic capacity of the human mind and pattern recognition: “a man is continuously exposed to a welter of data from his senses, and abstracts from it the patterns relevant to his activity at the moment. His ability to solve problems, prove theorems and generally run his life depends on this type of perception” (Selfridge and Neisser 1960). Extending this logic, McLuhan offers, “now, however, in the electronic age, data classification yields to pattern recognition, the key phrase at IBM. When data move instantly, classification is too fragmentary. In order to cope with data at electric speed in typical situations of ‘information overload,’ men resort to the study of configurations, like the sailor in Edgar Allan Poe’s Maelstrom” (McLuhan 1964).

In this different context, perception is still a human function, whatever the changes it undergoes in step with developments in media technologies and the built environment. At the same time, it is as if McLuhan is expanding on what is implied in the Selfridge and Neisser title “pattern recognition by machine,” and presenting how the unmarked human one must work in the specific conditions of modern information overload. His reference to IBM may indicate an awareness of the company not just as a technology firm but also as a sponsor of research on artificial intelligence, in papers like the one by Newell, Shaw, and Simon on Chess-Playing Programs, which appeared in IBM Journal in 1958. Along the way, it is the machine and its forms of data manipulation that move into the unmarked position of first degree connection with something called “pattern recognition.” McLuhan’s is a pattern recognition by the human, which no longer goes without saying, a little like Herbert Simon’s later work on “human problem solving” (1970).

Another point of note is that something of the Gestalt version of perception is retained in McLuhan’s reference to pattern as not “too fragmentary.” Perhaps the reason why McLuhan’s appropriation of pattern recognition has proved attractive to many, is that it takes the speed and processing capability of the machine, and combines it with the instantaneous sense for fields and wholes Gestalt theory associates with the human mind; two types of cognition that have all the appearance of being antagonists have been joined in the service of taming information overload.

The terminological application of pattern recognition to both machines and humans does not correspond to a factual state of equivalence: it is first part of the project of researchers working on machine learning and justified by the anticipatory horizon of perceiving machines; there is a lot that is “aspirational,” as Plasek puts it. But when media theorists like McLuhan pick up on it, with due reference to IBM, there is another layer where the machinic model grounds the understanding of the human in terms of certain select features (speed in a given media environment). This move of equivalence may elide important differences between the quantitative and the qualitative, and pass over the gaps between different understandings of learning and experience, but it also establishes a continuity around the detour of computation: it becomes possible (again?) to think that humans have pattern recognition capacities without this meaning they process discrete information bits. It is this kind of oblique continuity --from patterns before machinic pattern recognition to patterns after-- that would inform the use a cultural anthropologist like Margaret Mead would make of “pattern recognition,” which, in her vocabulary, takes on the sense of a cultural diagnosis enabled by nonquantitative and aesthetic means closer to intuition (Mead 1969, p. 19).

The inclusion of intuition in this picture follows a logic similar to the tensions created by the invention of photography in the arts concerned with pictorial representation, creating incentives to explore those aspects of perception not easily captured by mechanical techniques—the natural sciences of the time are not alien to this new conceptual direction and, in a new vision of objectivity, the scientist admits to being a “scanner” better than the mechanical sort: “More important than [my] negative reaction to the versatile pattern recognition abilities of digital computers is my strong positive feeling that human beings have remarkable inherent scanning abilities. I believe these abilities should be used because they are better than anything that can be built into a computer” (the physicist Luis Alvarez cited in Daston and Galison, 2007: 330).

Conclusion

In this article, I have traced the evolution of the concept of pattern from its early association with Gestalt theory and an emphasis on good forms to its role as a master metaphor for perception and media reception. Influenced by cybernetics and information theory, patterns came into contact with engineering fields like character recognition and gave impetus to machine learning. At the heart of this development lies the confrontation between Gestalt and cybernetic paradigms, advocating respectively for wholes and systems defined by either inner formal unity or dynamic feedback.

Rather than presenting a neat linear succession between the two paradigms, I have highlighted a territorial skirmish where cybernetics and associated cognitivist discourses repeatedly attempted to mechanistically explain key Gestalt phenomena and substitute digital for analog coding, following Gregory Bateson’s terms. However, in the outcome of this account of rivalry, qualitative models do not fully translate into quantitative ones. “Mechanized significance,” as Mendon-Plasek aptly puts it, continues to exert a strong pull toward delegating inquiries to “data in data out” operations. Nonetheless, evidence suggests that traces of nonquantitative intuition persist, even in coding operations, electronic media reception, and data analysis in social science. The exorcism of Gestalt theory from formalizing significant wholes appears incomplete, with elements like the figure and ground model serving as unacknowledged cognitive assets that find reassembly in computational language.

Both paradigms had encounters with and cautiously claimed to offer tools for social science research. I have dedicated space to unpacking the legacy of Gestalt theory in anthropology and highlighting the influence of cybernetics on the play of distance and proximity in social sciences. By referring to figures like Hall and Mead’s reservations about entrusting all meaningful historical and social sense-making to data, the article has also come around to the importance of “partial insight.” Fetishizing ever-expanding datasets does not guarantee success in sense-making within historical contexts. The inevitability of partial insight and perspectival determination, reminiscent of Donna Haraway’s advocacy of situated knowledge, challenges the construal of pattern recognition within social and historical realms as a straightforwardly computational affair.

The scientific ideology of a “God’s eye view” or a vision from nowhere, as Haraway problematized, has transformed into a new quandary: relinquishing scientific discovery to data-intensive or imaging operations and machine learning, claiming to transcend any partial perspective through informational redundancies, if not brute force. In this context, Haraway’s challenge that “vision is better from below the space platforms of the powerful” may urge us to contemplate what a pattern recognition from below would entail (Haraway 2004). In a climate where platform-driven data extraction, user profiling, and quantification of work and leisure are pervasive uses for machine learning and computational pattern recognition, the unmarked human capacities and occluded labor that inform mechanized judgment should receive stronger emphasis and command awareness to inform a collective reappropriation of data against alienating dispossession.

In a more speculative vein, if pattern recognition is not to be simply written off as an emblem of “the control vision of the world” (Franklin 2015), then it becomes imperative to locate or discover a version of it that is less beholden to instrumental platform logics, a version that does not disavow its truck with partiality, intuition and the qualitative. Such a model would certainly not be the dismal version of pattern recognition circulating through conspiracy theories, as it would have to involve a rigorous cultivation. Similarly, this does not entail a return to the purity of no data or a flight to the irrational. By its very development, the concept is open to a repurposing whereby it could bridge the abstract and the concrete, and the inferential with the intuitive.

The convergence between cybernetics and Gestalt theory that has profoundly shaped pattern recognition and given it an ambiguous status between humans and machines is an undeniable part of our present heritage. Yet, against the unimaginatively circular systemic normativity of new machines for more data, it might be possible to define a practice and skill whose real criterion is that it pay for the discovery of patterns with effort in pursuit of insight, recouping a Gestalt emphasis on the reorganizability of seemingly fixed structures.

As the capabilities of machine learning increase, so do those of machinic pattern recognition, bespeaking the need for the nuances of partial insight, contextual understanding, and situated knowledge. To pave the way for a deeper understanding of patterns in our complex world, pattern recognition today has to be reinscribed with the understanding that certain investigative tools and design elements may “sometimes be the means by which power both operates and can be confronted” (Fuller and Weizman). What is important is to institute new uses for advanced data operations that do not follow the presuppositions of extraction, and which may in turn, for instance, shape new types of design and interface with data structures. Since it connects human sciences, including humanities, and engineering perspectives with each other by definition, a rearticulation of pattern recognition has both critical and creative possibilities. At the very least, by owning up to the constitutive obscurities and limits of its vision, a pattern recognition from below would have a better claim to being a true empiricism, doing justice to the radical fragility and contingency of real experience.