“Art is clearly an expression of our aesthetic response to beauty,” wrote neuroscientist Vilayanur Ramachandran in the October 2006 issue of Scientific American Mind, before noting that there is no consensus in the worlds of art about the definition of 'art', 'aesthetic' or 'beauty'.

The lack of accepted definitions for such key terms has, however, proved to be no deterrent to their investigation by scientists. It seems that, released into an arena of analysis in which the rules of science are perceived as not operating, these researchers discard the rigour they would normally observe.

Yet our responses to artworks offer much for neuroscience to investigate, without resorting to such generalizations. Art's diversification has been matched by our sophistication in the perception of it. Happily, a few scientists and art historians are beginning to direct their efforts to the more pragmatic questions of how we perceive and attach significance to forms.

Mark Rothko's abstract art created new neurological responses. Credit: © 1998 KATE ROTHKO PRIZEL & CHRISTOPHER ROTHKO ARS, NY & DACS, LONDON/AKG-IMAGES

Studying the arts is not a science, but the field has standards in the taking and use of evidence. The process of hypothesis formulation and the quest for evidence, evaluation and analysis of sources, matching and reformulation have their own kind of discipline. As an art historian, I dislike one hypothesis sitting on the shoulders of another unproven one as much as any scientist.

Most art historians would not regard 'art', 'aesthetic' and 'beauty' as absolute terms that are the goal of their enquiries or even major guiding principles. Many, like myself, deny their use as absolutes at all. At best they serve as subjects of period study. Instead, in the past 30 years or so, the discipline of art history has increasingly focused on contexts — on the complex conditions under which artefacts were generated and have functioned in particular societies.

However, several prominent neuroscientists continue to address the absolutes, using the formal criteria of outdated Modernist theory to define problems in the study of art with little concern for complex content. It was this striking disjunction that erupted in a wonderfully bloody seminar in 2002 at the Getty Research Institute in Los Angeles, California, at which Ramachandran and fellow neuroaesthetics pioneer Semir Zeki were confronted by the resident and visiting art historians.

These art historians were mainly 'social deconstructivists' who dissected the social and political factors behind artworks' creation and public reception. They were concerned with the quality of communication and visual potency of the artworks, not with defining their beauty. What resulted was a discourteous dialogue of the deaf — and the two distinguished scientists did not stay to the end. I had a good deal of sympathy with them.

Neuroscience fares a little better with philosophical theory. However, in the tradition of Immanuel Kant, the certainty that art exists in a definable aesthetic realm that serves no purpose beyond itself is no longer sustainable.

Writers on art have accumulated terms to describe the characteristics of things that are considered beautiful or aesthetically pleasing: they may be uplifting, exalting, transcendent, delightful, graceful, poetic, harmonious, expressive and so on. We can describe something as beautiful to someone who shares our cultural instincts, but we cannot define beauty itself.

Art history is about the conscious creation of works by artists for viewers who make selective demands on what they are seeing.

The 'fuzzy group' concept offers a way out of this dilemma. Things we call beautiful share family traits. Although they have qualities in common, no individual characteristic is absolutely necessary to define their resemblance. The overarching descriptor of 'beauty' has no validity beyond this loose level of family likeness. This explains how we can categorize as 'art' both an installation of felt and fat by Joseph Beuys and an altarpiece of the Madonna and saints by Botticelli.

Neuroscientific data fit better with this interpretation of aesthetics as dealing with associations of overlapping groups than with the quest to define what the brain finds eternally beautiful. Zeki and others' demonstrations that formal arrangements of shapes activate different brain regions from figurative images help explain why, in the early twentieth century, many did not recognize abstract art as 'art'. The sight of a Rothko painting, with its abstract patches of colour, generated an unfamiliar neurological response.

Blurred boundaries

With each move of the avant-garde beyond the established boundaries of art, these groups of related art types are rendered even fuzzier and more ill-defined to the point that the boundaries disappear altogether. Although still referred to as art, there is no reason why the painting of a traditional portrait and the genetic engineering of a fluorescent rabbit, such as by artist Eduardo Kac, should be classified as the same kind of activity. It would be interesting to investigate to what extent comparable brain activity is triggered by such divergent creations; I suspect that there would be striking differences.

If not in the quest for beauty, where might joint research in science and art history flow? The most tractable areas are in viewing and reception, bearing on the messy business of selective looking and slanted cognition, which determine that we notice some things and filter out others. Art history is about the conscious creation of works by artists for viewers who make selective demands on what they are seeing. The investigation of the ways in which we view different artworks may also eventually bear on the big questions of 'art' and 'beauty', if only to dissolve them.

Some neuroscientists have begun to look at the psychological complexities of how we view and value artwork. Karl Friston, of the Wellcome Trust Centre for Neuroimaging at University College London, and his colleagues have proposed a method of 'psychophysiological interaction' to assess the effect that psychological factors have on physical processes in the brain through analysis of scans. This procedure has the potential to show how complex and interactive the viewing of artworks is, both between individuals and for an individual under different conditions. Another promising study, addressing how brand awareness affects the value we place on something, was that by psychologist Samuel McClure and his colleagues, then at Baylor College of Medicine in Houston, Texas, who reported on the neurology of preferences for the two main cola drinks (S. McClure et al. Neuron 44, 379–387; 2004).

A glimpse of further possibilities is provided by an exploratory study, yet to be published, which was undertaken this year by Mengfei Huang and Andrew Parker at the University of Oxford, UK, with my assistance. The research tested people's responses to authentic Rembrandt portraits and works by other artists that are close in appearance to the master's portraits. It is widely acknowledged in the art world that once a work is revealed as 'not right', it discernibly looks different to a viewer. Features that were previously overlooked suddenly become glaringly obvious. It is as if a former lover is no longer blinded by love.

Rembrandt was chosen because of the large number of portraits he painted, and the even larger number of portraits produced in his style, ranging from works by close followers to downright fakes. In our experiment, viewers were presented with a brief statement explaining Rembrandt's importance, his style development, the large number of imitations and the huge implications, not least financial, of a painting's authenticity.

Protocols were devised so that two sets of viewers were presented with the same group of pictures, a mix of Rembrandts and non-Rembrandts, but with each group labelled differently. The labelling did not correspond to the actual status of each painting and was reversed for each set of participants. The main purpose was to detect differential brain responses to the flagging of authenticity, but we were also able to detect if the portraits that were authentic triggered a different reaction. The results suggest some subtle interactions between neural activity, expectation, memory and value systems.

Viewers respond differently to authentic artworks and copies, such as this pair of portraits of Rembrandt, one painted by the master and the other by a follower 1 .Footnote

1Rembrandt's self-portrait is on the left.

Credit: BOTH IMAGES: BRIDGEMAN ART LIBRARY

This kind of investigation is only a beginning. It would be useful to extend the study to bodies of expert viewers. By expert, I do not just mean art historians who know their Rembrandts from their Flincks. Rather, I mean people who have different types of engagement with the faces of the historic people depicted, from psychologists to costume historians, from picture restorers to cartoonists. The problems of method here are legion, but protocols can be devised to overcome them.

Let us look beyond 'art', 'beauty' and 'aesthetics' and engage with concrete problems that tell us about varied modes of viewing. Art historians and scientists need to work together to define new questions that are both tractable and of genuinely shared interest.