Few animals provide a readout that is as objective of their perceptual state as camouflaging cephalopods. Their skin display system includes an extensive array of pigment cells (chromatophores), each expandable by radial muscles controlled by motor neurons. If one could track the individual expansion states of the chromatophores, one would obtain a quantitative description—and potentially even a neural description by proxy—of the perceptual state of the animal in real time. Here we present the use of computational and analytical methods to achieve this in behaving animals, quantifying the states of tens of thousands of chromatophores at sixty frames per second, at single-cell resolution, and over weeks. We infer a statistical hierarchy of motor control, reveal an underlying low-dimensional structure to pattern dynamics and uncover rules that govern the development of skin patterns. This approach provides an objective description of complex perceptual behaviour, and a powerful means to uncover the organizational principles that underlie the function, dynamics and morphogenesis of neural systems.
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Messenger, J. B. Cephalopod chromatophores: neurobiology and natural history. Biol. Rev. Camb. Philos. Soc. 76, 473–528 (2001).
Beck, J. & Gibson, J. J. The relation of apparent shape to apparent slant in the perception of objects. J. Exp. Psychol. 50, 125–133 (1955).
Julesz, B. Visual pattern discrimination. IRE Trans. Inf. Theory 8, 84–92 (1962).
Portilla, J. & Simoncelli, E. P. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vis. 40, 49–70 (2000).
Gatys, L. A., Ecker, A. S. & Bethge, M. A neural algorithm of artistic style. Preprint at https://arxiv.org/abs/1508.06576v2 (2015).
Ghiasi, G., Lee, H., Kudlur, M., Dumoulin, V. & Shlens, J. Exploring the structure of a real-time, arbitrary neural artistic stylization network. Preprint at https://arxiv.org/abs/1705.06830v2 (2017).
Cloney, R. A. & Florey, E. Ultrastructure of cephalopod chromatophore organs. Z. Zellforsch. Mikrosk. Anat. 89, 250–280 (1968).
Florey, E. & Kriebel, M. E. Electrical and mechanical responses of chromatophore muscle fibers of the squid, Loligo opalescens, to nerve stimulation and drugs. Z. Vgl. Physiol. 65, 98–130 (1969).
Hanlon, R. T. & Messenger, J. B. Adaptive coloration in young cuttlefish (Sepia officinalis L.): the morphology and development of body patterns and their relation to behaviour. Phil. Trans. R. Soc. B 320, 437–487 (1988).
Kelman, E. J., Osorio, D. & Baddeley, R. J. A review of cuttlefish camouflage and object recognition and evidence for depth perception. J. Exp. Biol. 211, 1757–1763 (2008).
Gonzalez-Bellido, P. T., Scaros, A. T., Hanlon, R. T. & Wardill, T. J. Neural control of dynamic 3-dimensional skin papillae for cuttlefish camouflage. iScience 1, 24–34 (2018).
Dubas, F., Hanlon, R. T., Ferguson, G. P. & Pinsker, H. M. Localization and stimulation of chromatophore motoneurones in the brain of the squid, Lolliguncula brevis. J. Exp. Biol. 121, 1–25 (1986).
Florey, E., Dubas, F. & Hanlon, R. T. Evidence for l-glutamate as a transmitter substance of motoneurons innervating squid chromatophore muscles. Comp. Biochem. Physiol. C 82, 259–268 (1985).
Reed, C. M. The ultrastructure and innervation of muscles controlling chromatophore expansion in the squid, Loligo vulgaris. Cell Tissue Res. 282, 503–512 (1995).
Liddell, E. G. T. & Sherrington, C. S. Recruitment and some other features of reflex inhibition. Proc. R. Soc. Lond. B 97, 488–518 (1925).
Ho, T. K. Random decision forests. In Proc. 3rd International Conference on Document Analysis and Recognition 278–282 (IEEE Computer Society, 1995).
Lucas, B. D. & Kanade, T. An iterative image registration technique with an application to stereo vision. In Proc. 7th International Joint Conference on Artificial Intelligence 674–679 (Morgan Kaufmann, 1981).
Mäthger, L. M., Chiao, C. C., Barbosa, A. & Hanlon, R. T. Color matching on natural substrates in cuttlefish, Sepia officinalis. J. Comp. Physiol. A 194, 577–585 (2008).
Bassaglia, Y. et al. Sepia officinalis: a new biological model for eco-evo-devo studies. J. Exp. Mar. Biol. Ecol. 447, 4–13 (2013).
Fioroni, P. Die embryonale Genese der Chromatophoren bei Octopus vulgaris Lam. Acta Anat. 75, 199–224 (1970).
Packard, A. Morphogenesis of chromatophore patterns in cephalopods: are morphological and physiological ‘units’ the same? Malacologia 23, 193–201 (1982).
Deravi, L. F. et al. The structure–function relationships of a natural nanoscale photonic device in cuttlefish chromatophores. J. R. Soc. Interface 11, 20130942 (2014).
Dubas, F. Innervation of chromatophore muscle fibers in the octopus Eledone cirrhosa. Cell Tissue Res. 248, 675–682 (1987).
Ferguson, G. P., Martini, F. M. & Pinsker, H. M. Chromatophore motor fields in the squid, Lolliguncula brevis. J. Exp. Biol. 134, 281–295 (1988).
Messenger, J., Cornwell, C. & Reed, C. l-glutamate and serotonin are endogenous in squid chromatophore nerves. J. Exp. Biol. 200, 3043–3054 (1997).
Maynard, D. M. in Invertebrate Nervous Systems: Their Significance for Mammalian Neurophysiology (ed. Wiersma, C. A. G.) 231–255 (Univ. Chicago Press, Chicago, 1967).
Dubas, F. & Boyle, P. R. Chromatophore motor units in Eledone cirrhosa (Cephalopoda: Octopoda). J. Exp. Biol. 117, 415–431 (1985).
Florey, E. Nervous control and spontaneous activity of the chromatophores of a cephalopod, Loligo opalescens. Comp. Biochem. Physiol. 18, 305–324 (1966).
Bell, A. J. & Sejnowski, T. J. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 (1995).
Hyvärinen, A. & Oja, E. Independent component analysis: algorithms and applications. Neural Netw. 13, 411–430 (2000).
Boycott, B. B. The functional organization of the brain of the cuttlefish Sepia officinalis. Proc. R. Soc. Lond. B 153, 503–534 (1961).
Andouche, A. & Bassaglia, Y. Coleoid cephalopod color patterns: adult skin structures and their emergence during development in Sepia officinalis. Vie Milieu 66, 43–55 (2016).
Packard, A. Size and distribution of chromatophores during post-embryonic development in cephalopods. Vie Milieu 35, 285–298 (1985).
Yacob, J. et al. Principles underlying chromatophore addition during maturation in the European cuttlefish, Sepia officinalis. J. Exp. Biol. 214, 3423–3432 (2011).
Packard, A. & Hochberg, F. G. Skin patterning in Octopus and other genera. Symp. Zool. Soc. Lond. 38, 191–231 (1977).
Hanlon, R. T. The functional organization of chromatophores and iridescent cells in the body patterning of Loligo plei (Cephalopoda: Myopsida). Malacologia 23, 89–119 (1982).
Gomez-Marin, A., Paton, J. J., Kampff, A. R., Costa, R. M. & Mainen, Z. F. Big behavioral data: psychology, ethology and the foundations of neuroscience. Nat. Neurosci. 17, 1455–1462 (2014).
Yamaguchi, M., Yoshimoto, E. & Kondo, S. Pattern regulation in the stripe of zebrafish suggests an underlying dynamic and autonomous mechanism. Proc. Natl Acad. Sci. USA 104, 4790–4793 (2007).
Cheng, C. W. et al. Predicting the spatiotemporal dynamics of hair follicle patterns in the developing mouse. Proc. Natl Acad. Sci. USA 111, 2596–2601 (2014).
Manukyan, L., Montandon, S. A., Fofonjka, A., Smirnov, S. & Milinkovitch, M. C. A living mesoscopic cellular automaton made of skin scales. Nature 544, 173–179 (2017).
Kröger, B., Vinther, J. & Fuchs, D. Cephalopod origin and evolution: a congruent picture emerging from fossils, development and molecules: extant cephalopods are younger than previously realised and were under major selection to become agile, shell-less predators. BioEssays 33, 602–613 (2011).
Butler-Struben, H. M., Brophy, S. M., Johnson, N. A. & Crook, R. J. In vivo recording of neural and behavioral correlates of anesthesia induction, reversal, and euthanasia in cephalopod molluscs. Front. Physiol. 9, 109 (2018).
Bergstra, J., Yamins, D. & Cox, D. D. Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In Proc. 30th International Conference on Machine Learning Vol. 28 I-115–I-123 (Journal of Machine Learning Research, 2013).
Schaefer, S., McPhail, T. & Warren, J. Image deformation using moving least squares. ACM Trans. Graph. 25, 533–540 (2006).
Fischler, M. A. & Bolles, R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981).
Meyer, F. Topographic distance and watershed lines. Signal Process. 38, 113–125 (1994).
Yoo, A. B., Jette, M. A. & Grondona, M. in Job Scheduling Strategies for Parallel Processing Vol. 2862 (eds Feitelson, D. et al.) 44–60 (Springer, Berlin, 2003).
Message Passing Interface Forum. MPI: a message-passing interface standard. version 3.1 http://mpi-forum.org/mpi-31/ (2015).
Jones, E. et al. SciPy: open source scientific tools for Python. http://www.scipy.org/ (2001).
Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Walt, S. et al. scikit-image: image processing in Python. PeerJ 2, e453 (2014).
We thank F. Bayer, A. Umminger and N. Heller for assistance in building the experimental setup; L. Jürgens, T. Klappich, J. Nenninger, E. Northrup and G. Wexel for animal care; R. Siegel for providing the squid skin image; E. Lamo Peitz, S. Trägenap and J. Racky for help with image alignment and segmentation; R. Hanlon and members of the Marine Biology Laboratory in Woods Hole for their hospitality to G.L. in the summer of 2016; Bruker Daltonics, J. Fuchser and C. Henkel for Fourier-transform ion cyclotron resonance–mass spectometry measurement time and discussions; M. Tosches and L. Fenk for comments on the manuscript; and the members of the Laurent laboratory for discussions. Funded by the Max Planck Society (G.L.), the European Research Council (G.L.) and the Bernstein Focus: Neurotechnology Frankfurt (M.K.).
Nature thanks C. Machens, D. Osorio and the other anonymous reviewer(s) for their contribution to the peer review of this work.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
a, Test patch of skin used for classifier testing. b, Segmentation by expert human. c, Segmentation by classification algorithm. d, Composite image comparing manual (annotation) and automatic (prediction) segmentation. There was agreement for 87% of pixels, with differences mostly on the edges surrounding chromatophores. e, Quantification of region overlap. Regions defined from watershedding the composite image shown in d. Correct detection: regions labelled by both methods. False detections: regions identified by automatic but not by manual segmentation. Failed detections: regions identified by manual but not automatic segmentation. f, For all regions, annotated versus predicted size. Line, identity.
Extended Data Fig. 2 Sensitivity of the correlation between skin patches to small image translations and rotations.
Left, skin patches from the two sets of matching master frames shown in Fig. 1e. Middle and right, composite images of the corresponding master frames (master frame 1 in green, master frame 2 in magenta, overlap in white). Small translations or rotations quickly lower the cross-correlation, as in the schematic in Fig. 1e, bottom.
a, Chromatophores before (left) and after (right) local application of 40 μM glutamate. b, Transmission spectra of a population of chromatophores before (circles) and after (squares) glutamate application, projected onto two dimensions defined by human L and S cone action spectra (n = 63 chromatophores). c, Direct infusion electrospray ionization Fourier-transform ion cyclotron resonance (ESI-FT-ICR) mass spectrum of the skin tissue extract showing high spectral complexity. d, HPLC–UV–MS chromatograms of skin tissue extract showing two main peaks with correlating ultraviolet-light (250 nm) absorption (blue) and mass spectrometry intensity (grey) consisting of eluting compounds with m/z 380.09 and m/z 424.08 (extracted ion chromatogram (EIC) traces, green). Experiments were replicated five times with similar results. e, Direct infusion ESI-FT-ICR mass spectra of skin tissue extract showing an overlay of the isolated precursor spectrum for decarboxylated xanthommatin (green, m/z 380.0876, theoretical: m/z 380.0877) and the fragment spectrum (red). Main fragments were assigned to putative structural losses of A (-NH3), B (-H2O, -NH3), C (-NH3, -HCOOH), D (-C2H3NO2) by accurate mass. f, Direct infusion ESI-FT-ICR mass spectra of skin tissue extract showing an overlay of the isolated precursor spectrum for xanthommatin (green, m/z 424.0774, theoretical: m/z 424.0775) and the fragment spectrum (red). Main fragments were assigned to putative structural losses of A (-NH3), B (-H2O, -NH3), C (-NH3, -HCOOH), D (-C2H3NO2) by accurate mass. g, Intensity distributions in laser desorption ionization Fourier-transform ion cyclotron resonance mass spectrometry (LDI-FT-ICR-MS) imaging and structures for putative xanthommatin derivatives (merged [M + H]+; [M + Na]+): decarboxylated, oxidized (m/z 380.0886; 402.0696), decarboxylated, reduced (m/z 382.1037; 404.0853), oxidized (m/z 424.0785; 446.0629) and reduced (m/z 426.0938; 448.0761). h, Intensity distributions of main xanthommatin and derivative fragments, corresponding to molecular species detected in ESI-FT-ICR fragmentation measurements. Experiments were performed on 12 tissue slices, producing similar results. i, Image of cryotome section of fresh-frozen Sepia skin showing chromatophores. j, Spatial segmentation map of section in i, showing distinct clusters for light and dark chromatophores (orange versus black colours) and surrounding tissue (grey). k, Intensity distributions for xanthommatin derivatives (merged [M + H]+ and [M + Na]+) obtained from LDI-FT-ICR-MS imaging experiments (n = 1 sample).
Two-dimensional density distributions for light and dark chromatophores over the mantle of an animal (n = 9,199 chromatophores). The composite images show the density of light chromatophores in green and the density of dark chromatophore in magenta. For visualization, densities were linearly scaled together within an image. This preserves relative densities within each image but leads to slightly different colours across images.
a, Schematic showing three hypothetical, partially overlapping motor units (defined by motor neurons (MNs) B, R and G), tracked over three epochs (i–iii), each characterized by different co-activation patterns (epoch i, R alone; ii, B + R; and iii, B + G). Even though chromatophores 1–4 all belong to the same motor unit (R), their average pairwise correlation during these three epochs would differ owing to the activity of the partially overlapping motor units B and G; identifying motor units using this metric would thus fail. This hypothetical example indicates that the units of coordination during behaviour could be smaller than single anatomical motor units (they could also be larger; for example, if some motor neurons are always centrally coupled). b, Single trials of minimal electrical stimulation experiments in in situ nerves. Composite images (one per trial), green, 10 ms pre-stimulus; magenta, 200 ms post-stimulus; white, overlap. Threshold stimulation either leads to the expansion of a set of three chromatophores (marked with red circles, for example, trial 1), or fails to activate any chromatophore (for example, trial 6, 114 motor units determined with this method). c, Colour assignment of chromatophores in situ. Colour label was assigned based on a threshold on the red channel of RGB space (0.3). Chromatophores (dots) belonging to the same motor unit (as determined in a) are connected by lines, revealing the monochromaticity of motor units. n = 114 chromatophores. d, Dark motor elements tend to be larger than light motor elements. Q–Q plot showing quantiles of the dark versus light motor element size distribution. Line, identity. e, Tail of distribution of motor element spread is heavier with dark than light chromatophores. Q–Q plot showing quantiles of the dark versus light motor element spread (calculated as in Fig. 3d). Line, identity.
Left, three similar points along the pattern trajectories shown in Fig. 4b after chromatophore alignment. Right, expanded view of a pattern border. Note the remarkably similar expansion states of the chromatophores at each of the three visits, and the rugged pattern borders at chromatophore scale, with interdigitation of expanded and contracted chromatophores, generating apparent noise. This apparent noise may be critical for natural realism.
a, Three example intermediate-level clusters of motor elements (threshold of 0.4 as in Fig. 3i, different animal), overlaid on the average aligned colour image for the dataset (216,160 images). The clusters are mostly composed of chromatophores of a single colour: cluster 1 (red) is light; clusters 2 and 3 (green and blue) are dark. b, The dynamics of a 60-min dataset, projected onto the first three principal components (48% variance explained, n = 1,437 chromatophores, 52,040 samples). A cluster activity direction can be defined in principal component space by projecting the cluster identity vector (vector of length = number of chromatophores, with 1s assigned to chromatophores in a cluster, otherwise it is assigned 0), onto the principal components. The coloured lines show the cluster activity directions for the three clusters in a. Projecting the dataset onto these directions shows the expansion strength of the cluster at different times. The images corresponding to the times of lowest and highest strengths are shown to the left and right, respectively. c, Full distribution of expansion strengths, projecting all time points onto cluster activity directions. In this dataset, cluster 2 is often expanded, whereas clusters 1 and 3 are rarely expanded. a.u., arbitrary units.
A gallery of aligned patches of skin centred on the position of chromatophore insertion is shown. Top, juvenile animal, 7 days old on the first day of observation (D1). Left-most column shows skin pre-chromatophore-birth. Over approximately 19 days, chromatophores that first appear pale yellow darken progressively, transitioning to orange and red, before finally turning black. Field of views (FOVs): from around 150 × 150 μm on day 1 of observation to 300 × 300 μm on day 25 of observation. Bottom, adult animal, 105 days old on day 1 of observation. Rows show chromatophores undergoing a similar light–dark colour transition as in the juvenile (top), but at a much slower rate. FOVs: around 200 × 200 μm (nonlinear alignment). Examples were chosen from aligned skin patches containing around 100 chromatophores.
a, Flowchart depicting the spatial-growth-model algorithm and highlighting the involvement of model parameters (Methods). b, Box plots of nearest-neighbour (NN) distances between young (<6 days old) and older chromatophores. Young chromatophores are significantly closer to both older light (>12 days) and dark chromatophores than to other young or middle-aged (6–12 days) light chromatophores. (P < 0.0001, Kruskal–Wallis followed by Tukey’s HSD, nchromatophores = 522 for <6 days, 541 for 6–12 days, 1,550 for >12 days, 1,910 dark chromatophores, 1 animal). Distances calculated on a single image, ages estimated by finding the day of chromatophore birth on aligned developmental datasets (Methods). c, Distributions of size for yellow, red (transitional (trans.)) and dark chromatophores, annotated manually (validation of analysis in Fig. 5d). Transitional chromatophores are significantly smaller than either yellow or dark ones (transitional versus yellow, P = 1.0 × 10−7; transitional versus dark, P = 6.3 × 10−4; n = 70 yellow, 16 transitional, 84 dark chromatophores; two-tailed Wilcoxon rank-sum tests, n = 1 animal). Box plots show the central line, median; box limits, quartiles; whiskers, ±2.7 s.d. d, Generation of the inhibitory surround used in the skin growth model (Fig. 6b). Blue, empirical radially averaged chromatophore centred density, inverted and normalized 0:1. Red, logistic function fit to the blue density, as in Fig. 6b. e–h, Manipulating single parameters of the skin growth model suggests the mechanisms underlying colour interdigitation. e, Difference between peak dark-triggered dark-chromatophore density (DtD) and dark-triggered light-chromatophore density (DtL), as a function of model skin growth rate. Points are from the average of three model runs. Line, linear fit. ANOVA F-statistic versus the constant model F = 96.6, P = 0.000186. f, Difference between peaks of radially averaged dark-triggered dark-chromatophore density and dark-triggered light-chromatophore density, as a function of age at which chromatophores transition from light to dark. Points are from a single model run, in which the colour class was changed according to chromatophore age. Line, linear fit. F-test for linear regression: F = 152, P = 5.26 × 10−6. g, Difference between first peak (first zero-crossing of derivative of radially averaged density) in the radially averaged light-triggered light-chromatophore density (LtL) and light-triggered dark-chromatophore density (LtD), as a function of ra, the rate at which the inhibitory surround changes with chromatophore age. Points are from the average of three runs of the model. Line, linear fit. F-test for linear regression: F = 21.9, P = 0.00226. h, Colour interdigitation is robust to stop-criterion used to define end of ‘day’ (parameter 5, Methods). Magenta, DtD − DtL (as in e, f). Black, LtL − LtD. Lines, linear fits. F-test for linear regression: F = 0.0206, P = 0.889 (DtD − DtL); F = 6.57, P = 0.0334 (LtL − LtD). Points in e–h are from the average of three model runs.
Extended Data Fig. 10 Exploration of developmental-model parameters reveals species-specific patterns.
Changing model parameters (see main text and Methods) can lead to the characteristic rings observed in some squid species, with single light chromatophores at the centre and a radial centrifugal darkening gradient. Top, skin of common squid, Loligo vulgaris (image by R. Siegel). Bottom, simulation of development using a profile of change of the inhibitory disc centred on each chromatophore ra different from that used in Fig. 6b for S. officinalis.
Model of colour development in Sepia.
Cropped video of the cuttlefish mantle, demonstrating image alignment. Sparse optical flow is used to retain individual chromatophore dynamics while removing movements at larger spatial scales such as breathing and swimming. Left: raw video. Right: aligned video. FOV: 10x15 mm.
Average aligned color images from single chunks (top left) are warped into a common reference frame (top right), using correlation-based alignment (Methods). All 145 chunks forming a single dataset are displayed, showing the range of rotations, translation, skin pattern changes, and variability in lighting and background that can be successfully aligned. FOV:28x14 mm. Bottom row: zoomed-in views of the skin. FOV (left) 7557x5084 µm, (right) 1031x1031 µm. Note how alignment is achieved at single chromatophore precision.
Registered video of a patch of skin on the cuttlefish mantle, showing spontaneous chromatophore activity in vivo. Note the small groups of spatially clustered chromatophores of single colors showing coordinated size changes, suggestive of common neural drive. FOV: 2366x1330 µm. Similar results observed in N=3 animals.
Aligned video from 3 separate chunks, taken from animal in Fig. 3g-k. Right shows 2 unilateral, bilaterally symmetric clusters of motor elements (MEs) identified through hierarchical clustering and thresholding at an intermediate level (as in Fig. 3h, threshold 0.25). Throughout the movie, changing the threshold shows how the clusters merge into one bilaterally symmetric cluster (threshold 0.75), or split into constituent MEs (threshold 0). Left shows how these intermediate-level clusters occasionally exhibit independent variations, such as unilateral expansion and contraction. At other times, they are coordinated, suggesting common drive. FOV: 18x10 mm. Similar results observed in N=3 animals.
Video of texture and coloration changes discussed in Fig. 4, in response to gentle hand movement. One full cycle lasts about 1min. Real-time playback speed. FOV: 42x22 mm. Chromatophore resolution pattern changes were recorded in N=4 animals.
Video showing aligned patches of skin, separated by 11 days (as in Fig. 5e). The transitions between day-1 and day-12 images and back are done by alpha blending. Note the pale yellow colour of the new chromatophores, and the darkening of existing chromatophores with age. FOV: 2.9 x 1.6 mm (at most zoomed out). Similar results observed in N=2 animals.
About this article
Cite this article
Reiter, S., Hülsdunk, P., Woo, T. et al. Elucidating the control and development of skin patterning in cuttlefish. Nature 562, 361–366 (2018). https://doi.org/10.1038/s41586-018-0591-3
- Skin Pattern
- Master Frame
- Average Image Color
Linking genotype, cell behavior, and phenotype: multidisciplinary perspectives with a basis in zebrafish patterns
Current Opinion in Genetics & Development (2020)
Current Opinion in Neurobiology (2020)
Annual Review of Animal Biosciences (2020)
Insect Biochemistry and Molecular Biology (2020)