The engineered eyeball, a tunable imaging system using soft-matter micro-optics

Abstract

We demonstrate a tunable imaging system based on the functionality of the mammalian eye using soft-matter micro-optical components. Inspired by the structure of the eye, as well as by the means through which nature tunes its optical behavior, we show that the technologies of microsystems engineering and micro-optics may be used to realize a technical imaging system whose biomimetic functionality is entirely distinct from that of conventional optics. The engineered eyeball integrates a deformable elastomeric refractive structure whose shape is mechanically controlled through application of strain using liquid crystal elastomer (LCE) actuators; two forms of tunable iris, one based on optofluidics and the other on LCEs with embedded heaters; a fixed lens arrangement; and a commercial imaging sensor chip. The complete microsystem, optimized to yield optical characteristics close to those of the human eye, represents the first fully functional, soft-matter-based tunable single-aperture eye-like imager.

Introduction

Whereas a variety of biomimetic ‘artificial eyes’ have been the subject of study and development, these have generally been either modeled on the compound eyes of insects^{1, 2, 3} or been shown to have limited functionality^{4, 5, 6, 7}. On the other hand, a wide spectrum of micro-optical implementations of tunable lenses^{8, 9, 10} and irises^{11, 12, 13} have been demonstrated, some with excellent optical functionality, yet these have rarely been incorporated into complete imagers. The engineered eyeball we present here shows for the first time that soft-matter materials employed for tunable optical components as well as actuators can be used to realize a complete single-aperture imaging system.

The mammalian eye comprises a fixed refractive surface (the cornea), a variable aperture (the iris), a tunable lens (the lens) and a sensor (the retina)¹⁴, as illustrated in Figure 1a. The engineered eyeball replicates this functionality using the components shown in the schematic sketch of Figure 1b by combining the technologies of elastomer-based micro-optics¹⁵, optofluidics¹⁶ and liquid crystal elastomers (LCEs)^{17, 18}. The complete system we realize here, seen in cross-section in Figure 1c, represents a tunable imager whose design is optimized to closely mimic the performance of the human eye. Although imaging systems of equivalent size fabricated using classical optical components may certainly display better optical performance, the engineered eyeball shows that microtechnology may be used to realize deformable and fluidic components, using muscle-like actuation, to realize an imaging system with roughly the same functionality as the natural eyeball.

Figure 1

The cornea, iris, lens and retina of the human eye (a) are replicated using soft-matter tunable technical components (b) and integrated into the engineered eyeball (c); the latter has a diameter of 7 cm.

In the following, we describe the individual soft-matter technologies used to realize the tunable lens and iris structures, and subsequently address the integration of these into a complete imaging system. The tunable lens is based on a deformable elastomeric refractive structure whose shape is mechanically controlled through application of strain, using LCE actuators, resulting in a change of focal length. This material represents an ‘artificial muscle’, a soft-matter elastomer/liquid-crystal hybrid that undergoes a large temperature-dependent length change and is used to tune the iris and lens^{19, 20, 21}.

Two forms of tunable iris were implemented, one based on optofluidics, using clear and opaque liquids whose circularly symmetric interface is tuned using electrowetting and the other on LCEs with embedded heaters, whose configuration allows controlled tuning of a clear aperture through radial contraction of the material.

These tunable components were integrated into a complete imaging microsystem, which also included a specially designed fixed lens arrangement (consisting of a Cooke triplet with an additional lens to correct field curvature) and a commercial imaging sensor chip²². The components and their integration were optimized to yield optical characteristics close to those of the human eye, namely, a field of view of 40°, a near point of 20 cm and a focal length tuning of the tunable lens of 10% permits accommodation between the near point and infinity.

Materials and methods

The engineered eyeball, as schematically outlined in Figure 1b, consists of a tunable lens, a tunable iris and fixed optical components. The lens and iris are soft-matter components whose structure and functionality we first present in detail before attending to the overall system design.

Liquid-crystal elastomers

Sometimes colloquially referred to as ‘artificial muscles’¹⁹, LCEs are a relatively new class of materials that combine the elasticity of an elastomer with the ordering properties of a liquid crystal²¹. The LCE thus has two distinct phases, one in which the liquid crystals are ordered and the other disordered, and a phase transition between the two is accompanied by a significant length change of the elastomeric material. Temperature-induced phase changes can lead to reversible length changes ranging 20–80%, such that LCEs are excellent soft-matter actuators.

LCE actuators may be structured and oriented using microsystems fabrication processes¹⁸. The liquid LCE precursor, a 4′′-acryloyloxybutyl 2,5-(4′- butyloxybenzoyloxy)benzoate monomer mixed with 10 mol% 1,6-hexanediol di-acrylate and 1 mol% diphenyl(2,4,6-trimethylbenzoyl) phosphinoxide, is deposited into a heated (95 °C) poly(dimethylsiloxane) (PDMS) mold. Metallic heater and temperature sensor structures, typically polyimide-coated Pt, can be placed in the mold to incorporate them into the elastomer during molding.

The key to the use of LCEs as actuators is the definition of the orientation of the mesogens, which defines the direction of contraction during the phase transition. Orientation is accomplished here by cooling the LCE precursor to 60 °C in a magnetic field, the direction of which then defines the mesogen orientation. The magnetic field is configured linearly for realization of linearly contracting LCEs, as those used for vectored deformation of the tunable lens, or radially in the case of the LCE iris, both considered below. The LCE precursor is then polymerized using ultraviolet light, completely cooled and demolded.

Tunable microlens

The tunable lens for the engineered eyeball is a stretchable, elastomer-based refractive lens whose shape is controllably deformed using LCE actuators¹⁵. The convex lens is fabricated in a molding process using commercial glass concave lenses as high-precision masters. Microfabricated anchors are placed into the mold before injection of the PDMS (w_b=75% Dow Corning SE 1740 mixed with w_b=25% Wacker Elastosil RT 604) liquid precursor, such that they are embedded into the periphery of the lens after curing of the elastomer.

Given the PDMS refractive index of n=1.405 (λ=589 nm, 20 °C), the resulting plano-convex lens, with radius of curvature |R|=24.82 mm, had a focal length of f₀=61.3 mm in its relaxed, unstrained state, with no observable birefringence. Mounted into the actuation set-up seen in Figure 2a, the eight anchors in the lens periphery are attached to linearly oriented LCE actuators with embedded heaters and temperature sensors, as described above. Activation of the actuators, generating a contraction force of roughly 120 mN, pulls on the anchors, thereby deforming the lens.

Figure 2

The deformable LCE-actuated PDMS-based lens, whose focal length is tuned by stretching. (a) Eight anchors embedded in the lens are attached to linear LCE actuators, which stretch the lens radially. (b) Tuning of BFL of two stretchable PDMS microlenses as a function of applied strain, a plano-convex lens (f=61 mm, red) and an equi-convex lens (f=31 mm, green). BFL, back focal length.

The actuators are individually activated and may thus be used to deform the lens asymmetrically, an effect that has been demonstrated to allow controlled tuning of astigmatism and other primary aberrations¹⁵. For the engineered eyeball, the fundamental tuning parameter is defocus, achieved by uniform actuation of all eight LCEs simultaneously. As shown in Figure 2b for biconvex and plano-convex lenses, application of symmetric strain allows tuning the focal length of the lens by almost 10%. We recall that the lens of the human eye is used for accommodation, and typically (for a young person) tunes 22–30 diopters (45–33 mm focal length), such that the mechanical tuning range of 10% achieved here is sufficient for the optical accommodation from a near point of 20 cm to infinity.

LCE tunable iris

The variable aperture of the engineered eyeball, the iris, is realized in two variants using two different approaches: the LCE, considered in this section, and fluidics, considered in the next. The LCE iris is fabricated entirely from a radially oriented LCE, using the same technology as outlined above for the linear actuators implemented in the tunable microlens. In this case, the applied magnetic field during cooling is radially oriented with a magnitude exceeding 100 mT, resulting in an elastomer that contracts in the radial direction upon actuation¹⁷.

The resulting iris structure, shown in Figure 3a, has a diameter of 12 mm, a thickness of about 300 μm and an initial aperture diameter of 2.7 mm; one advantage of this structure is that the open aperture is free space, with no surfaces or materials in the optical path. Polyimide/Pt heaters, with a resistance on the order of 3 kΩ, are again embedded in the elastomer. By application of a 50 V drive voltage to the heaters, the LCE contraction leads to an opening of the iris, increasing the aperture diameter to 3.8 mm, as seen in the time-dependent actuation characteristic of Figure 3b. The 10–90% time constant for opening the iris is about 7 s and for closing about 3 s, the asymmetry resulting from the thermal difference in active heating and passive (predominantly radiative) cooling. The response speed of the iris in the human eye is in the range 200–500 ms (for closing; opening is slower). The time constants of the optofluidic iris, however, are comparable or faster, as we shall see later^{23, 24}.

Figure 3

The LCE iris, fabricated entirely from a radially oriented LCE with embedded heaters. (a) Fabricated and mounted LCE iris, before application of opaque layer; the heaters are clearly visible. (b) Change in the iris aperture during actuation of the LCE iris: diameter (left axis) and roundness (right axis). Figure reproduced with permission from Ref. 17 © 2014, John Wiley and Sons.

Also considered in Figure 3b is the ‘roundness’ of the aperture, a parameter that will ultimately affect the image quality in the eyeball. As the open aperture of the iris is free space, and the inner diameter of the LCE is thus not mechanically defined, it is not clear that the iris will remain circular under actuation. That this is nonetheless almost always the case is seen from evaluation of the roundness parameter Ψ, given by Ψ=4A/(πM), in which A represents the area of the aperture and M the major axis of an ellipse fitted to the aperture, which is plotted as a function of actuation in the figure. The value Ψ remains above 0.96 (a value of unity implying a perfect circle), suggesting that the open iris remains very close to circular over its entire actuation range.

Fluidic tunable iris

The second variant for the tunable aperture is based on optofluidics. This concept is based on a completely liquid-filled cavity, using a combination of transparent and opaque fluids. As shown schematically in Figure 4a, an opaque aqueous liquid is configured as a ring embedded in an oil-based transparent fluid ambient. Electrowetting actuation is then used to pull the inside edge of the opaque ring toward the center, using an arrangement of concentric circular electrodes. The water-based opaque ink and the oil-based ambient have closely matched densities (1.06±0.005 g cm⁻³), such that gravitational and vibrational effects have virtually no effect on the interface shape. The fluidic iris may be fabricated on the wafer level, with separated chips giving rise to a compact, fully sealed and self-contained system, as shown in Figure 4b¹⁶. Key to efficient and reproducible operation of the fluidic iris is the correctly tuned hydrophobicity and electrical properties of the surface layers.

Figure 4

The fluidic iris, based on electrowetting-actuated opaque and transparent fluids in a completely sealed, self-contained chamber. (a) Image of the actuation chip with the dispensed opaque ink ring in the open state. (b) Packaged fluidic iris system; the ink is surrounded by a transparent oil.

By application of a 100 V root-mean-square (RMS) alternate current drive voltage (300 Hz) onto successive electrodes, the inner radius of the opaque ink ring can be pulled inwards, closing the iris; three electrodes provide four discrete steps in aperture diameter, from 1.77, 2.50, 3.54 to 5.00 mm, with newer designs allowing apertures down to 0.32 mm. As the aperture shape is given by the photolithographically defined circular contacts, roundness is not an issue and Ψ is very close to unity.

As is seen in Figure 5, the 10–90% time constants are below 280 ms for closing the iris and 500 ms for opening it, which is comparable or faster than the response of the iris in the human eye^{23, 24}. The asymmetry in this case results from the fact that closing the iris by application of V_drive=100 V is an active, electrowetting-driven effect, whereas opening it relies on forces resulting from surface tension, and is thus a passive process. Both the switching time as well as the power consumption (typically<5 mW) are a function of bias voltage and frequency.

Figure 5

Switching characteristics of the fluidic iris with V_drive=100 V: closing the iris by applying a bias to one (triangle), two (diamond) or three (square) electrodes; opening the iris (circles) by applying V_drive=0 V.

In contrast to the LCE iris, whose aperture is simply an air-filled hole with no optical structures in the beam path, the optical path of the fluidic iris passes through the transparent liquid and several interfaces, resulting from the top and bottom of the package. Considering absorption and Fresnel reflections in the absence of any antireflection coatings, transmission in the open state exceeds 86% for visible wavelengths, 390 nm≤λ≤750 nm; absorption of the opaque region exceeds 80 dB, thus providing excellent contrast. Measured induced wavefront errors are <88 nm RMS, implying extremely small induced aberrations. The device has been lifetime-tested through >100 000 cycles with no discernible degradation.

System concept

Given the two essential tunable components for realization of an accommodating engineered eyeball, these then need to be integrated into an optically optimized design also incorporating the fixed optical components, a static lens (the cornea) and the sensor (the retina). Optimization criteria are fulfillment of the design goals within certain constraints (typically spacings) as given in Table 1.

Table 1 Performance objectives and design constraints

For the static lens, incurring most of the optical power of the system, a Cooke triplet forms an ideal basis for an initial design²², primarily as this three-lens arrangement is the simplest that allows independent correction of all five primary Seidel plus chromatic aberrations. Whereas the classic Cooke triplet can yield diffraction-limited performance, the question in the system design becomes where to place the tunable elements; in addition, the individual lens curvatures, spacings and refractive indices (through material choice) need to be chosen for optimal performance. Through simulation using Zemax (Zemax, LLC, Kirkland, WA, USA), in which a user-defined merit function is iteratively maximized, it was determined that optimal performance can be achieved with the arrangement depicted in Figure 6a, which has been modified from a theoretical optimum to reflect manufacturability of the triplet combined with the rightmost lens for field curvature correction. Although still below the diffraction limit, the calculated modulus of the optical transfer function, which is commonly referred to as modulation transfer function (MTF), of the system, given in Figure 6b, shows that good performance can be expected.

Figure 6

The complete optimized refractive lens system, with fixed and tunable components. (a) The lens arrangement, incorporating a Cooke triplet with a tunable plano-convex lens (left) and a lens for field curvature correction (right). (b) Calculated MTF of the refractive lens system, for the S and T directions for rays at 0°, 10° and 20°. OTF, Optical transfer function; S, sagittal; T, tangential.

A subsequent Monte-Carlo-based tolerance analysis of the positions, spacings, tilt and decenter of the lens elements revealed the most critical parameters in the lens system assembly, essential for realization of a reliably performing micro-optics-based imager. Using a maximally acceptable increase in the radius of the spot diagram on the imager of 10% as an exclusion criterion, it was seen that the most critical parameter is lens tilt. Depending on which individual lens or group of lenses is considered, tilt must be kept <0.64°, decenter ≤0.11 mm and spacing variations ≤0.03 mm.

The refractive optical components, with one tunable element, are then finally combined with the tunable iris and the sensor into the complete engineered eyeball, shown in the exploded view of Figure 7. The sensor employed was a commercial complimentary metal oxide semiconductor (CMOS)-based board-level camera (uEYE LE, IDS Imaging Development Systems GmbH, Obersulm, Germany) with 1280 × 1024 pixels, a 5.3 μm pixel pitch, and a shortest chip dimension of 4.345 mm. The tunable components, the lens and both variations of the iris, are mounted on custom printed circuit boards, and the static lens assembly on an XY positioning mount. All of these were designed to be compatible with a 30-mm cage system, incorporating mechanical spacers and positioning rods, and the assembly was finally incorporated into a 7-cm diameter three-dimensional-printed spherical housing to eliminate stray light.

Figure 7

Exploded view of the engineered eyeball assembly. From left to right: the tunable iris, tunable refractive lens, fixed lens assembly, and sensor.

Results and discussion

We thus see how a complete eye-like imaging system can be conceived and fabricated using soft-matter micro-optics and actuation. Whereas there is a considerable body of work in realizing biomimetic imaging systems, the vast majority of these concern multi-facet (that is, insect) eyes^{1, 2, 3}. Biomimetic single-aperture (that is, mammalian) eyes such as that presented here more sparsely populate the literature. Notable among these is the work of Liang et al.⁴, who integrated a voice-coil-driven deformable lens with a fixed poly(methylmethacrylate) ‘cornea’ lens and a commercial image sensor. Unfortunately, the optical performance of this system, which does not include a tunable aperture and thus only yields accommodation (29.5–5.1 cm), was not described in any detail. Alternatively, the work of Santiago-Alvarado et al.⁵ presented the design of a system comprising a poly(methylmethacrylate) ‘cornea’ lens, a standard mechanical iris and an elastomeric lens, in a system considerably larger than that presented here using complex, classical mechanical actuation techniques; the system appears only to have been simulated, not constructed. Finally, Arianpour et al.⁶ have fabricated an optomechanical model eye, which was, however, intended for testing of intraocular lenses and did not include any tunable optical components.

To our knowledge, therefore, no biomimetic imager of any kind has used soft-matter ‘muscle’ actuation for either focal length or aperture tuning and we now consider the optical characteristics of this system in detail.

The characterization of imaging performance using the complete engineered eyeball system comprised measurement of MTF, depth of field (DOF), focus position as well as qualitatively evaluable images as a function of tuning behavior. MTF measurements were accomplished by Fourier transform of the measured edge spread function; sharpness (focal position) and DOF were determined by integration of the MTF, which attains a maximum value for an in-focus image. Maximum ‘sharpness’, as used in the characterization below, was then defined as the point at which this integral is maximal.

Using the fluidic tunable iris, the aperture could be tuned 1.77–5.0 mm, resulting in a variation of MTF as well as DOF. As is expected, MTF decreases as aperture size increases, as is noted in Figures 8. For a 2.5 mm aperture, the complete system showed a resolution of 2.0 line pairs per millimeter, compared with 2.24 line pairs per millimeter as expected from simulation. The increase in DOF with decreasing aperture diameter is clearly seen in Figure 9, which images numerous axially displaced objects at two aperture sizes. For the larger aperture size, only the front-most figure is in focus; for the smaller, all figures and the background text are sharply imaged. The two images have the same overall brightness due to positive gain compensation in the sensor as the aperture size is decreased.

Figure 8

Aperture tuning using the fluidic iris: variation in the measured MTF as the iris diameter is varied 1.7–5.0 mm. OTF, optical transfer function.

Figure 9

Aperture tuning using the fluidic iris: variation in the measured DOF for iris diameters of 5.00 (a) and 1.77 mm (b). Notice that all figures and the background text are in focus for d=1.77 mm; distance between the front and rear figures is 220 mm.

The LCE iris was also implemented for tuning, and allowed a continuous variation in aperture between 2.7 and 3.9 mm. In contrast to the fluidic iris, whose discrete step-like tuning derives from the discrete actuation electrodes, the LCE iris could be used to continuously vary both the DOF as well as overall light intensity imaged onto the sensor.

The variation in DOF with aperture tuning is quantitatively evaluated in Figure 10, which presents the measured sharpness as a function of axial distance for different apertures. A broader characteristic implies larger DOF, and we see that the full-width at half-maximum varies from 23 mm at d=5.0 mm to >110 mm at d=1.77 mm. The accompanying shift in focal length to longer values for decreasing aperture size is due to the obstruction of marginal rays for the smaller aperture, thereby decreasing spherical aberration. As positive spherical aberration shifts the focus of peripheral rays toward the lens, its reduction results in an increase of measured focal length.

Figure 10

Measurement of sharpness as a function of axial position for different aperture values; the width of the peak defines the DOF. The object distance is smaller than that of Figure 11, as a larger sensor/optics spacing was used here.

By tuning the focal length of the elastomeric lens (with an unactuated focal length of f₀=55.2 mm), an object-distance tuning range variation of up to 21% (76 mm) was determined, as is shown in the characteristic of Figure 11, in which the axial position of the point of highest sharpness is plotted as a function of voltage applied to the LCE actuators of the lens. The effect is seen qualitatively in Figure 12, which again images numerous axially displaced objects; the blue arrow emphasizes the object in focus for different focal length values of the tunable lens.

Figure 11

Axial position of the point of highest sharpness as a function of voltage applied to the LCE actuators of the tunable lens.

Figure 12

Accommodation using the deformable lens: variation in the focal position. The object-distance values correspond to the focal length tuning measurement shown in Figure 11.

When the overall performance of the system is compared with the human eyeball, our original optical inspiration, we see thus that the characteristics of the engineered eyeball are in the same ballpark as those of the natural model. Concerning the tunable components, the LCE-actuated lens tunes ~16–18 diopters (61–56 mm focal length), as compared with 22–30 diopters (45–33 mm focal length) for the eye. Demonstrated system performance did thus not allow complete accommodation from the near point (20 cm) to infinity as originally envisaged. Typical pupil diameter change for the human eye ranges 2–8 mm²², which is nonetheless easily technically implemented using the LCE or fluidic irises.

The human eye is known to be a highly aberrated system¹⁴ that displays far from diffraction-limited performance. Despite the fact that the various optical aberrations are to some extent mutually compensated through an interplay of the optical components, it is also well known that it is the brain, an indispensable ‘image processor’, which ultimately leads to acute human vision. The engineered eyeball is similarly limited in performance, but does not benefit from such further image processing.

Conclusions

With potential for further miniaturization and integration with other microsystems technologies, the engineered eyeball represents the first fully functional, soft-matter-based tunable single-aperture eye-like imaging system with optical performance on the same order as the human eye. The novel technologies and materials used in its realization pave the way for new concepts in compact tunable micro-optics.