## Introduction

Efficient manipulation of guided electromagnetic waves is of vital significance in numerous applications in nanophotonics and integrated optics. Distinctive from its electronic counterparts, photonic integrated circuits deploy guided light waves instead of electrical signals to carry information1,2. As the bandwidth and power density limit of data transport in electrical wires are increasingly manifesting with higher integration density3, integrated optical scenarios have shown promising inroads towards ultrafast and broadband information processing with low power consumption that may potentially circumvent current electrical bottleneck4,5,6. Various applications have been reported in this field such as chip-scale optical signal processing, communications, and analog computing7,8,9, as well as emerging technologies in quantum, biomedicine, and sensing10,11.

However, recent advancements of photonic integrated circuits are hindered by limitations in its most fundamental building block of traditional optical waveguides, in terms of restrained accessible functionalities, compromised efficiency and bulk footprint12,13,14. The very limited design library of conventional waveguide structures substantially constraints their function- alities12,15 to mostly mere waveguiding. For instance, dielectric waveguides are ubiquitously applied to control on-chip propagating modes, while plasmonic waveguides play an essential role in guiding surface waves. Optical fibers are ideal for long-distance information communications and optical connections between chips and boards. However, the continuous trend towards miniaturized and versatile photonic systems demands more complicated device functions to be realized in a compact, multifunctional, configurable and CMOS-compatible way1.

Fulfillment of these tasks will entail novel waveguide structures. Fortunately, recent advent of metasurfaces and metamaterials opens a new pathway towards powerful light manipulation by engineering photonic structures in subwavelength scale14,15. Transferring the concept of meta-optics into guided waves can help overcome the abovementioned challenges12, by infusing new degrees of freedom into waveguide landscapes to dramatically boost device performance and enable novel functionalities12,13,14,15,16.

Optical metasurfaces and metamaterials are generally composed of judiciously designed artificial structures with feature size much smaller than light wavelength16,17. Metamaterials constructed by three-dimensional bulk subwavelength architectures can realize spatially changing exotic optical parameters (including the permittivity and permeability), giving rise to the transformation optics for invisibility cloaks and slow light phenomena18,19,20. Metasurfaces, in contrast, applying two-dimensional arrays of scatterers21,22,23, also exhibit unprecedented flexibility in controlling the fundamental attributes of electromagnetic waves, such as the amplitude, phase, polarization, wavefront, and so on. Fruitful applications are also reported such as metalens24,25,26, efficient holograms27,28, functional coatings29, color display30,31, LiDAR32 and nonlinear optics33,34. Previous research attentions are mainly devoted to free-space applications. However, recent years have seen a tremendous interest in synergizing meta-optics with various optical waveguides to largely empower conventional photonic devices12,13,14,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110. The advancement of subwavelength meta-structured waveguides can not only extend meta-optics physics to the realm of guided electromagnetic waves, but also promise to reshape the landscapes of photonic integrated circuits and massive emergent applications such as lab-on-chip technologies and neuromorphic photonics8,12,13,59,60.

In this review, we discuss recent progress on various subwavelength meta-structured waveguides, encompassing a broad class of photonic devices and systems that ally metamaterials and metasurfaces with diverse optical waveguides (dielectric/plasmonic/optical fibers)35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110. Brief physical fundamentals with explicit design methods and representative applications for meta-waveguides are comprehensively summarized. We highlight how incorporating the concepts of meta-optics with waveguide technologies can propel photonic integrated circuits into new heights, by providing versatile efficient coupling interfaces35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50, novel on-chip optical signal processing paradigms12,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73 and diverse platforms for sensing, imaging and artificial intelligence57,58,59,60,74,75,76. We further comment on current challenges in device design and practical hurdles from ripening into viable technology. Potential future research directions are also discussed based on current perspectives.

## Meta-waveguide fundamentals and properties

As is illustrated in Fig. 1a, b, meta-waveguides can be classified via either design methods or underpinning waveguide platforms12,13:

• On the one hand, meta-waveguides can be conceived by physical intuition-based approaches (namely forward design henceforth) by leveraging the toolbox of metasurfaces and metamaterials with waveguide optics (discussed in Sections “Dielectric waveguide-integrated meta-structures”, “Optical meta-fibers”, and “Plasmonic meta-devices for controlling surface waves”).

• On the other hand, we can also use inverse design15, which relies on computer optimizations, to develop free-formed analog (curvilinear boundaries)57,58,59,60 or digital metamaterial waveguides61,62,63,64,65,66,67,68,69,111 (elaborated in Section “Inverse-designed metamaterial waveguides”).

Based on different waveguide platforms, metasurfaces and metamaterials can be synergized with dielectric waveguides and optical fibers to tailor optical modes, or with plasmonic waveguides to manipulate surface waves, as shown in Fig. 1b.

• By saddling subwavelength architectures on top of dielectric waveguides, exquisite control over the electromagnetic fields is attainable to facilitate versatile couplers36,37,38,39,40,41,42,43,44, compact polarization- or wavelength- routers48,49,50, on-chip structured light generators35,45,46,47,77,78,79,80,81, integrated mode convertors51,82,83,84,85,86,87,88,89, sensors90,91,92,93 and nonlinear devices52,94,95 (discussed in Section “Dielectric waveguide-integrated meta-structures”).

• The introduction of meta-structures into optical fibers also greatly enriches traditional fiber devices, ushering the creation of tremendous novel meta-fiber applications in information modulations96,97,98, beam transformations99,100,101,102,103, imaging74 and numerous high-performance biochemical sensors and detectors75,76,104,105,106,107,108,109,110 (detailed in Section “Optical meta-fibers”).

• Meanwhile, meta-optics are hatching as excellent candidates to construct plasmonic meta-waveguides for controlling surface waves. Abundant research has been invigorated, including efficient multifunctional surface waves excita- tions70,112,113,114,115,116,117,118,119,120,121,122 and manipulations71,72,73,123,124,125,126,127,128 with largely miniaturized device footprint (detailed in Section “Plasmonic meta-devices for controlling surface waves”).

In the following, we start by briefly introducing the fundamentals and unique features of subwavelength meta-structured waveguides. General design process and device operation scenarios are concisely covered. Next, we comprehensively review latest researches derived from three different types of forward-designed meta-waveguides in separate sections, according to three distinctive underpinning waveguide platforms: dielectric waveguides, optical fibers and plasmonic waveguides (shown as Fig. 1b). Then we catalog inverse-designed metamaterial waveguides, focusing on waveguide-based design tutorials and algorithms, key applications, and comparisons of different design approaches for tailoring guided wave. Finally, we discuss current challenges and outline exciting opportunities of this vibrant field for integrated photonics and beyond.

### Definitions, features and properties

Meta-waveguides here refer to a set of physical structures with engineered subwavelength features that guide electromagnetic waves. A simple case is the waveguiding media itself is man-made metamaterials129,130,131,132,133 consisting of intricate subwavelength building blocks. Leveraging the mathematical technique called transformation optics20, researchers can design waveguides with artificial refractive index distribution to realize exotic waveguiding phenomena such as theoretical arbitrary waveguide bend and light trapping129,130,131,132,133,134,135. However, in addition to the loss issues, these scenarios encounter experimental challenges in fabricating sophisticated bulk structures at high optical frequencies17,20, prohibiting its widespread implementation.

Another general case is applying meta-structures on waveguide surfaces to act as Mie resonators or Rayleigh scatterers21,22,23 to perform designer index perturbations35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,136. The structures can be metal or dielectric materials on top of as-fabricated waveguides after deposition and lift-off35,37,48,49,50,51,52. Alternatively, these subwavelength features (either fully38,39,47,54,55,56,63,64,65,66,67,68 or partially etched45,78,84,85,86,87,88,89) itself can be part of the waveguides defined by lithography. These devices are application-wise more appealing, for they are promising for mass-production by lithographic or imprinting-based nanopatterning12,137,138,139.

As shown in Fig. 2a, meta-waveguides as the penetration of meta-optics into waveguide optics51 can inherit the flexibility and versatility of light manipulation from its free-space counterparts of metasurfaces and metamaterials. The most significant distinction of meta-waveguides from their non-subwavelength equivalents like photonic crystal waveguides is waveguiding mechanism. As is illustrated in Fig. 2b12, structures of subwavelength dimension are crucial for meta-waveguides, because in this case the guided electromagnetic waves are experiencing an effective media with designed optical responses18,19. This mechanism is physically distinctive from photonic crystals (PhC) that rely on photonic bandgap140. Waveguiding in PhC is realized by introducing a line defect supporting guided modes against the photonic bandgap formed by the Bragg-like diffractions from periodic refractive index variations140. Taking subwavelength grating waveguides consisting of periodic silicon segments as an instance12,13,14, effective medium theory should be applied. Floquet–Bloch modes are supported without propagation loss12 if the period Λ is much smaller than light wavelength λ. As Λ gets bigger, light starts to decay as photons are forbidden from propagating within the photonic bandgap. Then the structure will behave as a diffraction grating when the structure period steps into radiation regime12.

### General design directions

Ramping up from scattered explorations by applying subwavelength structures141,142 and nanoparticles on waveguides143,144, meta-waveguides can now be systematically investigated by either forward or inverse-design methods.

In Fig. 2c we plot the two general design process. Except for some cases to elaborate later in next section83,84,85,86,87,88,89,90,91,129,130,131,132,133,134, most forward-designed meta-waveguides start from the analysis of single meta-atom unit41,42,43,44,48,49,50,51,52,70,71,72,73,74,75,76, which is the most fundamental element to perform field modulation35,145,146. By assembling designer meta-units into arrays, a functional meta-structure can be conceived145,146 and then integrated with different waveguides. However, this final step requires further attentions to properly engineer phase-matching and mode overlap issues35,44,51,52, the specific solutions of which will depend on application and the underlying waveguide platforms. Detailed design methods are elaborated in the following three sections.

As shown in the lower panel of Fig. 2c, inverse-design methods start from a different direction. The target device function is firstly specified in order to determine objective functions. Then computer optimizations are conducted to retrieve device structure under given constraints, such as gradient-based iterations with adjoint methods15,147 and deep learning algorithms148. Comprehensive design tutorials are detailed in later Section “Inverse-designed metamaterial waveguides”.

Forward-designed meta-waveguides have explicit physical pictures and are relatively straightforward to design with excellent performance. In contrast, inverse-designed metamaterial waveguides are computationally heavy in the implicit optimization process but can realize some sophisticated functionalities that may be hardly assessable by forward design57,60.

### General application scenarios

Considering all linear optical devices are mode converters149, meta-device functionalities can be ascribed to two general scenarios: linear mode conversion and nonlinear hybrid applications.

In conventional waveguides devoid of structural perturbations, conversion between two arbitrary electromagnetic modes is generally inaccessible due to wavevector mismatch83,149. However, in meta-waveguides, the meta-structures can provide an effective momentum keff to enable desired mode conversion51,82. The schematic is sketched as Fig. 2d, where Mode A and B can be free-space light beams, propagating waveguide modes, or surface waves. Lightwave propagating in a meta-waveguide undergoes consecutive and subwavelength-scale delicate field modulations (induced by resonance or form birefringence35,150,151,152) from the engineered structures, leading to an overall wavevector change of the electromagnetic modes51. Besides the collective scattering events from the structural perturbations, the mode conversion can be also interpreted as the contribution of multi-path interference from these subwavelength-dimensioned features153.

For nonlinear applications, phase-matching is pervasively required owning to energy and momentum conservations. Pronounced nonlinear phenomena usually demand rigorous phase-matched conditions. However, this requirement can be cir-cumvented by integrating meta-structures to waveguides of nonlinear materials to break mode conversion symmetry52,94,154,155. Considering the pumping optical mode as TE00(ω1) as an example, the generated nonlinear signal TE00(ω2) are then converted to other modes with same frequency ω2 but different mode orders: TEmn(ω2) and TEpq(ω2), which will be preserved and accumulated with propagation52,154. Back coupling from the retained nonlinear modes to TE00(ω2) is prohibited due to phase mismatch52 and minimal field overlap, as the effective wavevector keff provided by the meta-structures is unidirectional51,82. In addition, the nonlinear process can also be enhanced by meta-structures with optimized nonlinear overlap95.

## Dielectric waveguide-integrated meta-structures

Enabled by the ever-increasing resolution offered by nanofabrication technologies, dielectric meta-waveguides open exciting venues towards versatile chip-integrated applications. This section mainly includes dielectric waveguides with subwavelength surface structures and forward-designed metamaterial waveguides, focusing physical model, design method and applications.

### Tuning scattering properties of nanoantennas on a waveguide

Analogy to well-established microwave and radiofrequency antennas, optical antennas are essential building blocks for manipulating light radiation at subwavelength scale21,22,23,156. Before delving into the details of metasurface-patterned dielec- tric waveguides, we begin with easy interference model of plasmonic optical antennas40,48,49,50,94 and discuss their rational implementations on waveguides for directional coupling applications and so on.

Starting from the most fundamental model, the response of an optical antenna with dimension much smaller than light wavelength can be approximated by an electric dipole $${{{\mathbf{P}}}}_{{{\mathrm{a}}}} = {{{\mathbf{A}}}}\exp \left[ {{{{\mathrm{i}}}}\left( {{{{\mathbf{k}}}} \cdot {{{\mathbf{r}}}} + \alpha _a} \right)} \right]$$ under external electromagnetic stimulus37,40,48,49, where A denotes radiation amplitude, $$\left| {{{\mathbf{k}}}} \right| = 2\pi {{{\mathrm{/}}}}\lambda$$ is wavevector, λ stands for light wavelength and r = (x, y, z) is a spatial vector unit157. Acting as a resonator and scatterer, the optical antenna will have a characteristic phase αa depending on its shape, material, environment index, and light wavelength21,156. As is illustrated in Fig. 3a, by applying another antenna dipole $${{{\mathbf{P}}}}_{{{\mathrm{b}}}} = {{{\mathbf{B}}}}\exp \left[ {{{{\mathrm{i}}}}\left( {{{{\mathbf{k}}}} \cdot {{{\mathbf{r}}}} + \alpha _b} \right)} \right]$$, directional emission can be achieved48 by properly controlling the antennas displacement and initial phase responses αa and αb. For instance, if we have destructive interference (αa + αp) − αb = π in the right side (αp is the propagation phase determined by antenna displacement)40, directional emission to the left side is realized.

Leveraging this intuitive mechanism, antenna combos can be designed on top of dielectric waveguides to realize directional coupling40,94. Alternatively, some antenna elements can also appear as aperture, which can be investigated similarly using Babinet’s principle48. For example in Fig. 3b, the combination of a gold nanodisk and air nanoslit forms a Fano antenna49, which is featured by the asymmetric Fano line-shape originating from the overlap of a broad and narrow resonance from the nanodisk and nanoslit respectively. By saddling this Fano antenna (with opposite bidirectional scattering attributes on two different wavelengths) on rectangular48 or planar waveguides49, wavelength-selective demultiplexers are realized. In addition to the coherent interference from double-element antennas, directional emission can also take place in a single antenna, when the radiation from different internal plasmonic modes interferes destructively in the other side (Fig. 3c) for directional couplers158.

A more general antenna array configuration for high directivity is Yagi–Uda antennas, which are inspired by its radiofrequency counterparts consisting of metal rods acting as feed, reflector and directors156,159,160. Endeavors are pursuit for implementing Yagi–Uda antennas to waveguides37,40,50,160,161,162. Waveguide-hybridized gold Yagi–Uda antennas can not only directionally in-couple free-space light source to guided waves, but also out-couple waveguide modes to free space and localized plasmons37, with further applications in on-chip data communications, directional lasers, and metrology160,161,162,163.

Moreover, the scattering attributes of the antenna elements are also polarization-controllable. For instance, the two gold nanorods horizontally resting on a silicon waveguide along the x axis (left panel of Fig. 3d)50 directionally couple linear x-polarization Ex to right-propagating TM mode. However, this array barely interacts with y-polarized light Ey, as the electrical current in the slender nanorod antenna cannot be efficiently excited when the incident electric field vector is perpendicular to its orientation21. In contrast, the structure shown in the middle panel of Fig. 3d can couple incident Ey into left propagating TE mode with high directionality. By combining the two antenna arrays with opposite coupling direction and distinctive response to incident polarizations, chip-integrated polarization demultiplexers for high-speed optical communications are realized50.

Plasmonic chains are also incorporated to various dielectric waveguides to explore photon-plasmon interactions, coherent perfect absorptions164, coupling165, slow light phenomena166, all-optical switches167 and lab-on-a-chip applications168,169,170, by engineering the interplay between the localized surface plasmon modes and propagating waveguide modes144,165,166,171,172. Furthermore, this waveguide-hybrid platform is particularly promising for largely miniaturized spectroscopic and sensing applications168,169,170. Taking the device in Fig. 3e as an instance168, a subtle change in surrounding index can be captured by the highly enhanced field in the narrow gap between the gold antennas, resulting in a shift in plasmonic resonance harvested by the waveguide. Compared with conventional bulky free-space optical setups, plasmon resonance can be conveniently measured in this antennas-loaded dielectric waveguides for surface-enhanced infrared absorption spectroscopy, with high coupling efficiency over 70% and compact footprint168 around 2 µm2.

### Phase-gradient nanoantennas array on waveguide

Besides elementary interference model, next we discuss how to apply metasurface toolbox to design dielectric meta-waveguides for more sophisticated device functions with larger antenna arrays.

#### Guided modes to free-space light applications

Metasurface-decorated dielectric waveguides can efficiently out-couple waveguide signals to complex free-space light fields79,173. A guided wave-driven metasurface is schematically shown as the upper panel of Fig. 3f. The total phase shift of the extracted wave from waveguide-fed metasurfaces consists of two parts: (i) the abrupt and spatial-varying phase shift Δϕ(x) provided by each meta-unit21 at coordinate x, and (ii) phase βx accumulated from the propagation of guided waves, where β is the propagation constant35. Therefore, phase profile of the extracted wave along x direction can be formulated as below.

$$\phi \left( x \right) = \Delta \phi \left( x \right) + \beta x$$
(1)

By judiciously designing the meta-units to tailor Δϕ(x), diverse applications can be realized, such as LiDAR32,174, optical communications and display79,80,81. For instance, using the metal-dielectric-metal sandwiched antennas (for approximate 2π phase shift range) to from a phase-gradient metasurface atop of silicon waveguide, off-chip beam deflection and focusing can be achieved35. By arranging this metasurfaces on an active InGaAsP/InP microring waveguide to break the degeneracy of clockwise- and counterclockwise-propagating whispering gallery modes (as lower panels in Fig. 3f), a photonic integrated orbital angular momentum (OAM) laser is experimentally demonstrated35. In addition to waveguide top surfaces, subwavelength antennas or gratings-like structures can be implemented on waveguide sidewalls as well for structured light and OAM generations46,175,176.

#### Guided-mode conversions

As already discussed in the ‘General application scenarios’ section, the meta-structures resting on waveguide can bridge the wavevector mismatch between different modes to enable integrated mode convertors51. As is illustrated in Fig. 3g, when guided wave propagates against the unidirectional phase gradient $${{{\mathbf{k}}}}_{{{{\mathrm{eff}}}}} = {{{\mathrm{d}}}}\Phi {{{\mathrm{/d}}}}x$$ offered by the collective scattering effect from gradient metasurface, the total internal reflection angle decreases, which corresponds to conversion from low-order to high-order waveguide modes. In contrast, when light propagates along keff, its wavevector $$k_{{{{\mathrm{mode}}}}} = n_{{{{\mathrm{eff}}}}} \cdot 2\pi {{{\mathrm{/}}}}\lambda$$ picks up keff, leading to coupling from high-order to low-order modes51 (neff denotes effective mode index and λ is vacuum light wavelength).

If fundamental modes propagating along keff are injected, it will be coupled into surface waves and get absorbed by the plasmonic metasurfaces. However, light wave with opposite propagating direction can get passed with much lower loss while converting to high-order modes. This asymmetric power transfer facilitates broadband reciprocal optical diodes82. Optical neural networks can be also explored utilizing tunable gradient metasurface-based waveguide mode convertors. As shown in Fig. 3h, using Ge2Sb2Te5 (GST) phase-change materials as programmable metasurface waveguide mode convertor, the conversion of two waveguide spatial modes (TE0 and TE1 modes) can be precisely controlled with 64 distinguishable levels to encode the weight parameters in matrix-vector multiplication computation177. A prototypical optical convolutional neural network with 2 × 2 array of the GST convertors is experimentally demonstrated to perform image processing and recognition between handwriting digits “1” and “2”. The experimental recognition results are shown in the right panel of Fig. 3h177. Besides, invisibility cloaks and chip-integrated spectrometers can be also envisaged in similar platform67,93.

### Aperture antennas:

The waveguide-integrated plasmonic metasurfaces discussed above inherit Ohmic loss from metals150. Next, we discuss meta-waveguides with dielectric subwavelength architectures, which have lower optical loss and better CMOS-compatibility12,18. Specifically, the antennas can appear as air apertures defined on waveguides. Either fully- or partially etched aperture antennas can be designed on dielectric waveguides to facilitate mode conversion83,84,85,86,87,88,89 and efficient coupling applications38,39 using effective medium or coupled mode theory83.

An exemplary device structure is shown in the upper panel of Fig. 3i, where subwavelength features are partially etched atop of a rectangular silicon waveguide84 to create periodic index variations Δε(x, y, z) along the propagation direction and a graded index profile along the transverse direction. The periodic subwavelength structures along the propagation direction provide an additional momentum keff to enable phase-matched coupling to the desired output mode83, while the aperiodic transverse graded index profile enhances coupling strength by optimizing spatial modal overlap83. The lower panel of Fig. 3i illustrates the desired refractive index profile offered by the meta-waveguide for converting input TE00 mode to TE10 mode with high mode purity around 95% and high transmission of 88%84.

Besides shallowly etched structures83,84,85,88,89, dielectric waveguides with fully etched subwavelength apertures86,87,178 can also enable efficient mode conversions with simplified fabrication process. Figure 3j shows the SEM image of a fully etched meta-waveguide capable of converting TE00 mode to TE10 mode within an ultrashort length about 2.42 µm around λ = 1.55 µm87. Integrated multifunctional mode convertors can be also conceived by applying complex surface meta-structures that are also aperiodic in waveguide longitudinal direction89,178,179. For instance, the multimode convertor shown in Fig. 3k for simultaneously converting 3 different modes with low insertion loss and acceptable crosstalk89. Furthermore, polarization convertors85 and high-efficiency optical couplers (Fig. 3l)38,39 can be also designed using spatially penetrated waveguides.

#### Nonlinear mode conversions

Metasurfaces can also interface nonlinear mode conversions180,181 when incorporating on nonlinear waveguides, as already discussed in Section “General application scenarios”. Figure 3m shows the integrated lithium niobate (LN) waveguide patterned with dielectric gradient metasurface for phase-matching-free second harmonic (SH) generations52. As is shown in Fig. 3m, the generated SH signals TE00(2ω) from the pump TE00(ω) is coupled to high-order SH modes TEmn(2ω) and TMmn(2ω). They are then preserved and strengthened during propagation, as the inverse conversion from high-order SH modes to TE00(2ω) is prohibited due to phase mismatch52,154.

Simultaneous SH generation and radiation are also reported using grating metasurface-patterned LN slab waveguides155. By encoding the desired phase and amplitude information to the grating metasurfaces under phase-matching condition (Fig. 3n), the wavefront of the generated SH signals can be efficiently controlled for nonlinear beam-shaping functions like dual focusing and Airy beam generation155. Other nonlinear meta-waveguides94,95,182 also judiciously optimize nonlinear spatial overlap to significantly enhance nonlinear coupling efficiency95,183.

### Jones matrix model for waveguide-integrated metasurfaces

Next, we discuss geometric metasurface-patterned dielectric waveguides and then introduce a more general Jones matrix model for waveguides-integrated metasurfaces.

#### Geometric phase metasurface-on-waveguide

Geometric metasurface or Pancharatnam–Berry phase (PB) metasurface152,184 utilizes optical antennas with identical geometry but spatially varying rotation angle θ to tailor wavefront of circular polarizations. Meta-units with angular rotation distribution θ(x, y) can locally encode a dispersionless phase profile φ(x, y) to one certain circular polarization |σ 〉 and flip its handedness.

$$\varphi ^ - \left( {x,y} \right) = 2\theta \left( {x,y} \right)$$
(2)

The PB phase shift φ can be traced from the paths of polarization change on the Poincaré sphere (polarization state space)185. A PB phase metasurface acting like a half-wave plate will convert incident left-handed circular polarizations (LCP) $$\left| {{\it{\upsigma }}^ - } \right\rangle$$ to right-handed circular polarization (RCP) $$\left| {\sigma ^ + } \right\rangle$$ with designer phase modulations152: $$\left| {\sigma ^ - } \right\rangle \to {{{\mathrm{e}}}}^{{{{\mathrm{i}}}}2\theta \left( {x,y} \right)}\left| {\sigma ^ + } \right\rangle$$. In the meantime, the same structure will impart a conjugate phase profile φ+(x, y) = −2θ(x, y) to the orthogonal circular polarization $$\left| {\sigma ^ + } \right\rangle$$ with polarization conversion: $$\left| {\sigma ^ + } \right\rangle \to {{{\mathrm{e}}}}^{ - {{{\mathrm{i}}}}2\theta \left( {x,y} \right)}\left| {\sigma ^ - } \right\rangle$$. This attribute has been exploited to develop free-space applications such as dual-polarity metalens and holograms186.

To transfer geometric metasurface to integrated optics, gold and silicon antennas are patterned on silicon waveguides to realize integrated polarization sorters41. The gradually increased antenna rotation angle (with 30° step) creates opposite phase gradient between LCP and RCP light, leading to directional coupling of orthogonal circular polarizations to opposite directions41. Figure 3o shows the spin- and wavelength-selective demultiplexers42, with simulated coupling efficiency above 50% using silicon antennas-patterned silicon nitride waveguide.

#### Free-space Jones matrix model for metasurface

Despite the simple relationship between imparted phase φ and antenna rotation angle θ for convenient design, meta-waveguides discussed above inherit limitations from geometric metasurface. First, the design methods are only applicable for circular polarizations. Second, the optical fields that coupled into the waveguide are hybrid modes41,42, which impedes high-speed optical communications applications due to inter-mode dispersion. Another important category of metasurface exploring propagation phase, where antenna orientations θ are fixed but the width lx and height ly of each antenna cell are individually engineered to tailor the eigen-phases φx and φy encoded to two orthogonal linear polarizations152: $$\left| {{{\boldsymbol{x}}}} \right\rangle \to {{{\mathrm{e}}}}^{{{{\mathrm{i}}}}\varphi _x}\left| {{{\boldsymbol{x}}}} \right\rangle$$ and $$\left| {{{\boldsymbol{y}}}} \right\rangle \to {{{\mathrm{e}}}}^{{{{\mathrm{i}}}}\varphi _y}\left| {{{\boldsymbol{y}}}} \right\rangle$$. Propagation phase stems from form birefringence43,151, as the dielectric antennas acting as truncated small waveguides have different effective mode indices under different incident polarizations. This leads to different accumulated phase retardations when light passing through the antennas with different geometry.

Jones matrix metasurface model combines both geometric phase and propagation phase151,152. A periodic metasurface resembling a birefringent waveplate can be described by a Jones matrix J (connecting two input and output polarization vectors as $${{{\mathbf{J}}}}\left| \lambda \right\rangle = \left| \kappa \right\rangle$$ with two operation scenarios, as illustrated in Fig. 3p:

1. (i)

Impart two independent and arbitrary phase profiles φ+(m, n) and φ(m, n) to an arbitrary pair of orthogonal elliptical

polarizations $$\left| {\lambda ^ + } \right\rangle$$ and $$\left| {\lambda ^ - } \right\rangle$$ respectively151,152, where (m, n) denotes different antenna pixels43. The unitary and symmetric matrix nature of J guarantees the following two mappings simultaneously152,

$${{{\mathbf{J}}}}\left( {m,n} \right)\left| {{{{\mathbf{\lambda }}}}^ {+} \rangle = {{{\mathrm{e}}}}^{{{{\mathrm{i}}}}\varphi ^ + \left( {m,n} \right)}} \right|\left( {{{{\mathbf{\lambda }}}}^ + } \right)^ \ast \rangle \,{{{\text{ and }}}}\,{{{\mathbf{J}}}}\left( {m,n} \right)\left| {{{{\mathbf{\lambda }}}}^{-} \rangle= {{{\mathrm{e}}}}^{{{{\mathrm{i}}}}\varphi ^ - \left( {m,n} \right)}} \right|\left( {{{{\mathbf{\lambda }}}}^ - } \right)^ \ast \rangle$$
(3)

where superscript * represents complex conjugate. Here, the two independent phase profiles φ+(m, n) and φ(m, n) can be arbitrarily assigned, but the output polarizations $$\left| {\kappa ^ + } \right\rangle$$ and $$\left| {\kappa ^ - } \right\rangle$$ are fixed (as left panels of Fig. 3p): $$\left| {\kappa ^ + } \right\rangle = \left| {\left( {\lambda ^ + } \right)^ \ast } \right\rangle$$ and $$\left| {{{{\mathbf{\kappa }}}}^ - } \right\rangle = \left| {\left( {{{{\mathbf{\lambda }}}}^ - } \right)^ \ast } \right\rangle$$, denoting preserved polarization ellipse but flipped handedness. The desired Jones matrix J(m, n) can thus be solved to determine antenna structure151.

2. (ii)

Complete control over both the phase φ+(m, n) and output polarization $$\left| {\kappa ^ + } \right\rangle$$ of one certain incident polarization $$\left| {\lambda ^ + } \right\rangle$$, while the encoded phase φ(m, n) for its orthogonal polarization $$\left| {\lambda ^ - } \right\rangle$$ is not configurable151 (right panels of Fig. 3p).

#### Implementation on waveguide

Integrating the Jones matrix metasurface model on dielectric waveguides can propel integrated optics into new heights by offering a more general design method for versatile multiplexers for arbitrary polarizations and complete mode control over coupled lights. By assigning opposite phase gradients to two orthogonal incident polarizations $$\left| {\lambda ^ + } \right\rangle$$ and $$\left| {\lambda ^ - } \right\rangle$$ under operation scenario (i), integrated polarization demultiplexers can be devised with high directivity over 20 dB for arbitrary incident elliptical polarizations44 (lower panel of Fig. 3p). Furthermore, by optimizing antenna geometries at multiple wavelengths43,44, dispersion-managed wavelength demultiplexers or ultrabroadband directional couplers can be also conceived.

Notably, when transferring dielectric meta-units designed by Jones matrix method151,152 to waveguides, spatial modal overlap between antenna scattering nearfield and target waveguide mode for excitation requires further attention51. For instance, to selectively excite TEm,n mode, m + 1 rows of antennas are required44,187. The mode index n is then controlled by properly engineered phase-gradient Δφ/d of the metasurface by combining generalized Snell’s law with phase-matching condition21,

$$\left| {\Delta \varphi {{{\mathrm{/}}}}d} \right| = n_{{{{\mathrm{eff}}}}} \cdot k_0$$
(4)

where Δφ and d are the phase difference and distance between adjacent antennas respectively, neff represents the effective index of the target waveguide mode to excite, k0 is wavevector. As depicted in Fig. 3q, specific high-order mode of interest can be exclusively launched by the mode-configurable coupler with high purity187 over 90% under operation scenario (ii). On-chip OAM generators with configurable topological charge is also proposed using mode mixing method44,187.

### Holographic gratings and hybrid subwavelength surface structures

For chip-scale structured light generations, subwavelength holographic gratings are applied on top of silicon waveguides45,77,78,188,189 to generate surface-emitting vortex beams carrying OAM with specific topological charge77 $$\ell$$. To design such device, the holographic grating can be retrieved by interfering the target OAM mode $${{{\mathbf{E}}}}_{{{{\mathrm{OAM}}}}} = {{{\mathbf{A}}}} \cdot \exp \left( {{{{\mathrm{i}}}}\ell \theta } \right)$$ and approximated in-plane waveguide field $${{{\mathbf{E}}}}_{{{{\mathrm{waveguide}}}}} = {{{\mathbf{B}}}} \cdot \exp \left( {{{{\mathrm{i}}}}k_{{{{\mathrm{mode}}}}} \cdot x} \right)$$ as45

$${{{\mathrm{G}}}}_{{{{\mathrm{fork}}}}} = \left| {{{{\mathbf{E}}}}_{{{{\mathrm{OAM}}}}} + {{{\mathbf{E}}}}_{{{{\mathrm{waveguide}}}}}} \right|^2$$
(5)

where A and B are amplitudes, θ is azimuthal angle and kmode denotes the propagation constant of the waveguide mode propagating along x direction. Figure 3r shows the proposed broadband OAM generator using two superposed subwavelength holographic forks atop of a silicon waveguide with a compact footprint45 of 3.6 × 3.6 µm2, where Gfork is converted to binary phase hologram considering current fabrication conditions. Similar subwavelength holographic surface architectures are leveraged to realize waveguide-integrated holography as well (Fig. 3s)80,81.

Besides, inspired from the integrated trench metalens and one-dimensional transmit-array190, on-chip wavefront shaping and mathematical optical signal transformations are demonstrated using standard silicon-on-insulator (SOI) platform with subwavelength metalens apertures53,191,192. Metamaterials are also saddled on waveguides for various integrated photonics applications such as optical modulation, coupling and sensing193,194.

### Subwavelength grating waveguides

The meta-waveguides discussed in previous subsections are featured by subwavelength patterns on waveguide top surfaces, as summarized by Fig. 3. In the following two subsections, we focus forward-designed metamaterial waveguides (see Fig. 4), which are devised mainly by effective medium theory or transformation optics16,17,18,19.

Among them, subwavelength grating (SWG) waveguides exhibit promising potentiality to revolutionize conventional photonic integrated circuits12. Given that this topic is already reviewed in previous literatures12,13,14, here SWG waveguides as a specific member of dielectric meta-waveguides family are only very briefly mentioned to keep this review intact.

A typical structure for SWG waveguide is illustrated as Fig. 4a, where periodic silicon segments with subwavelength spacing Λ forms the waveguide core12. When is Λ much smaller than the half-wavelength of the guided light, localized Floquet- Bloch modes are supported without scattering loss in propagation14. Distinctive from Bragg and conventional diffraction gratings, the SWG structure instead behaving as a homogeneous uniaxial crystal195 controlled by grating structures and duty cycle. ‘Bridge’ elements are commonly applied for the low-loss transition between SWG and homogeneous waveguide parts.

Compact and broadband passive phase shifters56 and mode convertors196,197 are demonstrated as Fig. 4b, c respectively, via engineering the dispersion and anisotropy of the SWG metamaterial waveguides. Polarization beam split- ters198,199,200,201, polarizers202, SWG microring-based biosensors90,91 and high-efficiency fiber-to-chip optical couplers203,204 are also explored, with typical device structures shown in Fig. 4d–f.

### Other forward-designed metamaterial waveguides

#### Integrated photonics applications

Figure 4g shows the SEM image of an on-chip silicon polarizer with broad bandwidth exceeding 415 nm and high polarization extinction ratio over 20 dB at telecommunication band205, using a 180°-bend silicon waveguide coupled with SWG metamaterial claddings. Moreover, SWG ‘claddings’ can be exploited to realize exception coupling and suppress evanescent wave as well (see Fig. 4h, i)54,55, for higher integration density and crosstalk mitigation. Optical beams focusing and transformations for guided waves signals are also reported206,207.

#### Exotic optical physics

Metamaterial waveguides are ideal platforms for exploring exotic physical phenomena to theoretically enable light slowing and trapping131, such as the axially varying waveguiding heterostructure with a metamaterial core in negative refractive index (Fig. 4j). Although the pace of photons may not be stopped in realistic metamaterial waveguides with appreciable bandwidth132, abundant optical physics for light-matter interactions can still be explored, with potential applications in chip-integrated optical signal processing and communications133,208.

Leveraging transformation optics (TO)19,209, integrated metamaterial waveguides can be conceived129,135,210,211,212,213. Figure 4k illustrates the schematic for designing a low-crosstalk multimode waveguide bend with feasible fabrication constraints using TO135, where darker color indicates higher index. Compared with conventional bending waveguides, significant inter-mode coupling mitigation (over 14 dB) in this TO waveguide is experimentally verified135. Waveguide tapers, connectors and beam expanders are also designed under TO theory129,210,211.

Moreover, metamaterial waveguides with exotic refractive indices are also an active field of research214. For instance, theoretical perfect waveguide bending can be realized in zero-index metamaterials134,215. Figure 4l illustrates the on-chip metamaterial waveguide based on standard SOI platform with a refractive index approaching zero216, where phase-matching free light propagation is directly observed. Other exotic optical phenomena are also explored at optical and lower frequencies, such as strong field enhancements in epsilon-near-zero materials217,218, magnified light absorption (Fig. 4m)219,220, enhanced spin Hall effect and broadband mode conversions130,221,222,223, to name a few.

#### Quantum applications

Recently, quantum supremacy is demonstrated using programmable superconducting processors for qubits224, where the computation time can be dramatically accelerated from over millenniums to minutes compared with classical processors under certain calculation scenarios. In the meantime, computational advantages using quantum photonics are also reported6. In contrast to conventional structures, metamaterial waveguides can enable new degrees of freedom in controlling qubits225,226,227. Figure 4n shows a superconducting metamaterial waveguides to tailor the transition lifetimes of qubits225, where metastable qubit states with ultralong lifetime (about 24-fold enhancement) and short-lived states are observed and selectively tuned.

## Optical meta-fibers

Optical fibers are another well-established platform to guide propagating electromagnetic modes. Integrating meta-structures with fibers gives rise to optical meta-fibers as an important member of meta-waveguide family. Distinctive from dielectric nanophotonic waveguides for various on-chip applications, optical fibers are widely used for massive long-haul optical communications and versatile interconnects that interface chips to exterior systems2,3. Moreover, their flexibility, biocompatibility, and mechanical robustness have made optical fibers with small cross-section and extreme aspect ratio as excellent candidates for remote, in situ and in vivo applications228,229 beyond photonic integrated circuits, such as biochemical sensors and endoscopic optical imaging230.

However, conventional optical fibers are limited by silica material properties (in terms of transparency window, cut-off frequency and nonlinearity) and the cylindrical waveguide geometry pervasive in fiber drawling process229. Compared with microfibers devoid of subwavelength features230, optical fibers employing subwavelength meta-structures allow for much powerful control over light attributes, including phase, polarization, amplitude, dispersion, and optical impedance229. A novel technological roadmap (lab-on-fiber) was thus launched to develop a novel class of all fiber devices and components, by judiciously integrating functional structures and materials onto optical fiber substrates at the micro- and nano-scale105,231,232,233. This vison is expected to provide the foundational basis to enlarge the functionalities pertaining to optical fiber technology towards ‘plug-and-play’ platforms to be exploited in many strategic applications, ranging from optical processing and computing to environmental monitoring, life science, safety and security75,234.

In general, meta-structures can be designed on the flat tip of optical fibers74,75,76,96,97,98,99,100,101,102,232,235 to enable ‘meta-tips’, where the fiber facet acts as a unique light-coupled microscopic substrate. Alternatively, subwavelength architectures can be also devised along the fibers for exotic waveguiding or to interact with evanescent optical fields to enable metamaterial fibers236,237,238,239,240,241,242,243,244,245,246,247.

In this section we discuss optical fiber ‘meta-tips’ (fiber facet-integrated meta-structures) and ‘meta-fibers’ (intra-fiber subwavelength features), showcasing representative applications and corresponding fiber fabrication technologies.

### Optical fiber meta-tips

As highlighted in past review papers22,229, the judicious integration of metasurfaces to optical fibers could be a game-changing direction for next-generation fiber-optic devices and components to disruptively expand their conventional functionalities. As is illustrated in Fig. 5a, metasurfaces would, indeed, open unprecedented paths to manipulate light using optical fibers to significantly accelerate relevant developments such as active beam profilers, spatial light modulators and fiber-optic tweezers, just to name a few231. Moreover, translating flat optics onto unconventional substrates, as the case of optical fibers, would allow the creation of lab-on-fiber assisted platforms with extraordinary capabilities in biomedical imaging, scanning near-field optical microscopy, and single-molecule detection103,106,107,108,109,110,248,249,250,251,252,253.

Following this intriguing suggestion, one of the important milestones as ‘optical fiber meta-tip’ was first demonstrated by Principe et al.75, with extraordinary light manipulation capability. The proof-of-concept was realized via focused-ion-beam (FIB) milling of rectangular nanoantennas arrays on the gold-coated termination of a single-mode optical fiber. The metasurface was designed to display a linear-phase profile to split an impinging light beam into an ordinary polarized component, and an anomalous extraordinary component (with different polarization state) for beam steering, as shown in Fig. 5b. By judiciously designing phase-gradient metasurface on fiber tips, the same group later reported the efficient excitation of plasmonic surface waves with unprecedented localization levels for developing all fiber optrodes with unparalleled sensing capability for life science applications (Fig. 5c)76. A phase-gradient plasmonic metasurface-based device was demonstrated to detect biomolecular interactions, with a sensitivity more than two orders of magnitude higher than that of a gradient-free counterpart.

The alluring potential of integrating meta-structures on optical fibers are soon recognized by researchers, followed by massive emerging optical fiber meta-components and meta-devices for various applications. As shown in Fig. 5d, a high- quality metamaterial dispersive gratings is integrated on fiber facet with the quality factor exceeding 300 to efficiently tailor light dispersion107. Lab-on-fiber nanoprobes using in plane integrated distributed Bragg reflectors on the end-face of a single-mode optical fiber is also reported, enabling the excitation of two spatially separated high-Q resonance modes (Fig. 5e)254. Figure 5f shows the first focusing optical meta-tip made by directly patterning a gold meta-lens on the facet of a photonic crystal fiber100. Superfocusing is achieved at 1550 nm wavelength with maximum enhanced optical intensities reaching 234%.

This concept could also have an enormous impact on next-generation fiber-optic imaging systems, especially in life science applications231, with exceptional spatial resolutions and miniaturization levels255. The reliable integration of metalenses could considerably overcome traditional difficulties associated with optical aberrations, alleviating also the well-known trade-off between the transverse spatial resolution and depth of focus, which significantly limits the scope of optical imaging in precision medicine. Optical fiber meta-tips coupled with optical coherence tomography (OCT) tools may provide the key asset for advanced nano-endoscopes, for in vivo imaging with spatial resolutions that are hardly possible with conventional platforms. Figure 5g shows a novel matalens-assisted endoscopic OCT platform that can be easily integrated in needles and catheters to achieve near-diffraction-limited imaging through negating non-chromatic aberrations74. The proposed tool was validated in case of endoscopic imaging of human lung specimens and sheep airways (shown in Fig. 5g inset), demonstrating superiority as compared to commercial OCT endoscopes featuring to improve and enlarge the clinical utility of OCT platforms.

The extraordinary capability of metasurfaces to control light at nanoscale would also open potential disruptive developments in optical signal processing applications. As illustrated in Fig. 5h, coherently controlled absorption in a fully fiberized and packaged switching meta-device was recently reported96. An optical fiber meta-tip platform was conceived demonstrating logical functions (XOR, NOT and AND) at wavelengths between 1530 and 1565 nm. The proposed metadevice has been successfully tested at up to 40 gigabits per second, opening new opportunities for 100 THz bandwidth single-photon operation with potential impact in quantum information networks97,98. Meanwhile, fiber facet-integrated meta-structures are also exploited for structured light generations and beam shaping256,257,258,259. Figure 5i sketches the SEM images of the meta-facet fiber for generating OAM beam with topological charge $$\ell = + 1$$257.

These valuable examples highlight future potentials of lab-on-fiber technology, optical communications, and related optical fiber meta-tips to drive a new technological revolution in optical fiber technology, for many realistic applications with unrivaled advantages in terms of functionalities, miniaturization, power consumption and overall performances. Next development in this domain may also take advantage from the advances in integrating functional materials onto optical fibers with a full spatial control at nanoscale. It is now possible to conceive and create any arbitrary 2D and 3D micro and nanostructure on the facet of optical fibers, as demonstrated by the smallest micro-house never realized on the optical fiber termination using robotics nano-factory and origami techniques260 (see Fig. 5j).

### Metamaterial fibers

Innovations can be also made beyond fiber facets. Compared with optical meta-tips, metamaterial fibers provide a broader stage to deploy meta-structures to tailor the propagation and dispersion of fiber modes, yet they also demand more stringent quality control over the desired subwavelength structures along the entire fiber229. For meta-fibers with complex sidewall structures, Fig. 6a shows the SEM image of a microfiber Bragg gratings fabricated by FIB milling working around 1.55 µm communication wavelengths, with 576 nm pitch and 100 nm-depth grooves236. This fiber device with subwavelength features exhibit high transmission dip around 15 dB236 and high sensitivity for refractive index and temperature237,261,262,263. Simple nanostructures like a single gold sphere can be also applied to airlclad nanofibers to explore spin-orbit interaction of light and controlled directional coupling238 (Fig. 6b). Intra-fiber metasurfaces are proposed for mode conversion239. Other sophisticated fiber sidewall textures may be envisaged by adapting emerging fiber fabrication technologies to optical fibers264,265,266.

Subwavelength structures that are uniform along fiber length direction can be manufactured by preform-based fiber drawing methods (detailed in next subsection)267,268,269. Metamaterial fibers244,268,269,270,271 with hollow240,241,242,243,272,273, metallic274,275, holey276 and multiple cores246,247,276,277 are reported. Figure 6c shows the cross-section of an anisotropic metamaterial fiber with radially distributed internal nanopores around the aluminum core supporting exotic modes274 to strongly modify the waveguiding fiber attributes. For instance, hollow-core fibers with hyperbolic metamaterial claddings can guide light beyond cut-off frequency for enhanced light coupling278 and low latency communications. Metamaterial cladding with various material combinations enables powerful dispersion control from ultraviolet240, infrared241,242 to terahertz (THz)243,278 and microwave frequencies279 to circumvent the structure and material property limits of conventional silica optical fibers280. The meta-fiber shown in Fig. 6d exploits surface plasmon and classical fiber waveguiding240. Figure 6e sketches a single-mode, single-polarization hollow-core fiber with metal-dielectric hybrid metamaterial cladding for high-density integration of THz systems243.

Epsilon-near-zero (ENZ) materials such as indium tin oxide (ITO)-embedded hollow step index fiber is also proposed as Fig. 6f245, with strong field enhancement in the subwavelength ITO shell and potential applications in sensing, nonlinear optics and enhanced quantum emissions281. Figure 6g depicts the cross-section of a THz metamaterial fiber with hexagonally arranged subwavelength indium wires for sub-diffraction imaging246 produced by fiber drawing of the preform. This metamaterial fiber inspired by hyperlenses with indefinite permittivity tensor can collect high spatial frequencies over optically long distances to enhance imaging resolution from THz277 to infrared frequencies282.

### Fabrication technologies

The waveguiding flexibility, high aspect ratio and microscopic cross-section of optical meta-fibers provide unique advantages, but simultaneously challenge conventional nanofabrication technologies developed for large planner substrates228. Nevertheless, the maturity of diverse fiber fabrication techniques has made optical fiber meta-tips and metamaterial fibers one step closer from lab demonstrations to potential market products. Figure 7a summarizes currently available approaches to fabricate meta-structured optical fibers76,97,104,228,229,283,284,285,286,287,288,289,290,291,292. For optical meta-tips, ordinary photo- and electron beam lithography generally confront challenges in sample mounting and uniform resist coating for the small fiber facet229,287. Instead, nanotransfer103,110,229,288,289 and FIB milling75,76,96 can be applied to define exquisite subwavelength meta-patterns at fiber facets. Laser writing290,293 and nanoimprinting283,284,291,294 generally have comparatively lower resolution but higher yield. For metamaterial fibers, preform-based fiber drawing methods are most promising for mass-production. Other approaches such as FIB milling and nanoimprinting can produce subwavelength architectures at fiber sidewalls236,237,266. Different fabrication technologies for meta-structured fibers are briefly compared as the following.

#### Nano-transfer

Nanotransfer is a straightforward method that transports as-fabricated meta-structures to fiber facets to circumvent the difficulty of direct nanofabrication on small fiber tips, as shown in Fig. 7b. The transferred meta-structures inherit the high resolution from conventional nanofabrication methods. However, most transfer processes have relatively low throughput and defects can be induced during the transfer. Figure 7b1 illustrates a wet-transferred meta-tip working as a flexible passband filter288 using sacrificial layer and water bath scooping. The meta-structures can be also fabricated by ‘nanoskiving’ techniques in an inexpensive manner229,295 for wet-transferred to fiber facets. For dry transfer, a ‘decal transfer’ technique using sacrificial films (Fig. 7b2)296,297, and a cost-effective template stripping method using ultraviolet (UV) light-curable hybrid polymer (Fig. 7b3)289 are proposed. Another ‘stamping-like’ method is sketched as Fig. 7b4103, where the plasmonic metasurface is mounted at fiber facet via epoxy298. Direct mechanical transfer is also reported299 (Fig. 7b5). Figure 7b6 depicts a template transfer approach300 to produce high quality and robust plasmonic structures for real-time biosensing110,301.

#### Direct laser writing

Leveraging femtosecond laser pulses and two-photon polymerization302,303, direct laser writing can be applied to define micro- structures at fiber facets. Despite submicrometer resolution has been achieved by this multiphoton lithography technology302,304, currently demonstrated structures on fiber facets generally have feature sizes bigger than light wavelength290,292,293,305,306,307. Nevertheless, fiber tips-integrated subwavelength meta-structures can still be envisaged with the steady trend of resolution improvement. Fig. 7c1 shows the SEM image of the laser-written micro-spiral zone plate at fiber facet for OAM generation290.

Although currently available material library for two-photon polymerization is slightly restrained, this approach, more importantly, can fabricate three-dimensional (3D) complex structures that are hardly accessible by conventional top-down lithography292,305. Polymerization only takes place in the vicinity of the laser focal spot and the location of the solidified voxels can thus be manipulated much freely302. Beside fiber facets, intra- waveguide sophisticated structures may be also defined by this technology308,309. Optomechanical307 and bionic structures303 can be fabricated as well. Figure 7c2 shows the colored SEM image of a compact triplet lens objective attached to an optical fiber305 for miniaturized endoscopes and imaging applications.

#### Focused-ion beam milling

FIB milling is another technology to produce intricate 3D nanostructures with high fabrication resolution. Precise subwavelength features can be defined at fiber facets75,76,96 and fiber sidewall236,237,261,262,263,266. Figure 7d1 shows the SEM image of an optical fiber meta-tip76, where a thin gold layer is first deposited on fiber facet and the metasurfaces are then written by FIB milling. Besides, elegant fiber sidewall architectures are also demonstrated (see Fig. 7d2, d3)261,262. Albeit to the advantages in fabrication accuracy and 3D nanostructuring, FIB has a slow throughput that precludes mass production. The optical response of the engraved nanostructures may be influenced by the gallium ions doping during the process229.

#### Nanoimprinting

Nanoimprint lithography is a high-throughput and cost-effective technology that utilizes molds to replicate predefined stamp topographies138. This method can produce various nanopatterns on fiber facets284,291,294,310,311,312 and fiber sidewall in parallel266. Figure 7e1 illustrates a nanoimprint process to transfer subwavelength period metal grating from the mold to fiber facet310 with a sub-15 nm feature using PDMS and SU-8 for chip-scale probing and testing, where U-grooves are optimized to settle the alignment issue291. However, for complex metasurface-on-fiber structures, the imprinting and alignment accuracy may still need amelioration. 3D nanostructures are also accessible284,311, such as the 3D beam splitter shown in Fig. 7e3311. Meanwhile, when combined with fiber drawling, nanoimprinting may also produce surface structures at fiber sidewall using roller molds (Fig. 7e4)266. Figure 7e5 depicts the fabricated patterns for resolution test. For its potential applications to optical fibers, the imprinted nanostructures have to stay close to the fiber core in order to evanescently interfere with the optical modes.

#### Electron beam and photolithography

Electron-beam lithography (EBL) is a time-consuming process but can produce high-quality subwavelength structures with high resolution. To apply EBL to define patterns on the tiny facets of optical fibers, modifications on apparatus are quired313, as conventional spin coating and exposure platforms are designed for big wafers. Figure 7f illustrate the exemplary procedures to pattern on facets using EBL (left panels), and the SEM image of the fabricated structures after ion-beam etching (right panel)101. ‘Dip and vibration’ coating technique314 is proposed for better resist coating uniformity. Customized rotating chuck and exposure holders are also explored to obtain better lithographic patterns on fiber facets313.

Photolithography has higher yield when working for large substrates, yet it also encounters challenges on resist coating, sample mounting and alignment of the optical fiber315. Figure 7g shows an ‘align-and-shine’ photolithography process that can transfer microstructures from the mask fiber to target fibers286. Interference lithography is also applied to produce periodic nanopillars array on fiber facets285 (see Fig. 7g1). This approach has good resolution with simple experimental setup, but it is not applicable for metasurface with aperiodic or arbitrarily designed antennas. Furthermore, photolithography on fiber sidewall is also explored (as Fig. 7g2)316,317.

#### Preform-based fiber drawing

Fiber drawing of specific preforms can produce various metamaterial fibers with designed cross-sectional patterns extending over long lengths267,268,269,277, showing a promising inroad to high volume production. As is illustrated in Fig. 7h, a macroscopic preform assembled by stacking indium wires is applied to hot furnace and thermally drawn to produce microscopic features in the thin fiber246. During the process, the feature dimensions from the original preform can be reduced by order of magnitude268. The preform can be fabricated by assembly, drilling and 3D printing264,318,319. Currently demonstrated metamaterial fibers made by single drawling are mainly applied for THz and mid-infrared frequencies due to fabrication resolution246,247,282. Figure 7i shows metamaterial fiber preform with tin/soda-lime wire arrays and an exemplary transition region after drawling282.

To further produce structured fibers with subwavelength features, iterative drawling can be applied320,321 (see Figs. 7j, k1). After successive drawling, the diameter of the intra-fiber nanowires can be reduced from hundreds of microns to about 15 nm with good radial and axial uniformity320. As shown in Fig. 7k2, the drawn fibers can be further assembled into different metamaterial preforms with various spatial structure distributions for further thermal size reduction321. Despite these demonstrations are non-optical fibers, this technology may be further adapted to fabricate multifunctional optical metamaterial fibers by improving optical loss and structural quality.

## Plasmonic meta-devices for controlling surface waves

Distinctive from previous chapters discussing propagating modes in meta-structured dielectric waveguides and optical fibers, in this section we will elaborate on the excitation and manipulation of surface wave (SW) modes using plasmonic meta-structures.

As a comparison, the optical modes in conventional fibers and waveguides are electromagnetic (EM) guided modes based on total internal reflection. The size of these optical elements is thus still wavelength scale constrained by diffraction limit, which severely hinders the high demand on device miniaturization. In contrast, EM SWs, including surface plasmon polaritons (SPPs) and their low-frequency counterpart spoof SPPs, are also eigen EM modes but instead highly confined at the material interface, which can find numerous applications such as super-resolution imaging209, enhanced light-matter interactions322, high-integration optical circuits323,324,325, bio-326 and chemical-sensing327, thanks to their deep-subwavelength and local-field enhancement characteristics. While natural SPPs only exist in optical frequency, the so-called spoof SPP modes are created in terahertz and microwave frequencies via patterning the highly conducting metal surfaces with subwavelength structures328,329. Recently, various meta-devices, including meta-coupler, meta-waveguide and circuits, are proposed to control SWs, providing us versatile possibilities for future on-chip optoelectronic applications150. Next, we will introduce the recent advances in SPP excitation and wavefront manipulations using these meta-devices.

### Excitations of surface waves using plasmonic meta-waveguide couplers

To utilize the novel properties of SW modes, the first step is to excite them efficiently. Unfortunately, because of momentum mismatch, the impinging free-space light cannot be directly coupled to SWs in general. To this end, various conventional optical devices were proposed, including prism coupler and grating coupler330,331. However, these elements are either too bulky and/or low-efficiency, which greatly hampers their further applications in integrated photonics332.

In 2012, Sun et al. proposed a new strategy to couple free-space propagating waves (PWs) to localized SWs based on a gradient metasurface in metal-insulator-metal (MIM) configuration70. As the meta-atoms are illuminated by EM waves, anti-parallel currents will be induced inside two metallic layers, creating a magnetic resonance. Via carefully tuning their local geometric parameters, such metasurface can provide a linearly changed reflection phase with its gradient denoted as $$\xi = {{{\mathrm{d}}}}\varphi {{{\mathrm{/}}}}dx$$. While the reflection phase changes slowly (i.e., ξ < k0, and here k0 denote the total wavevector of EM waves in vacuum), the metasurface will deflect the impinging wave to non-specular direction. Nevertheless, as ξ is large than k0, the metasurface cannot construct a far-field EM wave with an equal-phase plane, thus achieving PW-SW conversion, as shown in Fig. 8a. Both of the anomalous reflection and SW conversion effects can be described by the following equation:

$$k_x^r = k_0{{{\mathrm{sin}}}}\theta _i + \xi$$
(6)

where $$k_x^r$$ represents the parallel wavevector of reflection beam and θi is the incident angle of impinging waves. It implies that the gradient metasurface will introduce an additional wavevector to input EM waves, addressing the momentum mismatch issue between PWs and SWs. Considering that the generated SWs on such inhomogeneous metasurface are not eigen EM modes, the authors further constructed a SPP meta-coupler via connecting a mushroom structure with the metasurface that can guide out the “driven” SWs to eigen spoof SPPs. Such generic idea was soon realized based on other meta-structures working at various frequency regimes115,116,333. For instance, a polarization-dependent directional SPP meta-coupler working at telecom wavelengths was experimentally demonstrated115. The building block consists of a gold nanopatch and a thick gold film that are separated by a 50 nm-thick glass spacer. Via carefully designing both widths of atop nanopatches, they constructed two-dimensional gradient-phase metasurface that can couple input x- or y- polarized PWs to eigen SPPs propagating along two orthogonal directions, as depicted in Fig. 8b.

Apart from the resonance-based metasurfaces, Pancharatnam–Berry (PB) phase metasurfaces are also exploited to achieve helicity-dependent directional SPP excitations. As shown in Fig. 8c, a transmissive metasurface is composed by the elongated aperture array with constantly varied orientation angles113. Except for Bragg reciprocal wavevector, such metasurface can also provide an additional geometric phase based wavevector for impinging light, achieving the helicity-dependent directional SPP excitations at near-infrared wavelengths. While the handedness of input CP light is flipped, the excitation direction of SPP will be switched to opposite direction. In a following work, a MIM typed PB meta-coupler is further demonstrated to couple SPPs with their amplitude and phase independently modulated by the polarization state and polarization angle of incoming light114. However, the early PB meta-couplers usually suffer from low-efficiency issue. To address such challenge, a new scheme is reported to solve two fundamental issues widely existing in previous studies334. First, via carefully optimizing the polarization conversion efficiency of the building block, the PB meta-coupler is able to convert impinging EM waves to driven SWs with nearly 100% efficiency. Second, the guided-out plasmonic metal should be carefully designed to support both TM and TE spoof SPP mode, solving the polarization mismatch issue between input CP wave and guided-out SPPs. A realistic meta-coupler working in the microwave regime is designed and fabricated, exhibiting a high spoof SPP excitation efficiency of over 80%.

While PB phase metasurfaces possess the advantages such as dispersionless phase response and easy design/fabrication, they still encounter a strong restriction of spin-correlated functionalities. To overcome this limitation, Mueller et al. proposed an approach to design a single metasurface carrying two arbitrary and independent phase distributions for the input lights of two orthogonal polarization states (including linear, circular and elliptical ones). The combination of propagation phase and PB geometric phase enables us to delink two functionalities of the metadevices for two orthogonal polarizations152. Various multifunctional meta-structures leveraging Jones matrix toolbox are demonstrated for achieving either SPP excitations or non-specular beam deflections modulated by the spin states of incoming CP lights118,335. A new design scheme was soon proposed to simultaneously achieve high efficiency surface plasmon excitations and wavefront controls based on a single meta-coupler119,120. Except for designing a linear-phase profile along x direction, which is mainly responsible for surface plasmon excitations, they also added a specific phase profile along y direction to simultaneously modulate the wavefronts of generated SPPs. Via combining both resonance phase and PB phase together, such meta-coupler can achieve either SPP focusing, or deflection effect shined by different spin-polarized EM waves (see Fig. 8d).

Except for the meta-couplers based on wavevector compensation scheme, narrow apertures are also widely utilized to launch SPP modes depending on diffraction effect336. In 2013, an aperture-based SPP meta-coupler was proposed112 to realize polarization-modulated directional SPP coupling. For a single aperture, it approximately behaves like an in-plane dipole source that can emit SPP waves along both sides but with a phase difference of π. As illustrated in Fig. 8e, such meta-coupler can enable constructive or destructive interference of SPP on two opposite sides via carefully designing the individual orientation angles and spatial separation of the aperture pairs. Such aperture-based metasurfaces can not only launch directional SPPs, but also tailor their wavefront simultaneously122,337,338. For instance, Fig. 8f depicts a plasmonic meta-slit that can achieve SPP focusing with its direction and focal length flexibly switched by the helicity of input CP light122. The meta-devices are composed of single or double arrays of subwavelength nano-slits with their orientation angles satisfying the parabolic distribution. To construct more complicated near-field patterns, the double-lined metasurface was adopted to modulate both the local amplitude and phase of generated SPP via controlling the tilted angles of nanoslit pair. Benefiting from these two degrees of freedoms, the plasmonic SPP caustic curves, Airy beam, and complex holography (Fig. 8g) are successfully demonstrated337.

### Manipulations of surface waves using plasmonic meta-waveguides

In the previous section, we have discussed how to employ meta-couplers to efficiently convert free-space light to near-field SPPs. For the future integration photonic applications, versatile on-chip manipulation the SPP beams is highly desirable, including SPP guidance, wavefront tailoring, and far-field emission.

#### Surface wave waveguides and circuits

Massive plasmonic meta-structures can be applied to construct meta-waveguides for SW transports and controls in deep- subwavelength scale, such as V-shaped grooves, hybrid nanowires, and plasmonic tapers339,340,341. In low-frequency counterpart, spoof SPP waveguides were also developed based on various structured surfaces328,342,343. In particular, an ultrathin and flexible comb-shaped plasmonic surfaces is reported to transport the conformal surface plasmons (CSPs) at the microwave regime344. As shown in Fig. 9a, such CSP modes can be bent, folded and even twisted with very low propagation and absorption losses on such surfaces with various curved configurations.

Considerable efforts have also been devoted to SPP based integrated optical circuits. In 2011, a simple double-nanowire network was proposed to perform compact logic operations, via illuminating the two nanowire terminals by excitation lasers of specific polarizations and relative phases. Based on the plasmonic interference effect, such plasmonic network can direct guided SPP waves to corresponding output, exhibiting various functionalities of beam splitting, routing and switching345. Soon, a binary NOR gate was experimentally verified in a plasmonic network of four-terminal nanowires via cascading OR and NOT gates346. Meanwhile, several high-integration all-optical logic gates were developed (including OR, NOT, XOR, and XNOR ones) based on plasmonic nanoslot waveguides, exhibiting high intensity contrast and lateral confinement (see Fig. 9b)347.

The local-field enhancement effect in plasmonic devices sheds light on many on-chip photonic applications, such as tunable sources, nonlinear microscopy, all-optical switching, etc.348,349,350,351. For instance, Li et al.348 experimentally demonstrated the second harmonic generation (SHG) in a plasmonic nanowire waveguide (Fig. 9c). The sample consists of a crystallized silver nanowire and a single layer of MoS2 with large second order susceptibility. Shined by the excitation laser, the excited SPP mode will propagate and then be reflected back inside the nanowire. For these two counter-propagating SPPs, the momentum matching condition is automatically addressed, thus facilitating the nonlinear effect inside such system.

#### Wavefront control of surface waves

While some basic optical elements (e.g., dielectric triangular prism and cylindric lens) are initially adopted to control the propagation of SPP, they usually possess bulky size and limited functionalities352. Relying on Bragg scattering effect, the concept of plasmonic crystals was proposed to realize SPP beam reflection, splitting, interference and demultiplexing347,353. To construct more complicated near-field wavefronts, the non-perfectly matched diffraction effect is employed to generate SPP Airy beam and experimentally demonstrate its non-diffracting and self-healing properties, as shown in Fig. 9d354. An inhomogeneous nanohole array is fabricated on the silver film to provide a nonlinear phase modulation for input SPP. In addition, various proposals such as transformation based devices355,356 and holographic plates357,358, also showcase diverse opportunities for on-chip SPP manipulations.

The rapid development of plasmonic meta-waveguides has also opened up a new direction for near-field manipulations. In 2013, Liu and Zhang proposed theoretically a plasmonic metasurface constituted by metallic subwavelength grating that can support SPP modes with non-trivial (including flat and hyperbolic) dispersion relations. Several alluring phenomena such as anomalous-diffraction, non-diffraction and negative refraction of SPPs were numerically demonstrated123. As illustrated in Fig. 9e, such concept of hyperbolic metasurfaces for surface plasmon controls was soon experimentally verified in visible regime124. Other than these bulky meta-deivces, Dong et al. proposed a new concept to design ultrathin gradient metawalls (i.e., near-field version of metasurfaces) to achieve the arbitrary wavefront tailoring of SPP, such as SPP anomalous reflection (Fig. 9f), near-field focusing and Bessel beam generations73,359. Such meta-wall is constituted by ultrathin gradient-index microstructures and a metallic plate that can totally reflect incoming SPPs with specific reflection phases profile, leading to the versatile wavefront reshaping of SPP. This idea was soon further developed to achieve one-dimensional SPP line wave excitations with a transmissive metawall (see Fig. 9g)71. This find may be applied in modulating on-chip photonic networks.

#### Out-of-plane emission of surface waves

For numerous photonic applications, we also need to decouple near-field SWs to far-field PWs in a desired manner. While encountering some discontinuous interface (e.g., bump, hole, groove), SW can be partially scattered to free space but with the radiation intensity and direction hardly controlled360. Periodic apertures are utilized for achieving directional out-of-plane emission of SPP361. As shown in Fig. 9h, holographic images can be created by incorporating the computer-generated hologram technology with bragg meta-grating to decouple SPP to free-space362. Other complex wavefronts, like accelerating Airy beam, focused beam, vortex beam are also suitable for this method363,364,365. In addition, Fig. 9i shows H-shaped gradient-phase metasurfaces to guide the spoof SPP modes on metallic strips to directional radiative waves127. According to the generalized Snell’s law, the emission direction can be flexibly modulated by the phase gradient of the metasurface. Further utilizing the dispersive nature of spoof SPP mode, a broadband frequency scanning antenna is created based on such gradient metasurface, exhibiting a continuous scanning range from 4.8° to 37.2° operating at 8.8–10.7 GHz128.

## Inverse-designed metamaterial waveguides

Forward-designed metamaterial discussed in previous sections generally starts from a well-established library of structure templates, followed by customized tuning with a handful of specific parameters based on applications. Despite the clear physical guidelines offered by these intuition-based strategies, the device structures are usually not fully optimal and do not consider the massive design space of irregular structures. Furthermore, some sophisticated functionalities such as ultracompact and multifunctional devices are hard to design using forward design methods15.

To settle these challenges, various numerical algorithms have been developed that allow a computer to design and optimize freeform metamaterial waveguide structures with dramatically boosted degrees of design freedom to maximize device performance. These computer algorithm-based inverse-design strategies offer a potentially effective and automated method to ‘design-by-specification’15,147, in a manner following computer aided design techniques that are well established in the integrated circuit design community.

Inverse-designed metamaterial waveguides can be classified into two categories: analog and digital62,67,68,69. To design a device, the design region is first specified and then discretized into M × N unit shapes, called “pixels”. The “pattern” is thus considered as a matrix of the pixels with permittivity distribution of [ε]M×N, where each pixel (i, j) has a permittivity of εi,j accordingly. Then, algorithms are employed to find an optimized permittivity distribution to fulfill figure-of-merit (FOM). As exemplified in the upper panel in Fig. 10a, analog metamaterial waveguides generally have relatively small pixels366,367,368 $$\left( { \ll\!\! \lambda {{{\mathrm{/}}}}10} \right)$$ to meet the perturbation theory369. Thus, the permittivity εi,j or its shape information can vary continuously to calculate the gradients information of the FOM (discretization is conducted later to produce binary devices)15. In contrast, pixel dimensions of a digital metamaterial waveguide are much larger (~λ/10) (lower panel in Fig. 10a)68,370, and the permittivity εi,j is restricted to binary distribution, e.g. corresponding to the logical “1” or “0” state.

In this section, we present explicit design tutorials and highlight representative applications using inverse-deigned metamaterial waveguide platforms. Diverse design algorithms are outlined and compared in terms of its versatility and applicability, including gradient-based, non-gradient-based and deep learning-inspired optimization methods.

### Analog metamaterial waveguides

Analog metamaterial waveguides using sufficiently small pixels may have tremendous degrees of design freedom. To manage and explore this high dimensional design space in a computationally tractable manner, two classes of gradient-based optimization methods are typically considered: density topology optimization (TO) and level-set method, as shown in upper panels of Fig. 11. These methods, based on the adjoint method, are computationally scalable to systems comprising thousands and even millions of pixels because the number of simulations required to calculate gradients is decoupled from the total number of pixels147,367, which vastly broadens the generality and computational efficiency of inverse design. Gradient-based adjoint methods were introduced into nanophotonics in the early 2000’s by researchers from the mechanical engineering371 and control theory372 communities, where the adjoint method was established.

### Design procedures

(i) The permittivity values εi,j of each pixel are randomized and relaxed to span a continuum of values. (ii) Then, gradient descent algorithms perturbatively adjust the permittivity values at each pixel in a manner that improves the FOM, producing an analog pattern (shown in the bottom left panel of Fig. 10b). (iii) The continuous permittivity values of each pixel obtained in Step (ii) are pushed towards binary values, where only two boundary values can be chosen, and the FOM is reevaluated. Iterations are thus formed by going back to Step (ii) and (iii) to produce a new binary pattern.

### Foundational results

Initial studies in the density topology optimization of nanophotonic devices are most linear devices, where the electromagnetic field satisfies the Hermitian condition. These demonstrations include, e.g. photonic crystal (PhC) waveguide Z-bend373 (Fig. 11a) and 90°-bend374 (Fig. 11b), where the optimized structure shows a significantly lower loss for a wider range of frequencies compared to forward designs. Density topology optimization has since been exploited to inversely design various waveguide devices with free-formed subwavelength features (Fig. 11c–f), including, for instance, high-performance 90° waveguide bend375, polarization beam splitter376, three-channel mode multiplexer377, and nanolens with structural integrity378. More recently, density topology optimization was used to tailor guided-mode phenomena in freeform metasurfaces, leading to high efficiency379,380,381 and multifunctional devices382,383. New diffraction phenomena are enabled as well by analyzing these complex mode scattering dynamics384.

In contrast to typical TO schemes, an “objective first method” was proposed to allow both electromagnetic filed and permittivity values to optimize individually385, which allows for non-zero physics residuals. Alternating directions method of multipliers (ADMM) algorithm were employed in the optimization process without using the adjoint method. A wider range of functional devices were investigated using the method58,386,387, including mode converter (Fig. 11g), hub (Fig. 11h), fiber coupler, optical diode, and polarization beam splitter, as well as multi-function device handling simultaneous focusing and wavelength demultiplexing (Fig. 11i).

For nonlinear devices, the electromagnetic field no longer satisfies the Hermitian condition and adjustments to the adjoint method are required. A 2D microcavity was designed for nonlinear frequency conversion by solving a linear adjoint equation with time-dependent material388 (Fig. 11j). Simplified nonlinear optimization was then explored based on a linear adjoint equation with time-invariant material366, followed by devising a compact photonic switch in Kerr nonlinear material (Fig. 11k).

### Design procedures

(i) The boundary of the material subjected to φ(r) = 0 is allowed to vary continuously. During the optimization process, the boundary is time dependent and can be expressed as the zero solution of the level-set function .

$$\psi \left( {{{{\mathbf{r}}}},t} \right) = \varphi \left( {{{\mathbf{r}}}} \right) + h\left( t \right)$$
(7)

The negative solution and positive solution of ψ(r, t) correspond to the regions filled with permittivity of two boundary values respectively. (ii) Then, the evolution of the boundary can be considered as a change of the level-set function with respect to t via gradients, as shown in the bottom right panel in Fig. 10b.

### Foundational developments

A shape derivative method was proposed based upon the adjoint variable method and perturbation theory in the early 2000’s389, which enables the gradient descent algorithm based on shape derivative method. Level-set method is mathematically distinct from the shape derivative method, but essentially equivalent. Initial studies in the level-set method involved bandgap engineering of PhC devices390, as shown in Fig. 11l. Later research validated the feasibility of using this method to devise metamaterial waveguides. By parameterizing the boundary of the material with the level-set method, a high-performance silicon Y-junction splitter was designed in an ultra-compact area367 (Fig. 11m). A general level-set method was proposed for subwavelength featured waveguides that directly incorporates fabrication constraints, especially for curvature constraints on device boundaries391. A 1 × 3 splitter, mode demultiplexer, directional coupler, wavelength demultiplexer, and cavity-waveguide coupler391,392 were later reported (Fig. 11n–r). Besides, level-set method was used to design nonlinear waveguides, such as PhC fibers with large nonlinear frequency-conversion efficiencies (Fig. 11s)393.

### Design procedures

(i) First, an optimized analog structure with gray permittivity is obtained using TO method. (ii) Then, the structure is binarized employed level-set method. The level-set method could theoretically prevent intermediate gray structures.

### Foundational developments

Figure 11t shows a wavelength demultiplexer with wide bandwidths designed by simultaneous TO and level-set method368. The method was also used to design general 1D grating couplers without any human input from start to finish (including a choice of initial condition)394, as depicted in Fig. 11w. Various functional integrated devices had also been demonstrated as Fig. 11u–z395,396,397,398,399, such as mode multiplexer and mode splitter, Fabry-Perot micro-resonator, non-reciprocal pulse router for chip-based LiDAR, laser-driven waveguide-integrated particle accelerator, and diamond photonic waveguides.

#### Fabrication constraints for gradient-based algorithms

Despite the benefits discussed above, such analog devices nevertheless encounter challenges in terms of fabrication constraints. The TO method results in intermediate “gray” structures, but only binary permittivity distribution can be efficiently implemented in experiments for most cases. A wide variety of proposals are reported to mitigate this issue of “gray” areas in structure, such as density filters400, sensitivity filters401, penalty functions402, artificial damping403, and temperature parameter trick404.

In level-set method, small features, such as narrow gaps or bridges, are usually inevitable. A typical approach is to exploit shape parameterizations that automatically satisfy the desired minimum feature constraint, known as a geometry projection method405. Another approach is to evaluate a dilated and eroded version of the device during the optimization process with improved fabrication robustness and length scale constraint406. Curvature limit techniques are also useful for the removal of narrow gaps391. To enforce hard design constraints, methods based on reparameterization have been developed in which geometric parameters are initially defined in a continuous latent space and then mapped onto a physical device407. This mathematical mapping is specified such that devices that violate a given physical constraint, such as minimum feature size, are not even considered. This method has the advantage not only of enforcing hard constraints, but also of simplifying the design space and improving the reliability of the optimization process through elimination of non-compliant device patterns.

### Digital metamaterial waveguides

Analog metamaterial waveguides utilize curvilinear device structures with high degrees of freedom but entail high fabrication resolution requirements. In contrast, digital patterns that utilize regular arrays of spatially coarse pixels can be designed and fabricated in an easier fashion. To date, most popular optimization methods for designing digital metamaterial waveguides fall into three classes: brute-force searching, heuristic, and adjoint methods, as shown in lower panels of Fig. 11.

### Design procedures

Take the direct-binary search (DBS) algorithm as an instance. (i) Fist is reversing the logical state of a randomly chosen or specifically designed pixel and evaluating the FOM. (ii) If FOM is improved, the new pixel state is kept and the next pixel is reversed; otherwise, the original state is recalled (Fig. 10c). (iii) One iteration ends when all the pixel states are inspected, and the final pattern is set as the initial one of the next iteration.

### Foundational developments

DBS algorithm is a representative brute-force searching method introduced from digital hologram communities408. Massive digital metamaterial waveguides were invigorated using DBS algorithm, such as free-space- to-waveguide coupler67 (Fig. 11a1), polarization beamsplitter68 (Fig. 11b1), waveguide cloak64 (Fig. 11c1), power divider409 (Fig. 11d1), nanolens410 (Fig. 11e1), and mode converter411 (Fig. 11f1), optical diode66, 180° waveguide bend65, polarization rotator63, active optical switch412, polarization splitter-rotator413.

In Fig. 11g1, a photonic-crystal (PhC)-like metamaterial structure was proposed for 3 dB power divider that exploits partial-filling hole pixels to suppress the etch depth fluctuations caused by the reactive ion etching (RIE) lag414 and thus helps to improve fabrication tolerances370. A 5 dB performance increment was achieved compared with a device of similar size optimized based on full-filling rectangle pixels. Various PhC-like waveguides were also reported, including mode multiplexer415 (Fig. 11h1), on-chip routing69 (Fig. 11i1), star crossing416, wavelength demultiplexer417, polarization rotator418, multimode bend419, dual-mode crossing420, and power splitter421. Furthermore, a rotatable DBS algorithm that adds a calculation of rotational dimension was proposed as Fig. 11j1 for simultaneous mode converting and power splitting422.

### Design procedures

Take the genetic algorithm (GA) as an instance. (i) Generating initial populations as the first generation. Each population is a matrix of pixel distribution. (ii) Evaluating each population’s FOM. (iii) Producing new populations with sequential application of selection, crossover, and mutation. Multiple strategies can be selected, e.g., roulette wheel or tournament selection. Each of the selected population has a specified or random probability to have a crossover with another population, where the crossover point is randomly picked (the left panel in Fig. 10d). After crossover, each population in the new generation has a probability of mutation, where a random number of pixels are selected to reverse (right panel of Fig. 10d). Iterations are thus formed by going back to Step (ii) to evaluate the new generation and population reproduction in Step (iii).

### Foundational developments

As DBS algorithm mostly converges to a local optimum, it is more suitable for optimizations in a small parameter space. With a large parameter space, heuristic method, such as genetic algorithm (GA) and particle swarming optimization (PSO), are more competitive. Fig. 11k1 sketched a GA-optimized SiO2/SiON telecom-fiber to ridge-waveguide coupler423, with a 2 dB enhancement in efficiency compared with direct coupling. As shown in Fig. 11l1–p1), GA was then extended to a wider range of technologically relevant applications, such as silicon planar resonator424, reflector425, long-pass filter426, wavelength router427, and light modulator428. More recently, GA method was combined with simulated-annealing algorithm to enable accurate phase control in freeform meta-waveguides, contributing to high efficiency devices for complicated mode conversion from guided modes to free space (e.g., orbital angular momentum emitter47 shown in Fig. 11q1).

On the other hand, the particle-swarm optimization algorithm also can be applied to design metamaterial waveguides (see Fig. 11r1, s1), such as 2 × 2 power splitter429 and polarization beam splitter430. Particle-swarm optimizations are based on the movements of a population of candidate solutions (particles) in the search space. During optimization, the initially distributed particles continue moving towards the then-current optimum particle in the swarm, until the FOM is reached431.

### Design procedures

(i) Tuning the permittivities of all pixels with a fixed shape continuously and individually to obtain an optimized analog pattern with “gray” pixels using adjoint method. (ii) Then, a forced biasing approach was applied to convert the analog pattern to a “quasi-digital” one in which the permittivities of most pixels are close to two boundary values. (iii) Fabrication-constraint brute-force quantization methods were designed to transform the “quasi-digital” pattern into an “N-ary digital” pattern, as illustrated in Fig. 11t1.

### Foundational developments

Distinctive from the TO or lever-set method, gradient information is generally hard to retrieve for digital meta-waveguides. Without gradients to efficiently update searching direction, convergence of these optimization algorithms is often considerably slower. Fortunately, recent research validated the feasibility of adapting the adjoint method into the inverse design of digital meta-waveguides. As shown in Fig. 11u1, a 3 dB power divider and a dual-mode multiplexer were demonstrated62, using PhC-like subwavelength structure with cylindrical-shaped base pixels in design region. At the implementation stage of “N-ary digital” pattern, cylinders with different “gray” permittivities in the “quasi-digital” pattern are replaced with air cylinders with original radius, silicon cylinders, or air cylinders with N − 2 different radii on the basis of effective medium theory to minimize the performance degradation due to the digitalization process. Compared with the conventional DBS method, the introduction of adjoint method can improve design efficiency by about five times, while the performance optimization can reach approximately the same level.

Artificial neural networks (ANN) are algorithms that utilize a series of nonlinear mathematical functions, in the form of neurons, to specify highly nonlinear mappings between inputs and outputs148. A training process, in which weights within the neurons are iteratively adjusted, is used to tailor these mappings. ANNs have been a topic of study for the last half century and have gained immense popularity in nearly every technical field of research over the last decade due to the emergence of the deep network framework, user-friendly software packages, and application-specific hardware432,433. The deep ANN framework, where many layers of neurons are utilized in the algorithm, has led to particularly versatile and expressive algorithms that can serve as universal functional approximators. Software packages, ranging from PyTorch to TensorFlow, have led to standardized and easy-to-program interfaces to implementing ANNs with arbitrary architectures432. Specialized hardware includes graphical and tensor programming units that can dramatically speed up the network training process. Furthermore, the culture of opening and sharing in the machine-learning community leads to the availability of many state-of-the-art algorithm resources easy to access and follow up online432,434 (https://github.com/iguanaus/ScatterNet, http://github.com/yuruiqu/Transfer-learning_Data, https://github.com/PRGatech/DimensionalityReduction).

In this section, we give an overview of inverse-design methods based on discriminative networks, generative networks, and reinforcement learning. While we will present examples from the broader context of nanophotonics design (see Fig. 12), the concepts can readily apply to meta-waveguide design without loss of generality148,432,435.

#### Inverse design with discriminative ANNs

Discriminative ANNs are algorithms that can capture data relationships of the form y = f(x) When x specifies a device pattern and y is its optical response, the discriminative ANN serves as a surrogate electromagnetic (EM) solver, solving the forward problem432. To train a discriminative ANN, a training set of known device patterns and optical responses is first created using a conventional EM solver. The ANN is then trained to minimize error between the outputted and known optical responses, for given device pattern inputs. Mathematically, the loss function is defined to be the mean square error between the outputted and ground truth optical responses, and this function is minimized during the training process by applying back-propagation algorithm148. While the training data collection and ANN training are computationally expensive, a trained ANN can perform computations orders of magnitude faster than a conventional solver. A challenge to training an accurate and generalized ANN is curating a sufficiently large and diverse training set that properly represents the desired computation space. A basic and typical strategy is to use a random, uniformly distributed set of devices432. If there are known, statistically rare features pertaining to the device geometry or optical response, these features can be learned by the ANN by augmenting the training set with a disproportionally large number of examples of these data.

Discriminative ANNs can be used for inverse design in multiple ways. One is that the surrogate solver can be used in conjunction with conventional optimization methods, which include evolutionary strategies436, particle-swarm optimization437, and brute-force sweeping and searching438,439. Many of these strategies typically require hundreds to thousands of simulations performed in series, and the utilization of an accurate high-speed solver can reduce total optimization times from hours or days to seconds, one example is illustrated in Fig. 12a439. Another way is to use the trained forward ANN to facilitate the training of an inverse network, which attempts to output a device pattern given a desired optical response input440 (shown in Fig. 12b). The reason why the tandem network is trained, as oppose to direct training of an inverse network, is that the inverse problem is a one-to-many mapping problem: there exist multiple device patterns that can produce the same optical response. This type of mapping cannot be captured by a discriminative network, which can only capture one-to-one or many-to-one relationships. The trained forward ANN reduces the one-to-many mapping issue because it is not an exact surrogate simulator and only approximates the complex, degenerate design space. Tandem networks have been used to design thin film stacks440, color filters441, and topological photonic structures442. To tackle the one-to-many problem, several solutions have also been developed, including conditional generative adversarial networks and conditional variational autoencoders, where a latent vector is introduced in the ANN to address potential multiple solutions433.

Discriminative ANNs can also perform inverse design using the backpropagation method. Previously, backpropagation was discussed in of network training, where gradients to neuron weights were calculated to minimize the loss function. To perform inverse design with a trained discriminative ANN using backpropagation, the loss function is defined as the difference between the output and desired optical responses and backpropagation is used to calculate gradients to the input device pattern x to minimize the loss function. This method has been used to inversely design scatterers443 and photonic crystals444. The concept can also be applied to more complex computational graphs that combine ANNs with analytic physical expressions445. Furthermore, a neural-adjoint method that adds the boundary loss to the loss function is proposed to improve performance446,447.

#### Inverse design with generative ANNs

Deep generative networks448,449,450 are architecturally similar to discriminative networks except that one of the inputs to the network is a latent random variable z. As such, the output to the network is a distribution, typically of device patterns, which can be generated by sampling z. There are two general inverse-design concepts based on generative networks. The first is to train a generative network to produce distributions that mimic a training set of device patterns. For networks where the input is the desired optical response and z, the generator functions as an inverse network: z is sampled to produce an ensemble of candidate devices, which are then evaluated and filtered to identify devices with suitable performance. For networks where the input is a device operating parameter (i.e., operating wavelength or material index) and z, the generative network can be used for interpolation and output devices with operating parameters not found in the training set.

There are different ways for generative networks to learn training set distributions. One way is with variational autoencoders (VAEs), which consist of two parts: an encoder that maps device patterns onto a low dimensional latent space distribution, and a decoder that maps latent space data representations back to device patterns. The loss function includes a reconstruction term, which minimizes differences between the inputted and outputted device patterns to the full network, and regularization terms that tailor the latent space distribution to fit a Gaussian profile. As shown Fig. 12d, VAEs have been used to design analog power splitters with various splitting ratios61, and digital multimode interference waveguides were designed by training a VAE in tandem with a pretrained forward network451. Another way to train generative networks is with the generative adversarial network (GAN) framework, where the generative network is trained together with a discriminative classification network that attempts to distinguish whether an input device is fake (i.e., from the generative network) or real (i.e., from the training set). Over the course of training, the discriminator gets better at distinguishing real and fake devices while the generator produces more realistic looking devices. Upon the completion of training, the generator fools a trained discriminator by outputting a distribution of devices mimicking the training set. GANs have been used to design freeform metasurfaces that support tailored spectral responses448 and high efficiency meta-gratings449,450 (Fig. 12e–g).

A second concept for inverse design with generators is based on the dataless training of generative networks to perform a population-based search for the global optimum452,453. These networks are termed global topology optimization networks (GLOnets) and the network training procedure works as follows. Initially, the outputted distribution from the generative network spans the full design space such that sampling z produces a batch of random devices. These devices are evaluated by an EM solver to quantify their performance and performance gradients (i.e., pattern modifications that improve performance), and these performance metrics are used to calculate a special loss function. Back-propagation is used to modify the network based on this loss function, such that the device distribution outputted by the network narrows and gets biased towards higher performance regions of the design space. This process is repeated and the device distribution from the network output distribution eventually collapses, ideally around the global optimum. Initial GLOnet demonstrations were used to optimize metasurface patterns, and the final freeform devices had efficiencies that were consistently better than those designed using gradient-based optimization452 (Fig. 12h). GLOnets were also demonstrated to be compatible with reparameterization to enable the global search of metasurfaces with hard minimum feature size constraints407. Subsequent improvements to the stability and performance of GLOnets have been made in thin film stack optimization, through judicious selection of the network architecture453. These concepts showcase the great potential of hybrid algorithms to interface physics and data sciences.

#### Inverse design with reinforcement learning

In reinforcement learning, a neural network learns to specify a sequence of actions within an environment in a manner that maximizes a cumulative reward454,455,456. These concepts have been popularized with their application to games, such as Atari games or Go, and have had significant technological impact in fields ranging from robotics to communications457. By training a deep network to play the game many times, initially with random actions (i.e., exploration) and later with actions informed by past experiences, the network will learn to specify an optimal action that maximizes current and future rewards. The consideration of cumulative rewards during the training process allows the algorithm to balance trade-offs between long-term and short-term reward gains432.

The framework of reinforcement learning naturally maps onto the iterative optimization process for nanophotonic devices. In this context, the state is the device pattern at a given iteration, the action is a modification to the pattern, and the reward combines the present and future device performance as quantified by the FOM. As the reinforcement learning algorithm learns from many trajectories through a design landscape and is based on maximizing cumulative reward, the device pattern modification suggested by the algorithm in a given optimization iteration is not simply the modification that produces the largest improvement to the FOM, which is the case in gradient-based optimization. Rather, the algorithm will suggest trajectories through a design landscape that can sidestep local optima in an attempt to maximize the performance of the final device. It is noted that the training process for reinforcement learning is computationally expensive, and the simulation of devices over many trajectories through a design landscape can take days on a conventional CPU, even for relatively basic problems. These algorithms are therefore more practically implemented when paired with accurate surrogate simulators. Reinforcement learning has been used to design periodic dielectric metasurfaces that support tailored spectral responses454, multi-layer structures455, perfect absorbers456, and metasurface holograms457 (Fig. 12i).

### Comparisons of different design methods

Table 1 summaries several representative metamaterial waveguides (usually with wavelength-scale dimensions) to briefly compare the inverse-designed methods discussed above. As the landscape of a metamaterial waveguide is generally non- convex, currently it is still not mathematically possible to guarantee global optimal solutions in inverse-design optimization problems15,58. Despite that certain global search or reinforcement learning algorithms47,432,452 attempt to maximize the performance of the final device, but global optimum still not surely undertaken. Some active research area have been developed to explore the computational bounds on the true optimal value of an given optimization problem, which essentially derive from Lagrange duality458, local power conservation459 or diagonal physics dual460 based on the basic properties of the constraints and objective functions. Such bounds not only help to provide guidelines to the maximal device performance possible within physical limits, but also can be used to rule out constraints and objective functions for which no device pattern can achieve such good objective value.

Gradient-based algorithms (e.g., TO or level-set method) have many advantages, such as faster convergence and powerful mathematics-handling capacity particularly with lager parameter space. However, fabrication constraints are usually necessary. Imposing a larger feature size constraint in the optimization may deteriorate the performance461. In some extreme cases, the inverse-design algorithm could not find photonic structures on-demand even without imposing fabrication constraint462, the theoretically achievable performance or the desirable functionality may be ultimately bound by material constraints.

Non-gradient-based algorithms (such as DBS, GA or PSO algorithms) are usually easy to implement and need few mathematical requirements (e.g. complex gradient calculations are not needed). However, the convergence is often considerably slower, particularly when the objective function or parameter space becomes complex. To mitigate this issue, parallelizing the algorithm and using larger clusters of processors would be necessary463. On the other hand, although digital devices inspired by these non-gradient-based optimization methods has developed rapidly and achieved remarkable successes in many fields, most of the optimization methods are less involve with more complex physics problems, such as some emerging applications in nonlinear optics366,388,393.

For deep learning methods, it is possible to accelerate the simulation and design of optical devices in an effective and efficient manner438,464,465. The choice of neural network architecture and training strategy depends largely on the type of photonic system being analyzed. For systems that are relatively low dimensional and can be described by fewer than approximately ten independent geometric parameters, a discriminative neural network can effectively serve as a surrogate solver for that system and can be used in various optimization strategies to perform inverse design. These networks do require a large, one-time computational investment into a training set, making the neural network approach appropriate only when it involves systems, such as color filters or meta-atoms, where a wide range of layouts and optical responses are of interest. If a library of useful device shapes is known and are useful guidelines for design, the GAN approach provides a direct route to learning and interpolating related shapes from this library. If the goal is to identify a particularly high-performance device from a computation space, GLOnets provides a strategy to effectively search for the global optimum, without the requirement of a training set.

## Summary and outlook

In summary, optical waveguides have proven a canonical platform to integrate diverse functional subwavelength meta-structures to enable meta-waveguides with either novel functionalities or largely boosted device performance. Compared with conventional waveguides devoid of subwavelength structures, meta-waveguides can not only just guide and confine light, but also perform various functionalities underpinned meta-structures. Compared with photonic crystal waveguides, meta-waveguides can act as ‘designer artificial media’ (with distinctive waveguiding mechanism to photonic bandgap) that enable versatile and powerful control over light propagation with subwavelength precision.

By allying functional subwavelength structures with dielectric and plasmonic waveguide platforms, versatile coupling interfaces, on-chip optical signal processors, photonic neural networks, multifunctional routers, mode convertors, sensors, quantum and nonlinear devices can be envisaged. By integrating meta-structures with optical fibers, massive applications in optical communications, imaging, biomedical sensing and lab-on-fiber technologies can be realized in a flexible, compact and multifunctional manner.

Despite the concept of meta-waveguides is still in its infancy, exciting progress are hatching with bright perspectives and profound potential applications. As conceptually illustrated in Fig. 13, the advancement of meta-waveguides can not only extend meta-optics to the realm of guided electromagnetic waves and waveguide technology, but also promise to reshape the landscapes of photonic integrated circuits and massive emergent applications. The awaiting challenges are outlined as the following. A brief roadmap is also presented as Fig. 14 for future research.

### Challenges

Despite the exciting progress discussed above12,16,17,466, practical applications of meta-waveguides are still limited. Several challenges must be properly handled before such ideas may transfer from scientific demonstrations to viable real-word technologies in future.

First, reconfigurable meta-waveguides are highly desired. For most current cases, the device attributes are fixed after fabrication35,36,37,41,42,43,44,70,75,76,77. Further advancement of photonic integrated circuits and other applications will demand multi- functionality and reconfigurability of diverse photonic devices in a way similar to their electronic counterparts1. To this end, alliance from material platforms would be necessary. For instance, the meta-structures on top of waveguides can be mold by phase-change materials177 or two-dimensional materials40,467,468 to facilitate reconfigurable integrated devices. Lithium niobate is also recently emerging as an excellent electro-optic (EO) and nonlinear material platform469,470 with new opportunities for creating meta-waveguides. Other tunable mechanisms such as electrical-32,471, thermal-174,472, mechanical-, and all-optical control may be explored as well473.

Second, challenges in device design still exist. (i) For meta-waveguides designed by physical intuitions, a universal design framework is still elusive. Most explorations start from distinct physical models under different waveguide platforms with unique features51,70,75. (ii) For inverse-designed meta-waveguides, in scientific research-wise, some academic questions still remain open15, such as the existence of solution under given optimization conditions and the ultimate computational boundary. In application-wise, fully automated photonic inverse-design kits and integrated software tools still await development.

Third, as shared by all emergent technologies struggling to migrate from lab to real-word, meta-waveguides also have to settle challenges in scalable fabrication and compatible integration. The meta-structures generally pose stringent requirements on fabrication resolution12. Fabrication constraints should be considered in design process to pursue “efficient-to-fabricate structures”. Complementary metal-oxide-semiconductor (CMOS) compatibility to electronics needs to be addressed as well to eventually enable large-scale manufactures fitting the foundry’s process design kits1.

Detailed challenges and potential research directions are discussed below, according to different design directions and distinctive waveguide platforms.

#### Challenges for dielectric meta-waveguides

For instance, the substrate light leakage during guided wave and free-space light couplings via metasurface-decorated waveguides will reduce conversion efficiency. This issue could be solved by using waveguide-integrated distributed Bragg reflector substrate, layered structures, and topologically protected structures474,475. Besides, the intrinsic dispersive characteristics of the meta-waveguides should be also addressed for broader band operations.

In addition, most of the meta-waveguides introduced above are based on local metasurfaces146, where the desired device functionality is realized by independent scatterers ignoring the neighboring coupling of the meta-units145. These devices generally operate at single wavelength with phase-only manipulation, for instance, dielectric waveguides patterned with phase- gradient metasurface35,41,42,43,51. To thoroughly handle the detrimental effect of neighboring coupling on device performance and ove