Introduction

Quantum systems that interact with an environment are ubiquitous in physics. The study of the interaction between such systems and baths is of fundamental interest as it leads to various novel quantum effects1,2 and applications3. In open quantum systems, although the system and the bath as a whole are Hermitian, the dynamics of the partial system alone can be described by an effective non-Hermitian model4. This has recently aroused enormous interest in uniform non-Hermitian quantum systems, where exotic physics have been found, including novel topological phases5,6,7,8,9,10,11,12,13,14,15,16,17, skin effects18,19,20,21,22,23,24,25,26,27,28,29, enriched classifications30,31,32,33,34,35,36,37,38,39,40,41, and unusual critical phenomena42,43,44.

It is, however, important to note that non-Hermiticity in realistic open quantum systems can be non-uniform, and in many cases, is only present at the system boundary. We consider systems interacting with their surrounding environments via short-range couplings across boundaries. The general Hamiltonian reads as H = HS + HE + VSE, where HS describes the closed quantum system, HE captures the environment consisting of a continuum of scattering wavefunctions, and VSE describes the coupling between them, which is short-ranged and restricted to the region around the boundary. The renormalization of a quantum system from the environment via VSE is non-negligible4, which formally leads to the effective Hamiltonian as

$${H}_{eff}={H}_{S}+\mathop{\sum}\limits_{E}{V}_{SE}\frac{1}{{\omega }^{+}-{H}_{E}}{V}_{SE}^{{{{\dagger}}} },$$
(1)

where ω+ = ω + i0+ and the second term includes the sum of all the scattering channels in the environment. This term arising from system-bath coupling is clearly non-Hermitian. Importantly, since VSE is restricted in real space around the boundary, the non-Hermitian terms take place only at the boundary. The non-Hermitian boundaries implicit in Eq. (1) are common in open quantum systems. However, their effects have not received enough attention in recent studies. Could new phenomena emerge from the non-Hermitian boundaries, and how the non-Hermitian boundaries would affect the quantum systems and their dynamics? These key questions are yet to be addressed.

In this work, we reveal an effect induced by the non-Hermitian boundary, i.e., the non-Hermitian proximity effect (NHPE). We show that the non-Hermiticity of the boundary can penetrate into the nearby quantum systems with a finite penetration depth, akin to the proximity effect of superconductors45,46,47,48. For gapped quantum systems, the NHPE induces “in-gap states" with imaginary eigenenergies, i.e., imaginary in-gap states. The imaginary in-gap states display peaks of the local density of states (LDOS) clearly distinct from that of the bulk states. In addition, the imaginary in-gap states decay into the bulk in a way phenomenologically similar to that of conventional impurity problems49,50,51,52,53,54, where the localized in-gap states decay into the bath54,55,56,57. Despite the similarity, the imaginary in-gap states also exhibit several distinct properties. First, the imaginary in-gap states are generally manifested as corner modes under open boundary conditions (OBCs), due to the combined effect of the NHPE and the non-Hermitian skin effect (NHSE). Second, the NHPE leads to unusual dynamical behaviors of the quantum system. In particular, in the cases where the imaginary part of the imaginary in-gap state eigenenergy is positive, the probability distribution of the imaginary in-gap state keeps increasing due to the gain from the environment. As a result, under time evolution, all wave packets will be evolved into the imaginary in-gap states displayed as corner modes under OBCs. Our work reveals that impurity-like behaviors with unusual dynamics can emerge from quantum models with non-Hermitian boundaries. This points to a new direction that interwinds impurity physics and non-Hermitian effects, which could commonly take place in open quantum systems.

Results

Imaginary in-gap states

We exemplify our study by starting with the Qi-Wu-Zhang (QWZ) model58 describing 2D Chern insulators on a square lattice (Fig. 1a), i.e.,

$${H}_{0}= \mathop{\sum}\limits_{x,y}\left\{\left[{c}_{x+1,y}^{{{{\dagger}}} }\frac{(i{\sigma }_{x}-{\sigma }_{z})}{2}{c}_{x,y}+{c}_{x,y+1}^{{{{\dagger}}} }\right.\right.\\ \left.\left.\frac{(i{\sigma }_{y}-{\sigma }_{z})}{2}{c}_{x,y}\right]+{c}_{x,y}^{{{{\dagger}}} }m{\sigma }_{z}{c}_{x,y}\right\}+h.c.,$$
(2)

where \(c(x,y)={[{c}_{a,x,y},{c}_{b,x,y}]}^{{{{{{{{\rm{T}}}}}}}}}\) is the spinor with a, b sublattice. We further consider the non-Hermitian terms along the boundary y = 1:

$${H}_{\gamma }=i\mathop{\sum}\limits_{x}{c}_{x,1}^{{{{\dagger}}} }\gamma {\sigma }_{x}{c}_{x,1}.$$
(3)

Eq. (3) describes a non-Hermitian boundary as indicated by the red sites in Fig. 1a, which can be generated by coupling H0 to an environment at y = 1.

Fig. 1: Imaginary in-gap states (IISs) induced by non-Hermitian boundary.
figure 1

a The Qi-Wu-Zhang (QWZ) model with on-site non-Hermitian boundary terms iγσx (marked by red circles) on the top layer of the system and on-site mass terms mσz (marked by blue circles). Under the cylinder geometry (the size of the system in the y-direction is denoted by Ly), for any fixed kx, the model is described by a 1D vertical chain with a non-Hermitian term on its first site. b, c The real and imaginary parts of the spectrum for the model H = H0 + Hγ under the cylinder geometry, where H0 is given by Eq. (2), and Hγ is given by Eq. (3). The imaginary in-gap states expand into the bulk states as “tail" states marked by the red dashed curves. The parameters used are m = 1.0, γ = 1.5 and Ly = 20. d The probability distribution of the imaginary in-gap states and the topological chiral edge states with kx = 0.2, which are marked by the arrows in (b) and (c).

Full size image

We first assume the cylinder geometry, i.e., the periodic boundary condition along x and OBC along y with y ∈ [1, Ly]. The real and imaginary parts of the energy spectrum are numerically obtained and shown in Fig. 1b, c, respectively. Two chiral edge states of the Chern insulator are observed in the real spectrum, connecting the conduction and valence band. These two edge states are respectively localized at the upper (y = 1) and lower (y = Ly) boundary, as shown by Fig. 1d. It is also shown in Fig. 1c that the upper chiral mode displays nonzero imaginary spectrum while the lower mode remains real. This is expected since the non-Hermitian term γ is only applied on the upper boundary.

Despite the chiral edge states, we also observe two channels of in-gap modes for −0.6 ≲ kx ≲ 0.6, as marked by the red solid curves in Fig. 1b. These in-gap modes do not connect the conduction and valence band, suggesting that they have a non-topological physical origin. Both channels describe edge states localized at the upper boundary, as evidenced by Fig. 1d. Moreover, as shown by Fig. 1c, each mode with fixed kx (marked by a red dot) in the two edge states exhibits nonzero imaginary eigenenergy, thus termed as the imaginary in-gap state. We note that the dispersion of imaginary in-gap states has finite extensions into the bulk states with displaying “tail" states, as marked by the red dashed curves in Fig. 1b.

Non-Hermitian proximity effect

To reveal the origin of imaginary in-gap states, we first perform a real-space renormalization group (RG) analysis (see Supplementary Note 1). We decompose the total system H = H0 + Hγ into a series of coupled 1D horizontal layers, i.e., \(H={\sum }_{{k}_{x}}[\mathop{\sum }\nolimits_{l = 1}^{{L}_{y}}{H}_{l,{k}_{x}}+\mathop{\sum }\nolimits_{l = 1}^{{L}_{y}-1}{H}_{l,l+1,{k}_{x}}]\), where \({H}_{l,{k}_{x}}\) is the Hamiltonian of the lth layer, and \({H}_{l,l+1,{k}_{x}}\) depicts the coupling between the lth and (l + 1)th layer. The non-Hermitian boundary, i.e., Eq. (3), is included in \({H}_{1,{k}_{x}}\), which reads as, \({H}_{1,{k}_{x}}=(\sin {k}_{x}+i\gamma ){\sigma }_{x}+(m-\cos {k}_{x}){\sigma }_{z}\). Then, we integrate out the l = 1 layer and obtain a renormalized Hamiltonian for the rest of the system, whose top layer (l = 2) now becomes non-Hermitian. Then, as indicated by Fig. 2a, by iteratively repeating the above procedure, the effective Hamiltonian for the (l + 1)th layer can be obtained after the lth RG step as,

$${H}_{l+1,{k}_{x}}^{eff}=\sin {k}_{x}{\sigma }_{x}+(m-\cos {k}_{x}){\sigma }_{z}+{f}_{l}({\sigma }_{0}+{\sigma }_{x}),$$
(4)

where fl is the complex self-energy with kx-dependence. In Fig. 2b, we plot the calculated \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]\), as a function of the RG step l. As shown, \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]\) decays with l for all kx, and saturates to the fixed point \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]=0\) for large l. This shows that the non-Hermitian terms on the boundary can effectively penetrate into the bulk with a penetration depth. Due to its analogy with the proximity effect of superconductivity where Cooper pairs leak into the adjacent system, we term it the NHPE.

Fig. 2: Non-Hermitian proximity effect and the corner modes under open boundary conditions (OBCs).
figure 2

a Schematic plot of the real-space renormalization group (RG) procedure. The renormalized (l+1)th layer can be obtained after integrating out the lth layer. b The imaginary part of the complex self-energy fl as a function of the RG step l for different kx. \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]\) decays with l and eventually saturates to the fixed point \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]=0\) for large l. The model we use is H = H0 + Hγ where on-site non-Hermitian boundary terms iγσx are on the top layer of the system. c–f The probability distribution of eigenstates under OBCs with size Lx × Ly = 60 × 30, the color bars on the right side of the panels indicate its value under normalization. c The case for the bulk states. d, e are results for the imaginary in-gap states with opposite values of γ, and f shows the result for the “tail state" deep in the bulk.

Full size image

To better visualize the NHPE, we adopt OBCs along both x and y directions and calculate the spatial distribution of the eigenstates of H. The typical spatial distribution of the bulk states is shown in Fig. 2c. It is clear that the bulk states remain as Bloch states and are barely affected by the non-Hermitian effects. This is expected from Fig. 2b since the non-Hermitian terms are negligible for bulk states far away from the upper boundary. Then, we focus on the imaginary in-gap states and pick up their corresponding eigenstates under OBCs. The typical spatial distribution of imaginary in-gap states is shown in Fig. 2d, e. Due to the NHSE, which is significant near y = 1, the imaginary in-gap states are found to localize either at the right or left boundary, depending on the sign of γ (Fig. 2d, e, respectively). Moreover, they are also localized at the upper boundary due to the NHPE, with a short localization length along y up to a few layers. Therefore, the combination of NHSE and NHPE generally drives the imaginary in-gap states into corner modes under OBCs. In addition, we also plot in Fig. 2f the distribution of the “tail states" (the dashed curve in Fig. 1b) deep in the bulk. Compared to the imaginary in-gap states, these states are found to exhibit longer localization lengths along y because of their smaller imaginary part of eigenenergies.

The analogy between imaginary in-gap states and impurity states

So far, we have shown that the imaginary in-gap states manifest themselves as edge modes and corner modes under the cylinder geometry and OBCs, respectively. For both cases, the imaginary in-gap states are localized at the upper boundary and decay along y, as a result of the NHPE induced by the non-Hermitian boundary.

To further reveal the physical nature of imaginary in-gap states, we now demonstrate their underlying similarity with the conventional impurity states in gapped systems. Under the cylinder geometry, kx is a good quantum number. The total system can then be written as \(H={\sum }_{{k}_{x}}{H}_{{k}_{x}}\), and

$${H}_{{k}_{x}}= \mathop{\sum}\limits_{y}[{c}_{{k}_{x},y+1}^{{{{\dagger}}} }{T}_{y}{c}_{{k}_{x},y}+h.c.]\\ +\mathop{\sum}\limits_{y}{\epsilon }_{{k}_{x}}{c}_{{k}_{x},y}^{{{{\dagger}}} }{c}_{{k}_{x},y}+i{c}_{{k}_{x},1}^{{{{\dagger}}} }\gamma {\sigma }_{x}{c}_{{k}_{x},1},$$
(5)

where \({c}_{{k}_{x},y}={[{c}_{a,{k}_{x},y},{c}_{b,{k}_{x},y}]}^{{{{{{{{\rm{T}}}}}}}}},{T}_{y}=(i{\sigma }_{y}-{\sigma }_{z})/2\) and \({\epsilon }_{{k}_{x}}=[\sin {k}_{x}{\sigma }_{x}+(m-\cos {k}_{x}){\sigma }_{z}]\). For fixed \({k}_{x},{H}_{{k}_{x}}\) describes a 1D vertical chain model with a non-Hermitian term on its first site, as indicated by Fig. 1a. It is clear that the localization and decay behaviors of an imaginary in-gap state are fully captured Eq. (5).

We now calculate the LDOS from Eq. (5). The density of states (DOS) can be obtained from the imaginary part of the retarded Green’s function GF, i.e., \(\rho (\epsilon )=-\frac{1}{\pi }{{{{{{{\rm{Im}}}}}}}}{{{{{{{\rm{Tr}}}}}}}}\left[{\sum }_{n}\frac{\left\vert {\psi }_{n}^{R}\right\rangle \left\langle {\psi }_{n}^{L}\right\vert }{\epsilon +i{0}^{+}-{E}_{n}}\right]\), where En is the nth complex eigenvalues. Here, compared to the Hermitian cases, the support of DOS is expanded to the complex plane59,60, i.e., ϵ = ϵr + iϵi. Besides, both the left and right eigenstates have been used to form the complete basis61 that expands the GF. Then, the LDOS at the l-th site of the chain can be derived as \({\rho }_{l}({\epsilon }_{r},{\epsilon }_{i})=\frac{1}{N}{\sum }_{n}\delta ({\epsilon }_{r}-{{{{{{{\rm{Re}}}}}}}}[{E}_{n}])\delta ({\epsilon }_{i}-{{{{{{{\rm{Im}}}}}}}}[{E}_{n}])| \langle l| {\psi }_{n}^{R}\rangle \langle {\psi }_{n}^{L}| l\rangle |\).

We show in Fig. 3a, b the calculated LDOS, with focusing on the imaginary in-gap state region (−0.6 ≲ kx ≲ 0.6). At the boundary site y = 1 (Fig. 3a) of the 1D vertical chain, except for the topological edge mode from the upper chiral edge state, we observe two significant LDOS peaks located at energies with ϵi < 0. These peaks come from the two imaginary in-gap states localized at y = 1. Moving away from y = 1, the imaginary in-gap state peaks quickly decay. Meanwhile, the LDOS from the bulk states emerges, which is located on the real energy axis. As shown by Fig. 3b, for y ≫ 1, both the peaks from the imaginary in-gap states and the topological edge state disappear, leaving only the bulk states. The LDOS projected to the real energy axis is also shown in Fig. 3c, d for clarity.

Fig. 3: Analogy between imaginary in-gap states and impurity states and their local density of states (LDOS).
figure 3

a, b The LDOS of the non-Hermitian 1D vertical chain model (for fixed kx) in Eq. (5) with a non-Hermitian term iγσx on its first site and on-site mass terms mσz. The parameters are m = 1.0, γ = 1.5, and kx = 0.2. In a, there emerge two imaginary in-gap state (IIS) peaks together with that from the upper topological edge state at y = 1. In b, far away from the boundary (y = 7), only the LDOS from bulk states is left which is located on the real energy axis. c, d The projections of (a) and (b) onto the real axis. The LDOS from the topological edge state, which is not our focus, has been excluded for clarity. The gap energy is at ω = ± 1, as marked by the dashed lines. e Schematic plot of an impurity embedded in the bath, which is mapped to a 1D Wilson open chain. f, g The calculated LDOS of the impurity model with the impurity level ϵf and the hybridization V. The parameters used are m = 1.0, ϵf = 0, and V = 3.0. In f, two impurity in-gap states emerge near the impurity site, and two minor peaks also arise from the mixing with the bulk states. In g, far away from the impurity, only the low-energy bulk states are left in the LDOS.

Full size image

To clearly show the similarity between imaginary in-gap states and impurity states, we consider a non-interacting pseudospin Anderson impurity model coupled to the QWZ model, i.e., Himp = H0 + Hf, where the impurity is described by

$${H}_{f}=\mathop{\sum}\limits_{\sigma }{\epsilon }_{f}{f}_{\sigma }^{{{{\dagger}}} }{f}_{\sigma }+V\mathop{\sum}\limits_{{{{{{{{\bf{k}}}}}}}},\sigma }(\,{f}_{\sigma }^{{{{\dagger}}} }{c}_{{{{{{{{\bf{k}}}}}}}},\sigma }+h.c.).$$
(6)

fσ is the annihilation operator of the impurity state with pseudospin (sublattice) σ, ϵf denotes the impurity energy level (independent of σ), and V its hybridization with the bath electrons. Although the impurity considered in Eq. (6) is located in the bulk of the 2D Chern insulator as shown by Fig. 3e, the low-energy impurity state is accurately determined by a 1D Wilson open chain following the numerical renormalization group mapping scheme (see Supplementary Note 4), as schematically shown by Fig. 3e. The Wilson open chain shares a similar structure with the 1D vertical chain model in Fig. 1a. Both of them describe a boundary site (y = 1) coupled to bath degrees of freedom (y > 1) via the nearest neighbor coupling. It should be noted that the y > 1 sites on the Wilson chain represent the bath states in discretized energy windows. The site is closer to the impurity (y = 1) and describes the states in the higher energy window, i.e., in the shorter length scale measured from the impurity.

We then calculate the LDOS on different sites of the Wilson chain, as shown in Fig. 3f, g. For y being close to the impurity, two in-gap states emerge, due to the coupling of the impurity to the conduction and valence band electrons. With tuning y away from the impurity, the LDOS from the impurity states decreases and that from the bulk states increases. For large y, only the LDOS of the low-energy bulk states are left, which are located around the gap energy, as shown in Fig. 3g. The LDOS shown in Fig. 3f, g is qualitatively in analogy with the projected LDOS in Fig. 3c, d, implying the similarity between the imaginary in-gap states and the impurity physics.

Distinct dynamical behaviors of imaginary in-gap states

Despite the similarity, the imaginary in-gap states also exhibit some distinct features. First, since the LDOS peaks of imaginary in-gap states are located in the complex energy plane with ϵi ≠ 0, they can persist even if their real parts of eigenenergies are immersed in the bulk continuum (see Supplementary Note 2). This explains the existence of the “tail states" observed in Fig. 1b. Second, the imaginary in-gap states exhibit unusual dynamical behaviors absent in the Hermitian systems, as a result of their imaginary eigenenergies. Under time evolution, the amplitude of an imaginary in-gap state either grows or decays, depending on the sign of γ (see Supplementary Note 3). This is a reflection of the non-Hermitian nature of the imaginary in-gap states, in contrast with the conventional impurity states.

For a Gaussian wave packet input at the bottom layer of the system, the time-evolution of the wave packet is calculated under OBCs and shown in Fig. 4a–c (for γ < 0). The wave packet first diffuses into the bulk (Fig. 4b) and then gradually evolves into the imaginary in-gap state eigenstate localized at the top layer. Due to the NHSE, it finally turns into the corner state as shown in Fig. 4c. We also start from a Gaussian wave packet placed at any generic site at the top layer. As shown by Fig. 4d, the wave packet also evolves into the imaginary in-gap state located at the corner. Hence, we observe that, for γ < 0 where the amplitude of imaginary in-gap states grows, a generic Gaussian wave packet will always evolve into the cornered imaginary in-gap state (see Supplementary Note 3 for the γ > 0 case). Such a dynamical feature is a clear distinction between imaginary in-gap states and conventional in-gap states.

Fig. 4: Time-evolution of the wave packet.
figure 4

a The initial state is set to be a Gaussian wave packet, placed at the bottom layer of a square lattice with size Lx × Ly = 20 × 20. The model is described by H = H0 + Hγ, where H0 is given by Eq. (2) which contains on-site mass terms mσz, and Hγ is given by Eq. (3) with the on-site non-Hermitian boundary terms iγσx located on the top layer of the system. The wave packet evolves according to the Schrödinger equation. b The Gaussian wave packet diffuses into the bulk. c The Gaussian wave packet eventually evolves into the imaginary in-gap state localized at the upper-left corner. d The evolution of a wave packet initially placed at the top layer. The color bars on the right side of the panels indicate the value of the normalized probability distribution at different times. Common parameters: m = 1.0, γ = − 1.5.

Full size image

Discussion

This work reveals an impurity-like non-Hermitian phenomenon induced by non-Hermitian boundaries. Although the imaginary in-gap state constitutes an edge mode, each individual imaginary in-gap state can be further explored to simulate impurity physics with novel non-Hermitian properties62,63. For example, by properly introducing Hubbard interaction U on the boundary, the corresponding 1D vertical chain model in Eq. (5) would bare similarities with the Wilson chain mapped from a finite U Anderson model. Hence, Kondo-like behaviors could emerge but be enriched by new features arising from the non-Hermiticity.

Although a non-Hermitian boundary with alternating gain and loss in Eq. (3) is studied as an example, as we show in Methods, the NHPE and the corresponding imaginary in-gap states remain intact for gain-only or loss-only boundaries as well. In addition, we also investigate a more general 2D insulator model with non-Hermitian boundaries. As shown in Methods, although the NHSE is absent in this model, the NHPE still persists, leading to imaginary in-gap states that are manifested by a localized edge mode shown in Fig. 5f (rather than the corner mode in Fig. 2d). The model independence indicates that the NHPE should be a more general non-Hermitian effect than NHSE.

Fig. 5: Generality of the non-Hermitian proximity effect (NHPE).
figure 5

a, b The real and imaginary parts of the spectrum for the model H = H0 + Hγ, where H0 is described by Eq. (2) which contains on-site mass terms mσz, and Hγ is described by Eq. (18) with the on-site non-Hermitian boundary terms iγσ0 located on the top layer of the system. The model obeys cylinder geometry with size in the y-direction being Ly = 50. The parameters used are m = 1.0 and γ = − 1.5. c The probability distribution of the imaginary in-gap states (IISs) under open boundary conditions (OBCs) with size Lx × Ly = 60 × 30, where the imaginary in-gap state (E = − 0.451 − 1.222i) is located at the corner due to the combined effects of the NHPE and the non-Hermitian skin effect (NHSE). The color bar on the right side of the panel indicates the value of the probability distribution under normalization. d–f Analogous to a–c, with H0 now being described by Eq. (19) and Hγ now being described by Eq. (3), where the parameters used are m = 1.0 and γ = 1.5. In f, the imaginary in-gap state (E = 0.746 + 0.464i) is an edge mode evenly distributed along the upper boundary, since only the NHPE exists in this model.

Full size image

The theoretical model studied here could be realized in different experimental platforms. For example, the Chern insulator model H can be realized by magnetically doped topological insulators64. Moreover, the non-Hermitian boundary terms can be realized on the basis of reservoir engineering. The loss or gain on the boundary can be achieved by using a nonlocal coupling to auxiliary degrees of freedom which undergo local loss or gain11, as discussed in detail in Methods. Moreover, the 2D topological insulating phases can also be realized by cold atoms in optical lattices or photons in coupled cavities65. In particular, in arrays of coupled micro-ring cavities, the photon gain and loss for each cavity can be controlled independently66,67. These provide promising platforms to further investigate the predicted NHPE, the imaginary in-gap states, as well as their dynamical behaviors.

Methods

Real-space renormalization group analysis of non-Hermitian proximity effect

For the open boundary perpendicular to the y-direction, the total system of a 2D lattice can be decomposed into a series of horizontal layers (labeled by l) with inter-layer coupling, and its partition function is then cast into \({{{{{{{\mathcal{Z}}}}}}}}=\int{\prod }_{l}{{{{{{{\mathcal{D}}}}}}}}{\bar{\psi }}_{l}{{{{{{{\mathcal{D}}}}}}}}{\psi }_{l}{e}^{-({S}_{l}+{S}_{l,l+1})}\), where

$${S}_{l}=-\mathop{\sum}\limits_{i{\omega }_{n}}\mathop{\sum}\limits_{{k}_{x}\in BZ}{\bar{\psi }}_{{k}_{x},i{\omega }_{n},l}[i{\omega }_{n}-{H}_{l}({k}_{x})]{\psi }_{{k}_{x},i{\omega }_{n},l},$$
(7)

where \({\psi }_{{k}_{x},i{\omega }_{n},l}={[{c}_{a,{k}_{x},i{w}_{n}},{c}_{b,{k}_{x},i{w}_{n}}]}^{T}\) is the Grassmann field. Hl(kx) is the Hamiltonian of the lth horizental chain. The action for the inter-layer coupling can be written as

$${S}_{l,l+1}=-\mathop{\sum}\limits_{i{\omega }_{n}}\mathop{\sum}\limits_{{k}_{x}\in BZ}({\bar{\psi }}_{{k}_{x},i{\omega }_{n},l}{T}_{y}{\psi }_{{k}_{x},i{\omega }_{n},l+1}+h.c),$$
(8)

where Ty is the matrix describing the interlayer hopping, which is assumed to be l-independent. Since the action is bilinear in terms of Grassmann fields, we can integrate out the first layer and obtain a renormalized effective action for the second layer as

$${S}_{2}=-\mathop{\sum}\limits_{i{\omega }_{n}}\mathop{\sum}\limits_{{k}_{x}\in BZ}{\bar{\psi }}_{{k}_{x},i{\omega }_{n},2}{G}_{2}^{-1}(i{\omega }_{n},{k}_{x}){\psi }_{{k}_{x},i{\omega }_{n},2},$$
(9)

where \({G}_{2}^{-1}(\omega ,{k}_{x})={\omega }^{+}-{H}_{2}({k}_{x})-{T}_{y}^{{{{\dagger}}} }{({\omega }^{+}-{H}_{1}({k}_{x}))}^{-1}{T}_{y}\) is the renormalized retarded Green’s function of the second layer with ω+ = ω + i0+. To extract the low-energy effective Hamiltonian, the low-frequency approximation can be made in \({G}_{l}^{-1}(\omega ,{k}_{x})\), which well preserves the low-energy physics as long as the layer integrated out remains gapped (which is indeed the case for the QWZ model studied here, where each layer describes a 1D Su-Schrieffer-Heeger (SSH) model). Then, the effective Hamiltonian of the second layer is read off as

$${H}_{2}^{eff}({k}_{x})={H}_{2}({k}_{x})-{T}_{y}^{{{{\dagger}}} }\frac{1}{{H}_{1}({k}_{x})}{T}_{y}.$$
(10)

By treating the renormalized second layer as the first layer on top of the remaining system, then the above procedure can be performed iteratively, leading to the effective action for the renormalized third, fourth... layers. After l RG steps, an iterative relationship between the effective Hamiltonian of the lth and that of the (l + 1)th layer can be obtained as

$${H}_{l+1}^{eff}({k}_{x})={H}_{l}({k}_{x})-{T}_{y}^{{{{\dagger}}} }\frac{1}{{H}_{l}^{eff}({k}_{x})}{T}_{y}.$$
(11)

Now we apply the real-space RG transformation to the QWZ model with a non-Hermitian boundary, the first layer Hamiltonian with non-Hermitian terms is given by

$${H}_{1}({k}_{x})=(\sin {k}_{x}+i\gamma ){\sigma }_{x}+(m-\cos {k}_{x}){\sigma }_{z},$$
(12)

which is a non-Hermitian version of the SSH model and exhibits a non-Hermitian skin effect18. We then derive the effective Hamiltonian for the renormalized lth layer under the real-space RG transformation. For any 2 × 2 matrix A, it can be expanded into a linear combination, A = aσ0 + bσx + cσy + dσz, where σx,y,z, and σ0 are the Pauli matrices and identity matrix, respectively. Then, we have

$${T}_{y}^{{{{\dagger}}} }{A}^{-1}{T}_{y}=\frac{a+b}{2{{{{{{{\rm{Det}}}}}}}}[A]}({\sigma }_{0}+{\sigma }_{x}),$$
(13)

with \({{{{{{{\rm{Det}}}}}}}}[A]={a}^{2}-({b}^{2}+{c}^{2}+{d}^{2})\). Using the Eqs. (10), (12), and (13), we have

$${H}_{2}^{eff}({k}_{x})=\sin {k}_{x}{\sigma }_{x}+(m-\cos {k}_{x}){\sigma }_{z}+{f}_{1}({\sigma }_{0}+{\sigma }_{x}),$$
(14)

where \({f}_{1}=(\sin {k}_{x}+i\gamma )/[2(2-2\cos {k}_{x}+2i\gamma \sin {k}_{x}-{\gamma }^{2})]\) is the self-energy due to the renormalization of the first layer. Accordingly, we can express the effective Hamiltonian of the (l + 1)th layer in the same form as Eq. (14), i.e., Eq. (4). Substituting it into the Eqs. (11) and (13), an iterative relation between fl+1 and fl can be obtained

$${f}_{l+1}=\frac{\sin {k}_{x}+2{f}_{l}}{2(2\sin {k}_{x}{f}_{l}-2m\cos {k}_{x}+{m}^{2}+1)}.$$
(15)

The fixed points, if exist, can be determined by requiring fl+1 = fl = fc for l → ∞. This leads to the following equation

$$4\sin {k}_{x}{{f}_{c}}^{2}-2m(2\cos {k}_{x}-m){f}_{c}-\sin {k}_{x}=0.$$
(16)

Since \(\Delta =4{m}^{2}{(2\cos {k}_{x}-m)}^{2}+16\sin {{k}_{x}}^{2}\geqslant \, 0\), the two roots fc1 and fc2 of Eq. (16) are given by

$${f}_{c1/c2}=\frac{2m(2\cos {k}_{x}-m)\pm \sqrt{\Delta }}{8\sin {k}_{x}},$$
(17)

which implies that the self-energy fl are real at the fixed points. This explains why the imaginary part of fl eventually vanishes for large l, as shown in Fig. 2b. In addition, we can prove that fl eventually flows to only one of the fixed points fc1. More subtle details can be found in the Supplementary Note 1.

The general existence of NHPE in gapped states with non-Hermitian boundaries

Although the non-Hermitian effects are derived from Eqs. (2) and (3), which describe a Chern insulator with a boundary that has alternating gain and loss (on a and b sites), the NHPE found here is in fact general. To demonstrate its generality, we first consider the non-Hermitian boundaries with only gain or only loss. Hence, we replace Eq. (3) by

$${H}_{\gamma }=i\mathop{\sum}\limits_{x}{c}_{x,1}^{{{{\dagger}}} }\gamma {\sigma }_{0}{c}_{x,1},$$
(18)

which describes the gain and loss on all the sites (y = 1) for γ > 0 and γ < 0, respectively. We then calculate the real and imaginary parts of the energy spectrum of the total system H = H0 + Hγ under cylinder geometry, which are respectively shown in Fig. 5a and b. As we see from Fig. 5a and b, two imaginary in-gap states still emerge when the only loss is present, similar to the results for the boundaries with alternating gain and loss. By comparing Fig. 5a and b with Fig. 1b, c, we find that the only slight difference is that the imaginary part of the eigenenergies of the upper chiral edge mode now becomes negative. This does not affect the main conclusions on the imaginary in-gap states and the NHPE.

The general existence of NHPE can be understood in the RG sense. The RG analysis discussed above is general and can be applied to gapped quantum systems with a generic non-Hermitian boundary. For instance, we consider Eq. (18) being added to the QWZ model and perform the real-space RG analysis. Similar to the RG analysis for the case with alternating loss and gain, the bulk Hamiltonian can be decomposed into different layers coupled via hopping matrix Ty. The key fact here is that the iterative relationship between the effective Hamiltonian of the lth and that of the (l + 1)th layer, i.e., Eq. (11), does not change as long as Ty remains the same. Thus, the effective Hamiltonian for the (l + 1)th layer can still be written as Eq. (4) which leads to the same result as Eq. (15). This indicates that the different non-Hermitian boundary terms only leads to different initial value f1, and the RG flow of fl remains qualitatively the same, independent of f1. Consequently, the same fixed point with \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]=0\) will always be reached for large l. This means that NHPE is general and does not depend on the form of non-Hermitian boundary.

Since the RG analysis shows that the non-Hermitian effect induced by the boundary would always decay and vanish for large distances from the boundary, the effects from different non-Hermitian boundaries would become independent from each other for systems larger than the decay length. Therefore, similar non-Hermitian phenomena are expected if non-Hermitian terms are considered on both the upper and lower boundaries.

To further support the RG results, we also calculate the typical spatial distribution of the imaginary in-gap state under OBCs. As shown in Fig. 5c, corner states still emerge for the gain-only or loss-only boundaries, as a result of the combined effect of the NHPE and NHSE, indicating the general existence of NHPE.

We now show that the NHPE is not only independent of the specific forms of the boundary but is also independent of the specific models of the insulating bulk. Instead of the QWZ model, we now consider a more general 2D two-band model with a bulk gap m described by the following Hamiltonian:

$${H}_{0}({k}_{x},{k}_{y})=\mathop{\sum}\limits_{i=x,y}(\cos {k}_{i}{\sigma }_{x}+\sin {k}_{i}{\sigma }_{y})+m{\sigma }_{z},$$
(19)

along with the non-Hermitian boundary term \({H}_{\gamma }=i{\sum }_{x}{c}_{x,1}^{{{{\dagger}}} }\gamma {\sigma }_{x}{c}_{x,1}\), i.e., Eq. (3). Then we calculate the energy spectrum under cylinder geometry. Fig. 5d, e show the real and imaginary part of the energy spectrum, which clearly indicate the emergence of the imaginary in-gap states, similar to those for the QWZ model.

As discussed above, the general existence of NHPE can be proved by real-space RG calculations. We, therefore, decompose the total system H = H0 + Hγ (where H0 is given by Eq. (19) above) into a series of coupled 1D horizontal layers. The Hamiltonian of the l = 1 layer reads as \({H}_{1,{k}_{x}}=(\cos {k}_{x}+i\gamma ){\sigma }_{x}+\sin {k}_{x}{\sigma }_{y}+m{\sigma }_{z}\) and the inter-layer hopping matrix is Ty = σ+. Then, we can derive the renormalized Hamiltonian for the (l + 1)th layer by iteratively integrating out the lth layer, leading to:

$${H}_{l+1,{k}_{x}}^{eff}=\cos {k}_{x}{\sigma }_{x}+\sin {k}_{x}{\sigma }_{y}+m{\sigma }_{z}+{f}_{l}({\sigma }_{0}-{\sigma }_{z}),$$
(20)

where complex self-energy fl encodes the non-Hermitian effect. Similarly, it can be proved that fl always saturates to the same fixed point with \({{{{{{{\rm{Im}}}}}}}}[{f}_{l}]=0\) for large l. This clearly suggests the boundary-induced non-Hermitian effect decays and finally vanishes deep in the bulk for large l. Since Eq. (19) is the minimal model of 2D insulators without any specific requirements, the NHPE found here is expected to be a general result for gapped quantum systems with a non-Hermitian boundary. This is also consistent with the emergence of imaginary in-gap states shown in Fig. 5d, e.

In particular, we mention that the NHSE is model-dependent and it does not occur in the above 2D insulator model. However, the NHPE remains intact. Correspondingly, the distribution of the imaginary in-gap state calculated under open boundaries no longer shows up as a corner mode, but is manifested by an edge mode evenly distributed along the upper boundary (Fig. 5f). This indicates that the NHPE could be a more general non-Hermitian effect than the NHSE.

Local density of states of the non-Hermitian vertical chain model

In the Hermitian systems, the DOS can be calculated from the imaginary part of the retarded Green’s function \(\rho (\epsilon )=-\frac{1}{\pi }\,{{\mbox{Im Tr}}}\,\left[{\sum }_{n}\frac{\left\vert {\psi }_{n}\right\rangle \left\langle {\psi }_{n}\right\vert }{\epsilon +i\eta -{E}_{n}}\right]\), where \(\eta ={0}^{+},H\left\vert {\psi }_{n}\right\rangle ={E}_{n}\left\vert {\psi }_{n}\right\rangle ,n=1,2,\ldots ,N\). For the LDOS at the lth site, we have

$${\rho }_{l}(\epsilon ) = -\frac{1}{\pi }\,{{\mbox{Im}}}\,\left[\mathop{\sum}\limits_{n}\frac{\langle l| {\psi }_{n}\rangle \langle {\psi }_{n}| l\rangle }{\epsilon +i\eta -{E}_{n}}\right]\\ = \frac{1}{\pi }\mathop{\sum}\limits_{n}\frac{\eta \langle l| {\psi }_{n}\rangle \langle {\psi }_{n}| l\rangle }{{(\epsilon -{E}_{n})}^{2}+{\eta }^{2}}\\ = \mathop{\sum}\limits_{n}\delta (\epsilon -{E}_{n})| \langle l| {\psi }_{n}\rangle {| }^{2}.$$
(21)

The last step in the above equation exploits the fact that the Dirac delta function is the limit of the Lorentzian function with η going to zero, and the final result goes back to the definition of the DOS. However, Eq. (21) no longer holds for non-Hermitian Hamiltonians, since En is now complex which makes it invalid. Thus, the way to calculate the LDOS has to be generalized to capture the non-Hermitian systems59,60. For a non-Hermitian Hamiltonian, the left or right eigenstates alone do not satisfy the orthogonality condition; both the left and right eigenstates should be used which satisfy the bi-orthogonal relation61, i.e., \(\langle {\psi }_{n}^{L}| {\psi }_{n}^{R}\rangle ={\delta }_{mn}\), where \(H\left\vert {\psi }_{n}^{R}\right\rangle ={E}_{n}\left\vert {\psi }_{n}^{R}\right\rangle ,{H}^{{{{\dagger}}} }\left\vert {\psi }_{n}^{L}\right\rangle ={E}_{n}^{* }\left\vert {\psi }_{n}^{L}\right\rangle ,n=1,2,\ldots ,N\). Incorporating it into Eq. (21), the LDOS at the lth site of the 1D vertical chain model can be expressed as

$${\rho }_{l}({\epsilon }_{r},{\epsilon }_{i})= \, \frac{1}{N}\mathop{\sum}\limits_{n}\delta ({\epsilon }_{r}-{{{{{{{\rm{Re}}}}}}}}[{E}_{n}])\delta ({\epsilon }_{i}-{{{{{{{\rm{Im}}}}}}}}[{E}_{n}])\\ \times | \langle l| {\psi }_{n}^{R}\rangle \langle {\psi }_{n}^{L}| l\rangle | ,$$
(22)

where ρl(ϵr, ϵi) is defined on the complex energy plane, where ϵr (ϵi) represents the real (imaginary) part of the complex energy ϵ. In our numerical calculations, since the 1D vertical chain model has sublattice degrees of freedom, N = 2Ly with Ly is the 1D chain length, and the phase factor \(\langle l| {\psi }_{n}^{R}\rangle \langle {\psi }_{n}^{L}| l\rangle\) includes the sum of the (2l − 1, 2l − 1) and (2l, 2l) matrix elements of the 2Ly × 2Ly matrix. Besides, in the plot of the LDOS, the δ-peaks are broadened by a finite width controlled by a factor b. This is achieved by treating the Dirac-δ function as a Lorentzian form \(\delta (\omega -{\omega }_{n})\to \frac{1}{2\pi }\frac{b}{{(\omega -{\omega }_{n})}^{2}+{b}^{2}}\).

Time-evolution of Gaussian wave packet

We set the initial state as a 2D Gaussian wave packet

$$\psi (t=0)=\frac{1}{{(4\pi {\sigma }^{2})}^{\frac{1}{2}}}\exp \left[-\frac{{(x-{x}_{0})}^{2}}{4{\sigma }^{2}}-\frac{{(y-{y}_{0})}^{2}}{4{\sigma }^{2}}\right]{(1,1)}^{T},$$
(23)

where (x0, y0) represents the center of the wave packet. According to the Schrödinger equation \(i{\partial }_{t}\left\vert \psi (t)\right\rangle =H\left\vert \psi (t)\right\rangle\), the time-evolution operator of the non-Hermitian system is expressed as

$$U(t)={e}^{-iHt}=\mathop{\sum}\limits_{n}{e}^{-i{E}_{n}t}\left\vert {\psi }_{n}^{R}\right\rangle \left\langle {\psi }_{n}^{L}\right\vert ,$$
(24)

which shows that those modes with \({{{{{{{\rm{Im}}}}}}}}[{E}_{n}] \, < \, 0\) will vanish due to the exponentially decaying factor, whereas those with \({{{{{{{\rm{Im}}}}}}}}[{E}_{n}] \, > \, 0\) will dominate in the long-time limit. For the model of the Chern insulator with a non-Hermitian boundary in our work, since the term Hγ contains an imaginary unit factor, the total Hamiltonian satisfies H*(γ) = H0 − Hγ = H0 + H−γ = H( − γ). So the sign of γ determines the sign of \({{{{{{{\rm{Im}}}}}}}}[{E}_{n}]\), which in turn determines whether the mode amplitude grows or decays.

Realization of the non-Hermitian boundaries via coupling to environments

In open quantum systems, the coupling between the system and the environment can be described by the Lindblad master equation, i.e.,

$$\frac{{{{{{{{\rm{d}}}}}}}}\rho }{{{{{{{{\rm{d}}}}}}}}t}=-i[H,\rho ]+\mathop{\sum}\limits_{\mu }\left(2{L}_{\mu }\rho {L}_{\mu }^{{{{\dagger}}} }-\left\{{L}_{\mu }^{{{{\dagger}}} }{L}_{\mu },\rho \right\}\right),$$
(25)

where ρ is the density matrix, Lμ’s are the Lindblad dissipators describing quantum jumps due to coupling to the environment. The short-time evolution is described by the Schrödinger evolution under the effective non-Hermitian Hamiltonian \({H}_{eff}=H-i{\sum }_{\mu }{L}_{\mu }^{{{{\dagger}}} }{L}_{\mu }\) with \({{{{{{{\rm{d}}}}}}}}\rho /{{{{{{{\rm{d}}}}}}}}t=-i({H}_{eff}\rho -\rho {H}_{eff}^{{{{\dagger}}} })\)43,68. Considering the single particle loss and gain with the loss and gain dissipators:

$${L}_{\mu }^{l} =\sqrt{{\gamma }_{l}}({c}_{\mu a}+{c}_{\mu b}),\\ {L}_{\mu }^{g} =\sqrt{{\gamma }_{g}}\left({c}_{\mu a}^{{{{\dagger}}} }+{c}_{\mu b}^{{{{\dagger}}} }\right).$$
(26)

Eq. (26) leads to the effective Hamiltonian, which reads in momentum space as

$${H}_{eff}=H+i({\gamma }_{g}-{\gamma }_{l})\left({\sigma }_{0}+{\sigma }_{x}\right).$$
(27)

The second term in Eq. (27) has a form that produces the non-Hermitian boundary studied in Eq. (3) and Eq. (18). The loss and gain dissipators in Eq. (26) can be realized in reservoir engineering by using nonlocal couplings to auxiliary degrees of freedom which undergo local loss or gain11.