Introduction

The study of topological phenomena arising from magnetically ordered spin systems that exhibit noncollinear and noncoplanar configurations has experienced rapid growth in recent years1,2,3,4. One notable example is the magnetic skyrmion5,6,7,8,9,10, a nanoscale vortex-like structure that is characterised by swirling spins with an integer skyrmion winding number, $${N}_{{Sk}}$$, defined as:

$${N}_{{Sk}}=\frac{1}{4\pi }\int {{{{{\bf{n}}}}}}\cdot \left(\frac{\partial {{{{{\bf{n}}}}}}}{\partial x}\times \frac{\partial {{{{{\bf{n}}}}}}}{\partial y}\right){dx}{dy},$$
(1)

which counts how many times the integrated solid angle between the neighbouring spin directions wrap a unit sphere. Here, $${{{{{\bf{n}}}}}}\left({{{{{\bf{r}}}}}}\right)={{{{{\bf{m}}}}}}\left({{{{{\bf{r}}}}}}\right)/\left|{{{{{\bf{m}}}}}}({{{{{\bf{r}}}}}})\right|$$ is the unit vector along the direction of the local magnetic moment $${{{{{\bf{m}}}}}}({{{{{\bf{r}}}}}})$$4,5. For an azimuthally symmetric skyrmion, one can write n$$\left({{{{{\bf{r}}}}}}\right)=\left(\cos \Phi \left({{{{{\bf{r}}}}}}\right)\sin f\left({{{{{\bf{r}}}}}}\right),\sin \Phi \left({{{{{\bf{r}}}}}}\right)\sin f\left({{{{{\bf{r}}}}}}\right),\cos f({{{{{\bf{r}}}}}})\right)$$ in spherical coordinates, with position vectors $${{{{{\bf{r}}}}}}=r\left(\cos {{\phi }},\sin {{\phi }}\right)$$ in polar coordinates. Here f(r) describes a polar angle, with f(r = 0) = π defining a skyrmion core moment alignment antiparallel with the magnetic field, and f(r → ∞) = 0 defining moments parallel with the field at the skyrmion periphery. When this Ansatz is inserted into Eq. 14,5, one obtains a definition of the skyrmion vorticity ω from the solutions of Φ(ø) alone, which for the considered boundary conditions f(r = 0) = π and f(r → ∞) = 0 is an integer such that NSk = −ω. Finally, if the moment azimuthal angle Φ varies smoothly with azimuthal direction ø of position r, it follows that

$$\Phi ({{\phi }})={\omega }\,{{\phi }}+{\chi }$$
(2)

where $${\chi }$$ is defined as the skyrmion helicity. For a common vorticity $${\omega }=1$$, values of $$\chi=\pm \frac{\pi }{2}$$; 0 or $$\pi$$ respectively correspond to Bloch- and Néel-type skyrmions, these being the two skyrmion types observed in the majority of experimental research to date. Representations for these skyrmion archetypes are depicted in Fig. 1a, b; we illustrate the skyrmion winding by displaying its form on a unit sphere, as well as the 2D projections, which show how the moments rotate either in a plane perpendicular (Bloch-type) or parallel (Néel-type) to the radial direction of the skyrmion. The uniform colour of the spheres and projections indicates the typical scenario where spatial anisotropy in $$\chi$$ is absent around the skyrmion. However, compared with standard Bloch- and Néel skyrmions, it can be anticipated that helicity anisotropies may give rise to hybrid skyrmions, which exhibit an anisotropic combination of Bloch- and Néel textures.

Despite these expectations, the experimental realisation for hybrid skyrmions is challenging. A key requirement is a non-standard pattern of antisymmetric interactions in the system, e.g. Dzyaloshinskii-Moriya interactions (DMIs), which are usually critical for the stabilisation of magnetic skyrmion phases. Since skyrmions normally form a magnetic crystal that modulates along multiple directions simultaneously—a so-called multi-Q structure7,11,12,13,14,15—the particular helicity of the magnetic texture imposed by the DMIs depends on the alignment of the magnetic propagation vectors Q with respect to the crystal6,16,17. When Q-vectors align with high crystal symmetry directions, the general DMI textures expected in structurally chiral magnets with T and O point group symmetries favour Bloch-type skyrmions, while in magnets with Cnv point groups Néel-type skyrmions are generally favoured. A departure from the Bloch and Néel archetypes is found in magnets of D2d or S4 symmetry, wherein antiskyrmion spin textures with alternating Bloch and Néel-type helicities have been observed in thin-plate and micron-sized samples18,19,20,21. However, due to an apparent lack of observation of antiskyrmions in bulk samples, it remains an open question if antiskyrmions can be considered as an intrinsic bulk phenomenon; their generally large size (>100 nm) implicates their stability and internal texture as governed by an interplay between anisotropic DMIs and sample thickness-dependent dipolar interactions. In synthetic magnetic multilayer systems hallmarked by two-dimensionality and highly tunable magnetic interactions, various evidence22,23,24 for hybrid spiral and skyrmion textures has been presented. Despite these examples, theoretical expectations25,26, and further experimental studies in various reduced dimension settings27,28,29,30, microscopic experimental evidence for a hybrid helicity of skyrmions as an intrinsic property of bulk magnets, and wherein sample thickness-dependent interactions play no role, has remained elusive.

One strategy to realise hybrid skyrmion formation in the bulk that has not been pursued to date, is to stabilise multi-Q textures with magnetic Q-vectors aligned with reduced symmetry directions, so that interactions promoting both Bloch- and Néel-type helicities can each be simultaneously active and compete to define new spin textures. In the following, we exploit this strategy and show a polar tetragonal magnet EuNiGe3 with C4v point symmetry (Fig. 1c) to be a remarkable host of two equilibrium hybrid skyrmion crystal phases. We construct the phase diagram (Fig. 1d) from magnetic, transport and neutron scattering measurements, and observe the material to display four distinct magnetic phases I to IV as a function of magnetic field below the Néel temperature. While the ground state phase I and highest-field phase IV display topologically trivial forms of incommensurate magnetic order, at intermediate fields we observe the direct magnetic field-induced transformation between a skyrmion crystal phase II (Fig. 1e and g), which displays a distorted hexagonal lattice structure, and phase III (Fig. 1f and h), which displays a square lattice structure. As seen in Fig. 1d, the adjacent hybrid skyrmion phases generate a strong enhancement of the magnetoresistivity. Microscopic evidence for an anisotropic combination of Bloch and Néel textures contributing to the hybrid skyrmion textures is provided by polarised neutron scattering, and backed up by numerical simulations. We explain both the stability of the hybrid skyrmion phases and the direct transition between them with an itinerant electron theory that invokes momentum-resolved exchange and antisymmetric Dzyaloshinskii-Moriya interactions that compete at the observed magnetic Q-vectors. The particular alignment of magnetic Q-vectors with the low symmetry directions of the polar lattice of EuNiGe3 is thus found to be a critical ingredient for the realisation of hybrid skyrmions in the bulk.

Results

To search for hybrid skyrmion textures, our target material is the 4f-electron compound EuNiGe3 with a polar tetragonal structure described by space group I4mm (No. 107) (Fig. 1c). The magnetic properties are governed by Heisenberg-like Eu2+ (S = 7/2, L = 0) ions, with both a spin density functional theory calculation and an earlier x-ray magnetic circular dichroism (XMCD) study31 indicating negligible moments due to Ni and Ge (<1% of Eu). Previous work shows the system transitions to an incommensurately modulated helical order below the Néel temperature, TN = 13.2 K32. The helical magnetic wavevector is confined within the tetragonal basal plane and found to be Q = (0.25, δ, 0), δ ~ 0.05 in reciprocal lattice units (r.l.u.)33,34, and thus a low-symmetry direction of the polar lattice. The application of a magnetic field H along the polar c-axis was found to induce several metamagnetic transitions33,34,35. However, details regarding the field-induced magnetic structures are either incomplete or missing34. Additionally, a non-monotonic H-dependence of the Hall resistivity was also found earlier and discussed in relation to topological Hall effects35 but without connection to the proper picture of the underlying magnetic order. Finally, in a recent preprint, observation of phase II is reported36; however, phase III is overlooked, and the hybrid skyrmion helicity and crystal coordination transitions representing the main messages of our paper are neither observed experimentally nor described theoretically.

To determine the magnetic phase diagram of our single crystalline EuNiGe3 samples, we performed bulk magnetic and transport measurements for magnetic fields applied along the polar c-axis (see Methods). Figure 2a, b respectively display typical H-dependent data at 2 K of the dc magnetisation (M) and the real part of the ac susceptibility (χac’) (see Supplementary Note I and Supplementary Fig. 1 for further details on the phase diagram determination by ac susceptibility). The M-H and χac’-H curves each show multiple step-like transitions prior to reaching the saturated state with M = 7.85 μB / Eu2+ at 6 T, and we identify four distinct phases I–IV. In both the M-H and χac’-H data, we observe hysteresis between H-increasing and H-decreasing sweeps, in addition to clear cusps in χac’, at the boundaries of phases II and III. These features infer both phases II and III to be equilibrium phases that each host a complex magnetic order.

From further H-dependent bulk measurements at higher temperatures, including neutron measurements discussed later, the μ0H-T phase diagram for μ0H || c was constructed (Fig. 1d). The phase space occupied by both phases II and III coincides precisely with where we observe both a large enhancement of the magnetoresistance ratio (MR), defined as $$\left[{\rho }_{{xx}}\left({\mu }_{0}H\right)-{\rho }_{{xx}}(0)\right]/{\rho }_{{xx}}(0)$$ (see Methods). In addition, Fig. 2c shows anomalies in Hall resistivity ρyx to accompany the μ0H-dependent magnetic transitions. Although the non-linear, non-monotonic μ0H-dependence of ρyx evades a conventional interpretation (see Supplementary Notes II and III, and Supplementary Fig. 2 for the detailed discussion of the ρyx data), the data nonetheless suggest an intimate relationship between magnetic order and electrical transport properties.

In this context, we also performed out-of-plane rotation measurements of the Hall resistivity as H was rotated by an angle φ with respect to the c-axis in the a–c plane. Here I || a, and φ = 0° corresponds to μ0H | |c. Figure 2d shows the normalised Hall resistivity ρyx(φ)/ρyx(φ = 0) obtained at 2 K, and magnetic fields appropriate for each phase. In phases I (1.5 T) and IV (3.5 T), ρyx(φ) shows a near cosinusoidal variation with φ and negligible hysteresis. This indicates M to closely follow H(φ), and that the main contributions to ρyx are due to the normal and conventional anomalous Hall resistivities. In contrast, for phases II (2.6 T) and III (3.2 T), ρyx(φ) displays abrupt anomalies as the field is tilted beyond φ  ±10-15° away from either c or –c. These features denote the occurrence of first-order magnetic phase transitions, and are hallmarked by a hysteretic range of the response in φ that spans up to 30°. In analogy with the more pronounced hysteretic features in ρyx(φ) observed in Gd2PdSi337, these data indicate each of the phases II and III in EuNiGe3 to host spin textures with a distinct topology compared with phases I and IV.

To clarify the magnetic order in each phase we performed single crystal small-angle neutron scattering (SANS) measurements. SANS is generally sensitive to magnetic order if the Fourier transform of the local magnetisation density contains incommensurately modulating spin components $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)\exp \left[i{{{{{\bf{Q}}}}}}\cdot {{{{{\bf{r}}}}}}\right]+{{{{{\rm{c}}}}}}.{{{{{\rm{c}}}}}.}$$ where components of $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ are normal to Q. Here $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ is a complex vector and c.c. denotes complex conjugate. Figure 3a illustrates the transmission geometry of the experiment, where H was applied parallel to both [001] and the neutron beam (see Methods for full details). Typical SANS data summarising the H-evolution of the magnetic order at 1.9 K during the H-increasing process are shown in Fig. 3b–h. Starting in the ground state phase I, the SANS data shown in Fig. 3e agree with previous works33,34. The pattern displays a four-fold symmetry due to the existence of four magnetic domains described by low symmetry Q-vectors equivalent to Q1 = (0.25, δ, 0), δ  0.05. Note that for each Q-vector, scattering appears at both ±Q, leading to the eight reflections observed in total (Fig. 3i). From the observed magnitude of |Q1 | 3.71 nm−1, we deduce the modulation period of 1.69 nm. The earlier allocation of phase I as being a multi-domain single-Q helically-ordered phase33,34 is broadly consistent with the present SANS data, and our further SANS data obtained in an in-plane magnetic field with μ0H | | [100] (see Supplementary Note IV, and Supplementary Fig. 3).

From the ground state phase I, the H evolution of the magnetic order manifests in wholescale changes in the observed SANS patterns. Figure 3f shows data obtained at 2.6 T that are typical for phase II. Twelve diffraction spots are observed which may each be described by one of three distinct types of Q-vector, Q2 (0.217,0.065,0), Q3 (0.147,−0.147,0), and Q4 (−0.065,−0.217,0) (Fig. 3j). We interpret the phase II SANS data to correspond to two equivalent domains of a triple-Q magnetic order that each break the four-fold symmetry, and thus can be converted into one another by a 90° rotation around the polar axis (Fig. 3j), with this allocation supported by further SANS measurements done in magnetic fields tilted from the c-axis (see Supplementary Note V, and Supplementary Fig. 4). Further increase of H leads directly to phase III which we find also hosts a multi-Q structure. Figure 3g shows the typical SANS pattern for this phase at 3.2 T. Sixteen Bragg spots are observed that can each be described by two distinct types of short and long Q-vector. As shown in Fig. 3k, the shorter Q-vectors include Q5 (0.15,−0.10,0) and Q6 (0.10,0.15,0) and other equivalents. Mutually orthogonal wavevectors such as Q5 and Q6 describe a square-coordinated multi-Q lattice, and two such domains co-exist that are tilted from one another in the tetragonal plane by 25°. The longer Q-vectors, which include Q7 (0.25,0.05,0) and other equivalents, satisfy relations such as Q7 = Q5 + Q6. We thus conclude these peaks amount to higher-order reflections, a telltale signature of a multi-Q structure38,39,40. Finally, in phase IV, a typical SANS pattern obtained at 3.5 T is shown in Fig. 3h. These data can be described similarly as phase I with Q8 (0.25,0.05,0) (Fig. 3l), and are consistent with phase IV hosting multi-domain single-Q magnetism. While the data at hand do not allow us to rule out phase IV also being multi-Q, e.g. double-Q, we note that features in our bulk data indicating non-trivial spin textures are absent. In stark contrast, multi-Q phases II and III display clear features in both bulk and electrical transport properties that are suggestive for topological spin textures, and therefore that each hosts a form of magnetic skyrmion lattice.

A quantitative analysis of the SANS data summarised in Fig. 3b–d, and Supplementary Fig. 5, demonstrate the first-order character of the field-driven transitions between the different phases. Sharp H-dependent changes are observed in SANS intensity when moving between each phase (Fig. 3b), the magnitudes of the various Q-vectors (Fig. 3c), and the skyrmion density $${n}_{{\tt{Sk}}}$$ relevant for phases II and III (Fig. 3d). In phase II, since $$\left|{{{{{{\bf{Q}}}}}}}_{{{{{2}}}}}\right|=\left|{{{{{{\bf{Q}}}}}}}_{{{{{4}}}}}\right| > \left|{{{{{{\bf{Q}}}}}}}_{{{{{3}}}}}\right|$$, the triple-Q skyrmion lattice is not perfectly hexagonally coordinated; instead it is distorted in momentum space along the direction orthogonal to Q3. The real-space modulation of this skyrmion lattice thus varies in the tetragonal plane from 1.92 nm due to |Q2| and |Q4| , and 2.10 nm due to |Q3| . The reciprocal lattice distortion can be parameterised by the axial ratio of an ellipse that overlays the six SANS spots making up each single triple-Q domain (see Supplementary Note VI and Supplementary Fig. 6). We find the ellipticity of the Q-vector distribution in phase II to vary from 1.1 to 1.15 as the field increases, leading to the direct implication that in real space the skyrmions in phase II are elongated elliptically along the direction of Q3. The regular square-lattice coordination in phase III indicates the skyrmions in this phase to display a real-space modulation of 2.30 nm. We note finally that phases II and III each display skyrmion densities $${n}_{\tt{Sk}}$$ in the region of 0.2~nm−2 (Fig. 3d), which are amongst the highest for 4 f electron magnets hosting skyrmions15,37,38,41,42,43.

Finally, we employed a polarised SANS setup as illustrated in Fig. 4a to investigate the spin orientation of the magnetic order in phases I to III in more detail (see Methods for further experimental details). In the experiment the incident neutron spin polarisation was aligned either parallel (Pi) or antiparallel (–Pi) to the incoming beam, and also H || c. The neutron spin orientation after the scattering process was analyzed, and the intensities of non-spin-flip (NSF) and spin-flip (SF) scattering processes measured separately at each Bragg peak. In accord with Fig. 4b, magnetic NSF scattering arises from components of $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ parallel Pi and normal to Q-vector, while SF scattering arises from components of $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ simultaneously normal to both Pi and Q44. These selection rules lead to straightforward expectations; for either a proper-screw helix (the components of $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ all lie strictly perpendicular to Q) or a pure Bloch skyrmion crystal phase, equal weights of NSF and SF scattering are expected at each Q-vector. For an easy-axis cycloidal texture (the components of $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ lie in the Q-c plane) or a pure Néel-type skyrmion crystal, only NSF scattering is expected at each Q, with SF scattering entirely absent. As shown in Fig. 4c–g, and summarised in Table 1, each of the symmetry-distinct Q-vectors in phases I–III always show both NSF and SF scattering with varying weights. These data prove directly that the magnetic structures in each of the phases I to III include both in-plane and out-of-plane spin components normal to the Q-vectors, with the latter a notable requirement for any form of skyrmion phase.

The result that arises directly from the polarised SANS data is that the varying weights of NSF and SF scattering at the different Q-vectors rule out proper-screw helix and cycloidal spin textures for phase I, as well as conventional Bloch- and Néel-type multi-Q skyrmion crystals for phases II and III. While unique determinations of the spin components $$\hat{{{{{{\bf{m}}}}}}}\left({{{{{\bf{Q}}}}}}\right)$$ cannot be achieved with the data at hand, the observed Q-dependent weights of NSF and SF scattering are a direct microscopic signature of anisotropy in the internal spin texture expected generally for hybrid skyrmions in phases II and III. As also seen in Table 1, the data further provide a quantitative benchmark against which we compare the expectations of our theoretical analysis discussed later, and from which we suggest the reconstructed skyrmion textures for phases II and III shown in Fig. 1e, f.

Our main experimental observations that require theoretical explanation are firstly the origin of the magnetic Q-vectors observed in EuNiGe3, secondly the direct transition between the skyrmion phases II and III, and finally the origin of the Q-dependent weights of NSF and SF polarised SANS scattering, since this provides the key insight for suggesting the anisotropy of the internal skyrmion textures.

First, we discuss the microscopic origin of the propagation vector Q = (1/4,±δ,0) in EuNiGe3. By utilising spin density functional theory calculations, we derive a classical spin model composed of Eu spins through the local force method45 (see Methods). Figure 5a presents a Fourier transform of the spin-spin interaction, J(q), on the qz = 0 plane. We observe two peaks at Q = (1,0,0) and Q (1/4,0,0), both indicating the presence of an inherent spin modulation instability in EuNiGe3. Although the heights of these two peaks are comparable, we focus our discussion on the peak at Q (1/4,0,0) as it appears to be a modulation vector close to that observed in the experiment. To investigate the origin of the Q (1/4,0,0) peak, we decompose the total spin-spin interaction J(q) into its orbital contributions due to Eu atoms46. From the orbital-decomposed J(q), we find that the primary contribution arises from the Eu-5d orbitals, which is similar to Gd-based skyrmion compounds46,47. In particular, the Eu-t2g orbitals give rise to a peak at Q (1/3,±δ,0) as depicted in Fig. 5b, which moves to Q (1/4,0,0) by including the other sub-dominant contributions. Figure 5c illustrates the band structure of EuNiGe3 calculated by assuming the paramagnetic state in the conventional unit cell. Although many bands intersect the Fermi level, we can see that band near the Γ-point in band 23 is predominantly formed by the t2g orbital. The corresponding Fermi surfaces are drawn in Fig. 5d. Interestingly, the aforementioned band dominated by the t2g orbital forms a square-box shape Fermi surface, which possesses a nesting vector Q (1/3,0,0) as indicated by the black double-headed arrow. These results imply that this Fermi surface is responsible for the magnetic modulation observed in EuNiGe3. It is noteworthy that the 5d orbital plays a crucial role in the magnetic modulation, like in the well-studied short-pitch skyrmion compound GdRu2Si238,46,48. Therefore, we expect that the generalised Ruderman–Kittel–Kasuya–Yosida (RKKY) interaction, rather than the conventional RKKY interaction, acts as a driving force behind the formation of the observed skyrmion lattices.

Next, we turn towards obtaining an understanding for the H-driven transition between the skyrmion phases II and III, and the origin of their anisotropic internal magnetisation textures. Due to the non-centrosymmetric polar crystal structure and itinerant nature of the 4f magnetism in EuNiGe3, RKKY interactions, DMIs, and higher-order exchange interactions can all in principle contribute to stabilising multi-Q skyrmion phases. To understand our observations, we studied the following classical Heisenberg model given by49,

$${{{{{\mathcal{H}}}}}}{{{{{\mathscr{=}}}}}}{{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{ex}}}}}}}+{{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{DM}}}}}}}+{{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{Z}}}}}}}$$
(3)

where

$${{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{ex}}}}}}} =-\mathop{\sum}\limits_{v}{J}_{{{{{{{\bf{Q}}}}}}}_{v}}\left[\left({S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{x}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{x}+{S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{y}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{y}+{\alpha }_{{{{{{{\bf{Q}}}}}}}_{v}}{S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{z}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{z}\right)\right. \\ \quad \left.+{\beta }_{{{{{{{\bf{Q}}}}}}}_{v}}\left({S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{x}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{x}-{S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{y}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{y}\right)+{\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}}\left({S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{x}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{y}+{S}_{{{{{{{\bf{Q}}}}}}}_{v}}^{y}{S}_{{{{{{{\boldsymbol{-}}}}}}{{{{{\bf{Q}}}}}}}_{v}}^{x}\right)\right]$$
$${{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{DM}}}}}}} =-\mathop{\sum}\limits_{v}{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{v}}\cdot \left({{{{{{\bf{S}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{v}}\times {{{{{{\bf{S}}}}}}}_{-{{{{{{\bf{Q}}}}}}}_{v}}\right)$$
$${{{{{{\mathcal{H}}}}}}}_{{{{{{\rm{Z}}}}}}} =-{H}^{z}\mathop{\sum}\limits_{i}{S}_{i}^{z}.$$

Here $${{{{{{\bf{S}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{v}}$$ is the Fourier transform of the classical localised spin $${{{{{{\bf{S}}}}}}}_{{{{{{\bf{i}}}}}}}$$ with wave vector $${{{{{{\bf{Q}}}}}}}_{v}$$, where v is the label of the dominant ordering vectors. Parameters $${\alpha }_{{{{{{{\bf{Q}}}}}}}_{v}}$$, $${\beta }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ and $${\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ have their physical origin in the RKKY interaction and spin-orbit coupling as described in refs. 17,50, and thus in general depend on the electronic structure. The first term in Eq. 3 represents the symmetric exchange interaction and the second term corresponds to the antisymmetric DM-type exchange interaction. The third term is the Zeeman Hamiltonian, which introduces the effect of an external magnetic field. Contributions from higher-order exchange interactions are neglected.

To simplify the model, we extracted the dominant momentum-resolved interactions at the experimentally observed Q points reported in Fig. 3 and Supplementary Fig. IV. We take into account the interactions in momentum space by supposing global minima at, for example, Q1 and symmetry-equivalent wavevectors, for example, Q’1 Q1. Phases I and IV are described by single-Q spiral states, and their dominant interactions are determined by specific wave vectors along the directions ± Q1. For the distorted hexagonal skyrmion lattice (SkL) in phase II, the relevant ordering vectors are Q2 to Q4. Accordingly, for the square SkL in phase III, the high symmetric wave vectors are Q1 ( ≡ Q7) and Q’1. The shorter Q-vectors are defined as Q5 and Q6, so that Q7 = Q5 + Q6. We neglect the contributions from the other Q components in the interactions for simplicity. We set the interactions to be $${J}_{{{{{{{\bf{Q}}}}}}}_{1}}={J}_{{{{{{{\bf{Q}}}}}}}_{2}}/{\kappa }_{1}={J}_{{{{{{{\bf{Q}}}}}}}_{3}}/{\kappa }_{1}={J}_{{{{{{{\bf{Q}}}}}}}_{4}}/{\kappa }_{1}={J}_{{{{{{{\bf{Q}}}}}}}_{5}}/{\kappa }_{2}={J}_{{{{{{{\bf{Q}}}}}}}_{6}}/{\kappa }_{2}$$, $$D=|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{1}}|=|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{5}}|/{\kappa }_{2}=|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{6}}|/{\kappa }_{2}$$, $${D{{\hbox{'}}}}=|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{3}}|$$ and $$|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{2}}|=|{{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{4}}|=0$$; we also appropriately set the other parameters of $$({\alpha }_{{{{{{{\bf{Q}}}}}}}_{v}},{\beta }_{{{{{{{\bf{Q}}}}}}}_{v}},{\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}})$$ to reproduce the phase sequence observed in experiments. In particular, we introduce $${\beta }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ and $${\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ only for Q2 and Q4 as a key ingredient for deriving the hybrid hexagonal skyrmion crystal50 (see Supplementary Note VII for full details of the parameters used in the simulation). Here, κ1 and κ2 describe the relative amplitudes of competing exchange and DMIs. Hereafter, we fix JQ = 1, D = 0.2, D’ = 0.03 κ1, κ1 = 0.935, and κ2 = 0.85. The model Hamiltonian described with Eq. 3 was minimized using the iterative simulated annealing procedure and a single-step Monte-Carlo dynamics with the Metropolis algorithm49 (see Methods). The numerically obtained spin configurations are distinguished by spin- and chirality-related quantities described in Supplementary Note VIII.

Figure 6 shows the magnetic field-dependence of the stable spin textures resulting from the simulations for H parallel to the polar axis. Consistent with the experimental results, we find four distinct modulated phases are stable before reaching saturation. Figure 6a–d show the simulated real-space spin textures deduced for each phase that are respectively consistent with phases I to IV, with the corresponding momentum space distribution of the spin structure factors shown in Fig. 6e to h. The stable spin textures for phases I and IV are found to correspond respectively to single-Q tilted-helical and conical spiral phases, while those for phases II and III correspond to multi-Q skyrmion lattices. The simulation further suggests the competing stabilisation mechanisms of skyrmion phases II and III. While the distorted-hexagonal phase II is stabilized by the interplay between the easy-axis interaction $${\alpha }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ and the symmetric anisotropic exchange interactions $${\beta }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ and $${\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}}$$, the square skyrmion phase III is stabilized by the DMI $${{{{{{\bf{D}}}}}}}_{{{{{{{\bf{Q}}}}}}}_{v}}$$. Figure 7 shows the H-dependence of the magnetisation along the polar axis Mz and the square of the scalar spin chirality $${\left({\chi }_{0}\right)}^{2}$$, with the latter becoming finite only in phases II and III, consistent with these phases hosting skyrmions.

In addition to reproducing the experimentally observed H-dependent sequence of phase transitions, the simulation results also provide insight for the origin of Q-dependent weights of NSF and SF polarised SANS scattering in skyrmion phases II and III, and ultimately the hybrid internal texture of the skyrmions. The phenomenon for hybrid skyrmions has its roots in the assumption that the DMIs in the spin model can be described generally as a superposition of multiple types that favour different windings17. In the normal case, where spin textures modulate with high-symmetry Q-vectors, the DMIs are polar (namely perpendicular to both Q and the polar axis) and result in a conventional Néel-type SkL. However, for spin textures with low-symmetry Q-vectors, such as in EuNiGe3, the DMIs can be a superposition of polar and radial types, the latter favoring proper-screw spirals but different from pure chiral-type DMIs since its direction depends on the precise orientation of the wavevector.

According to our simulations, in phase II, the presence of superposed polar and radial DMIs at Q2 and Q4 hinders the formation of a conventional hexagonal Néel-type SkL. Simultaneously, symmetric anisotropic exchange interactions arising from $${\beta }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ and $${\gamma }_{{{{{{{\bf{Q}}}}}}}_{v}}$$ favour the Bloch-type SkL50. This leads to competition between the Néel-type and Bloch-type SkLs, resulting in an anisotropic texture combining the two types. The square SkL phase III is also a hybrid of Néel and Bloch types due to a helicity competition caused by the superposition of different DMI types, without contributions from symmetric anisotropic exchange interaction. By analysing the stable spin textures expected from the simulations, we obtained the ratio of NSF to SF scattering expected at each Q-vector which can be compared directly with the polarised SANS data. As shown in Table 1, the model expectations capture the experimental result generally well, particularly in phases II and III. While fine-tuning of the model parameters and/or consideration of further interactions may further improve the agreement between the theory and the experimental data, our main insight remains valid in general; hybrid skyrmion crystals can arise in the presence of a Q-dependent competition between interactions favouring different helicity spin textures.

Discussion

In Fig. 1e, f, we visualise the skyrmion textures in phases II and III resulting from the simulations. The hybrid aspect of the internal textures is visible in the contrast between the spatial distributions of the calculated helicity parameter χ of phases II and III, and those expected for pure Bloch (Fig. 1a) and Néel (Fig. 1b) skyrmions. From a quantitative analysis of the calculated χ values for each skyrmion type (see Supplementary Note IX for the full details), we describe the skyrmions in phase II to show a ‘weak’ Bloch-type winding, with certain directions related to Q2 and Q4 showing a stronger Bloch character than the direction related to Q3. Phase III is hallmarked by a generally weak Néel character near the core, with a noticeable in-plane azimuthal variation between Bloch and Néel character at larger radial distances. Since conduction electrons can couple to noncoplanar spin structures through the Berry curvature, the anisotropy of spin noncoplanarity displayed by hybrid skyrmion structures should be detectable in various emergent electrodynamic responses51, particularly when the direction of current flow is varied in the tetragonal plane. Although the coupling between the conduction electrons and the magnetic structure appears to be weak in EuNiGe3 (see Supplementary Note III), our results suggest a possible pathway to finding unique transport responses in other hybrid skyrmion host systems with a stronger coupling.

A second unique aspect of our observations is the direct transition between hexagonal and square equilibrium skyrmion crystal phases, which previously in bulk materials was observed only under far-from-equilibrium conditions52,53. Visualisations of the skyrmion crystal phases we observe in EuNiGe3 are shown in Fig. 1g, h. Apart from EuAl4, which displays a less drastic transition between double-Q square and rhombic skyrmion phases with slightly different mutual orientations of the fundamental Q-vectors42, the majority of 4f-electron skyrmion systems host a single skyrmion phase with a lattice structure that reflects the underlying crystal symmetry37,38,41,43. In EuNiGe3, our theoretical analysis indicates the stability of the ground state magnetic order with low symmetry Q-vectors as most likely determined by the Fermi surface structure, while the H-tuned competition between various competing interactions at the skyrmion crystal Q-vectors is responsible for the transition between skyrmion crystals that each spontaneously and differently break the crystal symmetries. The present study thus provides not only a demonstration of an externally driven and drastic reconfiguration of equilibrium skyrmion crystals hitherto unique to bulk magnets, it also hints at the associated materials principles and magnetic interaction landscapes that can both support and provide control over such phenomena. Possible ways forward for the latter include precise tuning of the chemical composition, or uniaxial strain engineering. Finally, the effects we have uncovered experimentally may be exploited in principle for various applications, for example, those involving a dynamic reconfiguration of skyrmion-based magnonic crystals.

Methods

Crystal growth and characterisation

Single crystals of EuNiGe3 were grown by the In-flux method. The phase purity and crystallographic parameters of single crystal samples were confirmed by single crystal X-ray diffraction at room temperature using a STOE STADIVARI four-circle diffractometer equipped with a Dectris EIGER 1 M 2 R CdTe detector and incoming micro-focused Ag Kα radiation of wavelength 0.56083 Å. Data reduction and correction were performed using the X-Area package54. Solution, refinement, and absorption correction were done utilizing the JANA 2020 software (version 1.3.51)55. Good agreement between measured and refined intensities of over 331 reflections (184 unique) confirmed our crystals to display the nominal occupancy of the different atoms within uncertainty, and tetragonal lattice parameters of a = 4.341(1) Å and c = 9.896(4) Å, consistent with those reported previously32,33,34. Since our bulk magnetic measurements on the crystals reveal characteristic transitions to take place at temperatures and magnetic fields consistent with those reported previously32,33,34,35, we conclude our crystals to be comparable in terms of quality to those studied in earlier work. For neutron scattering and transport measurements, crystal orientations were determined by Laue X-ray diffraction, and then the samples were cut out from the crystal with a wire saw and mechanically polished.

Magnetisation, ac susceptibility and transport measurements

Magnetisation was measured using a SQUID magnetometer (Magnetic Property Measurement System, Quantum Design). Ac susceptibility measurements were performed using commercial magnetometers from Quantum Design (QD). The excitation field was kept constant at 1 Oe, while the two excitation frequencies were used for all measurements, 911 Hz and 917 Hz. The longitudinal ($${\rho }_{{xx}}$$) and Hall ($${\rho }_{{yx}}$$) resistivities were measured using the resistivity option or ac-transport option in a physical property measurement system (PPMS, Quantum Design). Measurements were performed on crystls with dimensions of 1 mm × 0.9 mm × 0.5 mm (for magnetisation and longitudinal resistivity) and 1 mm × 0.5 mm × 0.1 mm (for Hall resistivity), with the widest surface parallel to the (001) plane. For the transport measurements with out-of-plane magnetic field rotation, the single crystal was cut into a lamella with a thickness of about 500 nm to ensure sufficient current density and data quality by using focused-ion-beam (FIB) machining. The largest surface was parallel to the (001) plane while the electric current was applied along the [100] axis.

Small-angle neutron scattering (SANS) measurements

Unpolarised SANS measurements were carried out using the SANS-I instrument at the Swiss Spallation Neutron Source (SINQ), Paul Scherrer Institute, Switzerland. The SANS data were obtained from 5 mg single crystals of typical volume 2 × 2 × 0.15 mm3. The widest (001) plane was aligned initially along the neutron beam. The crystal was mounted inside a superconducting cryomagnet providing temperatures down to 2 K, and magnetic fields up to 6.8 T along the c-axis. The neutron wavelength was selected to be 3.1 Å (Δλ/λ = 17 %). The incoming beam was collimated over 3 m before the sample, and scattered neutrons collected by a multi-detector placed 1.85 m behind the sample. The momentum-space resolution in the region of the observed Q-vectors was 0.06 Å−1. The SANS diffraction patterns like those in Fig. 3 were obtained by summing together two-dimensional multi-detector measurements taken over a focused range of sample + cryomagnet rotation (rocking) angles close the Bragg conditions for the various diffraction peaks. Background data acquired above TN was subtracted from the data at low temperature to leave only the magnetic signal. All magnetic field scans were conducted after zero-field cooling (ZFC) and during an H-increasing process.

Polarised SANS measurements on the same samples were carried out using the D33 instrument at the Insitut Laue-Langevin (ILL), France56. The instrument was operated in a time-of-flight mode that passed a wavelength band of 2–14 Å neutrons. A supermirror polariser was use to produce a polarised neutron beam with neutron spins aligned transverse to ki. A longitudinally polarised beam was obtained by using guide fields, with the neutron spin polarisation at the sample position directed to be either parallel or antiparallel to the c-axis by using a spin flipper. The sample was positioned inside a horizontal-field magnet that could apply a maximum magnetic field up to 3.0 T (thus precluding the study of phase IV). A cryostat insert provided sample temperatures down to 2 K. A nuclear spin-polarised 3He analyzer cell was placed between the sample and the detector. Its polarisation direction could be also flipped to be either parallel or antiparallel to the direction of the incoming beam polarisation, and thus allow distinction between SF and NSF scattering signals. The magnetic scattering at specific Q-vectors in phases I to III was observable at specific scattering angles and selected neutron wavelengths within the 3-4 Å portion of the wavelength spectrum with Δλ/λ  10 %. The incoming beam was collimated over a distance of 7.8 m before the sample, and scattered neutrons collected by a multi-detector placed 1.9 m behind the sample. In the defined wavelength band, the momentum-space resolution in the region of the observed Q-vectors was 0.03 Å−1. The polarising efficiency of the setup, characterised by the flipping ratio, was measured and found to vary from 6-8 depending on the wavelength and H. By measuring all possible neutron-spin-state combinations of the unscattered beam, fully quantitative corrections for imperfections of the overall polarising efficiency are applied in the data analysis. All unpolarised and polarised SANS data analysis was carried out using GRASP software57.

Theoretical calculations

Spin density functional theory (SDFT) calculations for EuNiGe3 were performed based on the projector augmented wave scheme58 implemented in the Vienna ab initio simulation package59,60. The lattice constants and the atomic positions were taken from the experiment61. The exchange correlation functionals proposed by Perdew-Burke-Ernzerhof 62, Ecut = 450 eV as the cutoff energy for the plain wave basis set, and Nk = 10 × 10 × 10 as the number of k-points were used in the self-consistent field calculations. For the calculations of the paramagnetic state, we used the open core potential with 4f7 configuration for Eu element. To illustrate the Fermi surfaces with dense k-grid, we used the Wannier interpolation technique by using the WANNIER90 package63,64. Here, 182 Bloch bands on the 8 × 8 × 8 k-grid were taken into account to obtain a 52-orbitals tight-binding model, where the Eu-d, Ni-s, p, d and Ge-s, p orbitals were used as the trial localised orbitals. Based on the obtained tight-binding model, the Fermi surfaces are drawn by FermiSurfer package65 with the eigenvalues evaluated on 96 × 96 × 96 k-grid. For the calculations of spin-spin interaction J(q), we used the Wannier interpolation for the ferromagnetic solution of the SDFT + U calculations, where U = 6.7 eV and J = 0.7 eV were added into the Eu-4f orbital. Here, 184 Bloch bands on the 8 × 8 × 8 k-grid were used to obtain a 66-orbitals tight-binding model, where the trial orbitals of Eu-d, f, Ni-s, p, d and Ge-s, p were used. The total and orbital resolved spin-spin interactions J(q) were then calculated by the local force method as implemented in refs. 45,46,47. Here, Nk = 36 × 36 × 36 as the number of k-points and T = 20 K were used in the calculations.

The numerical simulations of the magnetic phase stabilities were done on a classical spin model with contributions from symmetric exchange, DMI, and Zeeman energies. The magnetic energy density has been minimised using the iterative simulated annealing procedure and single-step Monte-Carlo dynamics with the Metropolis algorithm. We imposed periodic boundary conditions and performed simulations for lattices of different sizes to check the stability of the numerical routine.