## Abstract

The entanglement of the spin and orbital degrees of freedom through the spin-orbit coupling has been actively studied in condensed matter physics. In several iridium oxide systems, the spin-orbital entangled state, identified by the effective angular momentum *j*_{eff}, can host novel quantum phases. Here we show that a series of lacunar spinel compounds, GaM_{4}X_{8} (M=Nb, Mo, Ta and W and X=S, Se and Te), gives rise to a molecular *j*_{eff} state as a new spin-orbital composite on which the low-energy effective Hamiltonian is based. A wide range of electron correlations is accessible by tuning the bandwidth under external and/or chemical pressure, enabling us to investigate the cooperation between spin-orbit coupling and electron correlations. As illustrative examples, a two-dimensional topological insulating phase and an anisotropic spin Hamiltonian are investigated in the weak and strong coupling regimes, respectively. Our finding can provide an ideal platform for exploring *j*_{eff} physics and the resulting emergent phenomena.

## Introduction

Spin-orbit coupling (SOC) is a manifestation of Einstein’s theory of relativity in condensed matter systems. Recently, SOC has attracted a great deal of attention since it is a main ingredient for spintronics applications^{1,2}, induces novel quantum phases^{3,4} and generates new particles and elementary excitations^{5,6}. Moreover, when incorporated with electron correlations, SOC can give rise to even more fascinating phenomena^{7,8}. In the iridium oxide family, where the IrO_{6} octahedron is the essential building block, various quantum phases have been predicted or verified according to the electron correlation strength on top of the large SOC of the Ir 5*d t*_{2g} orbital: topological band insulator for weak coupling^{9,10}, Weyl semi-metal, axion insulator, non-Fermi liquid and TI* phases for intermediate coupling^{11,12,13,14,15}, and topological Mott insulator and quantum spin liquid phases for strong coupling^{7,16,17}.

Emergence of the spin-orbital entangled *j*_{eff} states induced by SOC^{18,19} is the key feature to host all the above phases, yet the existence of such states is limited to a small number of iridate compounds only. Here, the series of lacunar spinel compounds^{20,21}, GaM_{4}X_{8}, where early 4*d* or 5*d* transition metal atoms occupy the M-site, are found to provide the molecular form of the *j*_{eff} basis in their low-energy electronic structures. The idealness of the molecular *j*_{eff} state is guaranteed by the formation of the M_{4} metal cluster and the large SOC. Combined with the ability to control the electron correlation from the weak to strong coupling limit, the lacunar spinels can manifest themselves as the best candidates to demonstrate this so-called *j*_{eff} physics.

## Results

### Formation of the molecular *j*
_{eff} states in GaTa_{4}Se_{8}

The chemical formula and crystal structure of the GaM_{4}X_{8} lacunar spinels are easily deduced from the spinel with half-deficient Ga atoms, that is, Ga_{0.5}M_{2}X_{4}. Due to the half removal of the Ga atoms, the transition metal atoms are strongly distorted into the tetrahedral center as denoted by the red arrows in Fig. 1a, and a tetramerized M_{4} cluster appears. The M_{4} cluster yields a short intra-cluster M–M distance, naturally inducing the molecular states residing on the cluster as basic building blocks for the low-energy electronic structure. On the other hand, the large inter-cluster distance results in a weak inter-cluster bonding and a narrow bandwidth of the molecular states.

As a representative example of the lacunar spinels, we investigate the electronic structure of GaTa_{4}Se_{8} (Fig. 1b–d). Figure 1b shows the band structure and the projected density of states (PDOS) of GaTa_{4}Se_{8} in the absence of SOC. In consistency with previous studies^{21,22,23}, the triply degenerate molecular *t*_{2} bands occupied by one electron are located near the Fermi level with a small bandwidth of ~0.75 eV. As shown in the PDOS plot, the molecular *t*_{2} bands are dominated by Ta *t*_{2g} orbital components; the small admixture of Se 5*p* and the strong tetramerization imply that the molecular *t*_{2} states consist of direct bonding between Ta *t*_{2g} states.

The molecular nature of the low-energy electronic structure can be visualized by adopting the maximally localized Wannier function scheme^{24,25}. The three molecular *t*_{2} Wannier functions depicted in Fig. 1c read

where *D*_{α} and *d*_{α} denote the molecular *t*_{2} and atomic *t*_{2g} states, respectively, and *i* is a site index indicating the four corners of the M_{4} cluster. Each *D*_{α} originates from a *σ*-type strong bonding between the constituent *t*_{2g} orbitals in the M_{4} cluster. (See Supplementary Note 1, Supplementary Fig. 1 and Supplementary Table 1 for details on the molecular *t*_{2} Hamiltonian.) Owing to the exact correspondence between the molecular *t*_{2} and the atomic *t*_{2g} states, as revealed in equation 1, the molecular *t*_{2} triplet carries the same effective orbital angular momentum *l*_{eff}=1 as the atomic *t*_{2g} orbital^{18}. By virtue of SOC, the *l*_{eff}=1 states are entangled with the *s*=1/2 spin, and two multiplets designated by the effective total angular momentum *j*_{eff}=1/2 and 3/2 emerge. The band structure and PDOS of GaTa_{4}Se_{8} in the presence of SOC verify the above *j*_{eff} picture (Fig. 1d); the molecular *t*_{2} bands split into upper *j*_{eff}=1/2 and lower *j*_{eff}=3/2 bands. The separation between the two *j*_{eff} subbands is almost perfect owing to the large SOC of the Ta atoms as well as the small bandwidth of the molecular *t*_{2} band. An alternative confirmation of the *j*_{eff} picture can also be given by constructing the Wannier function from each of the *j*_{eff} subbands, which shows a 99% agreement with the ideal molecular *j*_{eff} states. (See Supplementary Fig. 2.) Consequently, the electronic structure of GaTa_{4}Se_{8} can be labelled as a quarter-filled *j*_{eff}=3/2 system on a face-centered cubic lattice.

### Robust *j*
_{eff}-ness in the GaM_{4}X_{8} series

The aforementioned *j*_{eff}-ness in GaTa_{4}Se_{8} remains robust in the GaM_{4}X_{8} series with a neighbouring 5*d* transition metal (M=W) as well as the 4*d* counterparts (M=Nb and Mo). Among the series, M=W compounds have not been reported previously in experiments; thus we use optimized lattice parameters by structural relaxations. In Fig. 2a–d, the electronic structures of GaTa_{4}Se_{4}Te_{4}^{26}, GaW_{4}Se_{4}Te_{4}, GaNb_{4}Se_{8}^{21} and GaMo_{4}Se_{8}^{27} are shown—band structure, PDOS and Fermi surface with projection onto the molecular *j*_{eff} states. In Fig. 2a,b, one can see the clear separation and identification of the higher *j*_{eff}=1/2 doublet and the lower *j*_{eff}=3/2 quartet driven by the large SOC of the 5*d* transition metal atoms. The overall band dispersions are quite similar, except for the location of the Fermi level; the M=Ta and M=W lacunar spinels are well characterized by the quarter-filled *j*_{eff}=3/2 and the half-filled *j*_{eff}=1/2 systems, respectively. In 4*d* compounds, the separation between the *j*_{eff} subbands is reduced due to the smaller SOC compared with that of the 5*d* systems (Fig. 2c,d). Nevertheless, there is a discernible splitting between the *j*_{eff}=1/2 and 3/2 bands, which is comparable to or even better than that in the prototype *j*_{eff} compounds, Sr_{2}IrO_{4} and Ba_{2}IrO_{4}^{28}.

To acquire a well-identified *j*_{eff} band, we need the *j*_{eff} state as a local basis, and the inter-orbital hopping terms between the *j*_{eff} subspaces should be suppressed. Hence, there are three important conditions to realize the ideal *j*_{eff} system: high symmetry protecting the *l*_{eff}=1 threefold orbital degeneracy, small bandwidth minimizing the inter-orbital mixing and large SOC fully entangling the spin and orbital degrees of freedom. The lacunar spinel compounds comfortably satisfy the above conditions; the tetrahedral symmetry of the M_{4} cluster protects the orbital degeneracy, the long inter-cluster distance leads to the small bandwidth and a large SOC is inherent in 4*d* and 5*d* transition metal atoms.

Figure 2e introduces one important controlling parameter—the bandwidth. By changing the inter-cluster distance via external pressure and/or by substituting chalcogen atoms, the bandwidth of the molecular *t*_{2} band can be tuned over a wide range. In the M=Ta series, for example, the bandwidth varies from 0.4 to 1.1 eV. Consequently, the effective electron correlation strength, given by the ratio between the bandwidth and the on-site Coulomb interactions, can be controlled to reach from the weak to the strong coupling regime. In fact, the bandwidth-controlled insulator-to-metal transitions were observed in GaTa_{4}Se_{4} and GaNb_{4}Se_{4}^{23,29}, implying that both the weakly and strongly interacting limits are accessible in a single compound.

### Effective Hamiltonian

From the apparent separation between the *j*_{eff} subbands, as well as the similar band dispersions, the GaM_{4}X_{8} series are governed by a common effective Hamiltonian composed of two independent *j*_{eff}=1/2 and 3/2 subspaces, that is, . (See Supplementary Notes 2 and 3.) Therefore, the compounds with M=Nb/Ta and M=Mo/W are described by the quarter-filled and the half-filled systems, respectively. The nearest-neighbor hopping terms for each subspace are written as

where **S**^{1/2} and **S**^{3/2} are the *j*_{eff}=1/2 and 3/2 pseudospin matrices, respectively, and **Γ** are the 5-component Dirac Gamma matrices. *t*^{0} and **t**^{Q}’s are even, and **t**^{D}’s are odd functions under the spatial inversion; **t**^{D}’s are allowed by the inversion asymmetry of the M_{4} cluster. The pseudospin-dependent hopping terms **t**^{D} and **t**^{Q} can be interpreted as the effective magnetic dipolar and quadrupolar fields acting on the hopping electron, respectively.

### DFT+SOC+*U* calculations

So far, we have discussed about the *j*_{eff}-ness without containing electron correlations, which provides a valid picture in the weak coupling regime. Once taking electron correlations into account, one important question arises on the robustness of the molecular *j*_{eff} states under the influence of the on-site Coulomb interaction. To answer this question, we perform DFT+SOC+*U* calculations for GaTa_{4}Se_{4}Te_{4}, GaW_{4}Se_{4}Te_{4}, GaNb_{4}Se_{8} and GaMo_{4}Se_{8}. We consider two simplest magnetic configurations, ferromagnetic and antiferromagnetic order, and the antiferromagnetic solutions for each compound are shown in Fig. 3. In the 5*d* compounds, the molecular *j*_{eff} states remain robust with developing a SOC-assisted Mott gap within each *j*_{eff} subspace (Fig. 3a,b). For the 4*d* compounds, the *j*_{eff} character is enhanced from the non-interacting cases in Fig. 2c,d; the occupied states in GaNb_{4}Se_{8} (Fig. 3c) and the unoccupied states in GaMo_{4}Se_{8} (Fig. 3d) are dominated by *j*_{eff}=3/2 and 1/2 characters, respectively. The strengthened *j*_{eff} character by the cooperation with electron correlations is consistent with the recent theoretical results on Sr_{2}IrO_{4}^{28,30}. See the Supplementary Note 4, Supplementary Figs 3–6 and Supplementary Tables 2–5 for more details.

## Discussion

The effective Hamiltonian of the lacunar spinel series has intriguing implications both in the weak and strong coupling regimes. As suggested in previous studies^{3,9,31}, the effective fields exerted on the hopping electron can induce a topological insulating phase in the weak coupling regime. In fact, a non-trivial band topology is realized within the molecular *j*_{eff} bands in-thin film geometries: the monolayer (Fig. 4a) and the bilayer thin film (Fig. 4b) of the M_{4} clusters normal to the (111) direction. Each system corresponds to the triangular and honeycomb lattice, respectively, and the inter-layer coupling enhanced by a factor of three is adopted in the bilayer system. Non-trivial gaps emerge in the half-filled *j*_{eff}=3/2 bands in the monolayer and the half-filled *j*_{eff}=1/2 bands in the bilayer system. A two-dimensional (2D) topological insulator phase is indicated by an odd number of edge Dirac cones at time-reversal invariant momenta in ribbon geometries (Fig. 4a,b). Such 2D geometries might be feasible with the help of the state-of-the-art epitaxial technique prevailing in oxide perovskite compounds^{32}, or by mechanically cleaving the single crystal to get clean surfaces as done in previous studies on GaTa_{4}Se_{8}^{33,34}.

In the strong coupling regime, the large on-site Coulomb terms are added to the kinetic Hamiltonian, and the hopping terms are treated as perturbations. The localized *j*_{eff} pseudospins become low-energy degrees of freedom and exchange interactions between the neighbouring *j*_{eff} moments emerge. In the simplest example, the one-band Hubbard model within the half-filled , the resulting spin Hamiltonian for the *j*_{eff}=1/2 moments is written as^{35,36}

among the exchange interaction terms, the Dzyaloshinskii–Moriya **D**_{ij} and the pseudodipolar interaction **A**_{ij} depend on , whose direction is determined by the two mirror planes, as illustrated in Fig. 4c (details in Supplementary Note 5). As shown in Fig. 4d, the relative magnitude of each exchange term is changed with different chalcogen atoms, so that systematic study of the anisotropic Hamiltonian in equation 3 can be made in the M=Mo/W compounds. Especially, GaMo_{4}S_{8} and GaW_{4}Se_{8} satisfies the limit of |*t*^{0}/**t**^{D}|→0, where the spin Hamiltonian becomes highly anisotropic and bond direction dependent such that

with . In addition to the Heisenberg term, the Hamiltonian contains the bond-dependent and Ising-like pseudodipolar interaction, called as a Heisenberg-compass model^{37}. It can be further reduced to distinct 2D spin models in thin-film geometries. Figure 4e,f shows two examples—the (001) and (111) monolayer lead to the 90°- and 60°-compass model with the Heisenberg exchange term on a square and a triangular lattice, respectively.

The *j*_{eff}=3/2 systems in the strong coupling limit could also have a significant implication in terms of unconventional multipolar orders^{38,39,40}. On top of the nonmagnetic insulating behaviour, the weak tetragonal superstructure and the anomalous magnetic response observed in GaNb_{4}S_{8} at T~31 K^{41} could give some clues on the quadrupolar ordered phase as well as the spin liquid phase suggested in ref. 39, which promptly calls for further research on the *j*_{eff}=3/2 spin model.

The formation of the M_{4} cluster and SOC are the essential requisites to realize the molecular *j*_{eff} state in these 3D intermetallic compounds. The strong tetramerization sustains the isolated molecular bands with threefold orbital degeneracy and narrow bandwidth, and the large SOC fully entangles the spin and orbital components. The existence of the pure quantum state has been shedding light on studying the ideal quantum model systems in strongly correlated physics; the Hubbard Hamiltonian or the frustrated spin Hamiltonian based on the pure spin-half state has been realized in several organic compounds^{42,43,44}. Likewise, the molecular form of the ideal *j*_{eff} state as a pure quantum state might be of great use to explore the emergent phenomena in the spin-orbit-coupled correlated electron systems.

## Methods

### First-principles calculations

Structural optimizations were done with the projector augmented wave potentials and the PBEsol^{45} generalized gradient approximation as implemented in the Vienna *ab initio* Simulation Package^{46,47}. Momentum space integrations were performed on a 12 × 12 × 12 Monkhorst-Pack grid, and a 300-eV energy cutoff was used for the plane-wave basis set. The force criterion was 10^{−3} eV Å^{−1}, and the pressures exerted were estimated by using the Birch–Murnaghan fit.

For the electronic structure calculations, we used OPENMX code^{48} based on the linear-combination-of-pseudo-atomic-orbital basis formalism. Four hundred Rydberg units of energy cutoff was used for the real-space integration. SOC was treated via a fully relativistic *j*-dependent pseudopotential in a non-collinear scheme. Simplified DFT+*U* formalism by Dudarev *et al*.^{49}, implemented in OPENMX code^{50}, was adopted in the DFT+SOC+*U* calculations. *U*_{eff}≡*U*−*J*=2.5 and 2.0 eV was used for the 4*d* and 5*d* compounds, respectively.

## Additional information

**How to cite this article:** Kim, H.-S. *et al*. Spin-orbital entangled molecular *j*_{eff} states in lacunar spinel compounds. *Nat. Commun.* 5:3988 doi: 10.1038/ncomms4988 (2014).

## References

- 1.
Datta, S. & Das, B. Electronic analog of the eletro-optic modulator.

*Appl. Phys. Lett.***56**, 665–667 (1990). - 2.
Pesin, D. & MacDonald, A. H. Spintronics and pseudospintronics in graphene and topological insulators.

*Nat. Mater.***11**, 409–416 (2012). - 3.
Kane, C. L. & Mele, E. J. Quantum spin Hall effect in graphene.

*Phys. Rev. Lett.***95**, 226801 (2005). - 4.
Chang, C.-Z.

*et al.*Experimental observation of the quantum anomalous Hall effect in a magnetic topological insulator.*Science***340**, 167–170 (2013). - 5.
Qi, X.-L., Li, R., Zang, J. & Zhang, S.-C. Inducing a mangetic monopole with topological surface states.

*Science***323**, 1184–1187 (2009). - 6.
Fu, L. & Kane, C. L. Superconducting proximity effect and Majorana fermions at the surface of a topological insulator.

*Phys. Rev. Lett.***100**, 096407 (2008). - 7.
Pesin, D. & Balents, L. Mott physics and band topology in materials with strong spin-orbit interaction.

*Nat. Phys.***6**, 376–381 (2010). - 8.
Witczak-Krempa, W., Chen, G., Kim, Y. B. & Balents, L. Correlated quantum phenomena in the strong spin-orbit regime.

*Annu. Rev. Condens. Matter Phys.***5**, 57–82 (2014). - 9.
Shitade, A.

*et al.*Quantum spin Hall effect in a transition metal oxide Na_{2}IrO_{3}.*Phys. Rev. Lett.***102**, 256403 (2009). - 10.
Guo, H. M. & Franz, M. Three-dimensional topological insulators on the pyrochlore lattice.

*Phys. Rev. Lett.***103**, 206805 (2009). - 11.
Wan, X., Turner, A. M., Vishwanath, A. & Savrasov, S. Y. Topological semimetal and Fermi-arc surface states in the electronic structure of pyrochlore iridates.

*Phys. Rev. B***83**, 205101 (2011). - 12.
Go, A., Witczak-Krempa, W., Jeon, G. S., Park, K. & Kim, Y. B. Correlation effects on 3D topological phases: from bulk to boundary.

*Phys. Rev. Lett.***109**, 066401 (2012). - 13.
Wan, X., Vishwanath, A. & Savrasov, S. Y. Computational design of axion insulators based on 5

*d*spinel compounds.*Phys. Rev. Lett.***108**, 146601 (2012). - 14.
Moon, E.-G., Xu, C., Kim, Y. B. & Balents, L. Non-fermi-liquid and topological states with strong spin-orbit coupling.

*Phys. Rev. Lett.***111**, 206401 (2013). - 15.
Maciejko, J., Chua, V. & Fiete, G. A. Topological order in a correlated three-dimensional topological insulator.

*Phys. Rev. Lett.***112**, 016404 (2014). - 16.
Okamoto, Y., Nohara, M., Aruga-Katori, H. & Takagi, H. Spin-liquid state in the S=1/2 hyperkagome antiferromagnet Na

_{4}Ir_{3}O_{8}.*Phys. Rev. Lett.***99**, 137207 (2007). - 17.
Chaloupka, J., Jackeli, G. & Khaliullin, G. Kitaev-Heisenberg model on a honeycomb lattice: possible exotic phases in iridium oxides

*A*_{2}IrO_{3}.*Phys. Rev. Lett.***105**, 027204 (2010). - 18.
Kim, B. J.

*et al.*Novel*J*_{eff}=1/2 Mott state induced by relativistic spin-orbit coupling in Sr_{2}IrO_{4}.*Phys. Rev. Lett.***101**, 076402 (2008). - 19.
Kim, B. J.

*et al.*Phase-sensitive observation of a spin-orbital mott state in Sr_{2}IrO_{4}.*Science***323**, 1329–1332 (2009). - 20.
Pocha, R., Johrendt, D. & Pöttgen, R. Electronic and structural instabilities in GaV

_{4}S_{8}and GaMo_{4}S_{8}.*Chem. Mater.***12**, 2882–2887 (2000). - 21.
Pocha, R., Johrendt, D., Ni, B. & Abd-Elmeguid, M. M. Crystal structures, electronic properties, and pressure-induced superconductivity of the tetrahedral cluster compounds GaNb

_{4}S_{8}, GaNb_{4}Se_{8}, and GaTa_{4}Se_{8}.*J. Am. Chem. Soc.***127**, 8732–8740 (2005). - 22.
Camjayi, A., Weht, R. & Rozenberg, M. Localised Wannier orbital basis for the Mott insulators GaV

_{4}S_{8}and GaTa_{4}Se_{8}.*Europhys. Lett.***100**, 57004 (2012). - 23.
Ta Phuoc, V.

*et al.*Optical conductivity measurements of GaTa_{4}Se_{8}under high pressure: evidence of a bandwidth-controlled insulator-to-metal Mott transition.*Phys. Rev. Lett.***110**, 037401 (2013). - 24.
Mazari, N. & Vanderbilt, D. Maximally-localized generalized Wannier functions for composite energy bands.

*Phys. Rev. B***56**, 12847–12865 (1997). - 25.
Souza, I., Mazari, N. & Vanderbilt, D. Maximally-localized Wannier functions for entangled energy bands.

*Phys. Rev. B***65**, 035109 (2001). - 26.
Guiot, V., Janod, E., Corraze, B. & Cario, L. Control of the electronic properties and resistive switching in the new series of Mott insulators GaTa

_{4}Se_{8−y}Te_{y}(0≤*y*≤6.5).*Chem. Mater.***23**, 2611–2618 (2011). - 27.
Francois, M.

*et al.*Structural phase transition in GaMo_{4}Se_{8}and AlMo_{4}S_{8}by X-ray powder diffraction.*Zeitschrift für Kristallographie***200**, 47–55 (1992). - 28.
Arita, R., Kuneš, J., Kozhevnikov, A., Eguiluz, A. & Imada, M.

*Ab initio*studies on the interplay between spin-orbit interaction and Coulomb correlation in Sr_{2}IrO_{4}and Ba_{2}IrO_{4}.*Phys. Rev. Lett.***108**, 086403 (2012). - 29.
Abd-Elmeguid, M.

*et al.*Transition from Mott insulator to superconductor in GaNb_{4}Se_{8}and GaTa_{4}Se_{8}under high pressure.*Phys. Rev. Lett.***93**, 126403 (2004). - 30.
Zhang, H., Haule, K. & Vanderbilt, D. Effective

*J*=1/2 insulating state in Ruddlesden-Popper iridates: an LDA+DMFT study.*Phys. Rev. Lett.***111**, 246402 (2013). - 31.
Haldane, F. D. M. Model for a quantum Hall effect without Landau levels: condensed-matter realization of the ‘parity anomaly’.

*Phys. Rev. Lett.***61**, 2015–2018 (1988). - 32.
Xiao, D., Zhu, W., Ran, Y., Nagaosa, N. & Okamoto, S. Interface engineering of quantum Hall effects in digital transition metal oxide heterostructures.

*Nat. Commun.***2**, 596 (2011). - 33.
Dubost, V.

*et al.*Resistive switching at the nanoscale in the Mott insulator compound GaTa_{4}Se_{8}.*Nano Lett.***13**, 3648–3653 (2013). - 34.
Dubost, V.

*et al.*Electric-field-assisted nanostructuring of a Mott insulator.*Adv. Funct. Mater.***19**, 2800–2804 (2009). - 35.
Micklitz, T. & Norman, M. R. Spin Hamiltonian of hyper-kagome Na

_{4}Ir_{3}O_{8}.*Phys. Rev. B***81**, 174417 (2010). - 36.
Jackeli, G. & Khaliullin, G. Mott insulators in the strong spin-orbit coupling limit: from Heisenberg to a quantum compass and Kitaev models.

*Phys. Rev. Lett.***102**, 017205 (2009). - 37.
Nussinov, Z. & van den Brink, J. Compass and Kitaev models—Theory and Physical Motivations. Preprint at http://arxiv.org/abs/1303.5922 (2013).

- 38.
Jackeli, G. & Khaliullin, G. Magnetically hidden order of Kramers doublets in

*d*^{1}systems: Sr_{2}VO_{4}.*Phys. Rev. Lett.***103**, 067205 (2009). - 39.
Chen, G., Pereira, R. & Balents, L. Exotic phases induced by strong spin-orbit coupling in ordered double perovskites.

*Phys. Rev. B***82**, 174440 (2010). - 40.
Pi, S.-T., Nanguneri, R. & Savrasov, S. Y. Calculation of multipolar exchange interactions in spin-orbital coupled systems.

*Phys. Rev. Lett.***112**, 077203 (2014). - 41.
Jakob, S.

*et al.*Structural and magnetic transitions in the Mott insulator GaNb_{4}S_{8}.*J. Mater. Chem.***17**, 3833–3838 (2007). - 42.
Yamashita, S.

*et al.*Thermodynamic properties of a spin-1/2 spin-liquid state in a*κ*-type organic salt.*Nat. Phys.***4**, 459–462 (2008). - 43.
Kagawa, F., Miyagawa, K. & Kanoda, K. Magnetic Mott criticality in a

*κ*-type organic salt probed by NMR.*Nat. Phys.***5**, 880–884 (2009). - 44.
Yamashita, M.

*et al.*Highly mobile gapless excitations in a two-dimensional candidate quantum spin liquid.*Science***328**, 1246–1248 (2010). - 45.
Perdew, J. P.

*et al.*Restoring the density-gradient expansion for exchange in solids and surfaces.*Phys. Rev. Lett.***100**, 136406 (2008). - 46.
Kresse, G. & Hafner, J.

*Ab initio*molecular dynamics for liquid metals.*Phys. Rev. B***47**, 558–561 (1993). - 47.
Kresse, G. & Furthmüller, J. Efficient iterative schemes for

*ab initio*total-energy calculations using a plane-wave basis set.*Phys. Rev. B***54**, 11169–11186 (1996). - 48.
Ozaki, T. Variationally optimized atomic orbitals for large-scale electronic structures.

*Phys. Rev. B***67**, 155108 (2003). - 49.
Dudarev, S., Botton, G., Savrasov, S., Humphreys, C. & Sutton, A. Electron-energy-loss spectra and the structural stability of nickel oxide:An LSDA+

*U*study.*Phys. Rev. B***57**, 1505–1509 (1998). - 50.
Han, M. J., Ozaki, T. & Yu, J. O(N) LDA+U electronic structure calculation method based on the nonorthogonal pseudoatomic orbital basis.

*Phys. Rev. B***73**, 045110 (2006).

## Acknowledgements

We thank Yong-Baek Kim, Eun-Gook Moon, Tae-Won Noh and Je-Geun Park for helpful discussions. This work was supported by the Institute for Basic Science (IBS) in Korea. Computational resources were provided by the National Institute of Supercomputing and Networking/Korea Institute of Science and Technology Information with supercomputing resources including technical support (Grant No. KSC-2013-C2-005).

## Author information

## Affiliations

### Department of Physics, Korean Advanced Institute of Science and Technology, Daejun 305-701, Korea

- Heung-Sik Kim
- & Myung Joon Han

### Department of Physics and Astronomy, Northwestern University, Evanston, Illinois 60208, USA

- Jino Im

### KAIST Institute for the NanoCentury, Korean Advanced Institute of Science and Technology, Daejun 305-701, Korea

- Myung Joon Han

### Center for Correlated Electron Systems, Institute for Basic Science (IBS), Seoul 151-747, Korea

- Hosub Jin

### Department of Physics and Astronomy, Seoul National University, Seoul 151-747, Korea

- Hosub Jin

## Authors

### Search for Heung-Sik Kim in:

### Search for Jino Im in:

### Search for Myung Joon Han in:

### Search for Hosub Jin in:

### Contributions

H.-S.K. and J.I. performed DFT calculations and data analysis with assistance from M.J.H. and H.J. All authors contributed to the discussion and the writing of the paper. H.J. was responsible for the conception and the overall direction.

### Competing interests

The authors declare no competing financial interests.

## Corresponding author

Correspondence to Hosub Jin.

## Supplementary information

## PDF files

- 1.
### Supplementary Information

Supplementary Figures 1-6, Supplementary Tables 1-5, Supplementary Notes 1-5 and Supplementary References

## Rights and permissions

To obtain permission to re-use content from this article visit RightsLink.

## About this article

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.