Introduction

It has been more than 40 years since Anderson predicted quantum spin liquid (SL) states using his resonating–valence bond model1. These SL states have attracted much attention, as they are theoretically expected to not freeze, even at 0 K. In the last decade, efforts have been made to discover an actual example of a quantum SL state in triangular, Kagomé and pyrochlore lattices2. All of these lattice types are based on a triangular arrangement of the spin sites, which makes them incompatible with antiferromagnetic ordering. These triangular lattices hinder any ordering pattern of the spin arrangements being made when the spins are affected by antiferromagnetic interactions, which leads to geometrical frustration. It is within this context that some two-dimensional (2D) molecular solids have attracted a lot of interest, as their spin systems possess triangular lattices with S = 1/23,4,5,6,7,8,9,10,11,12,13; examples of such solids include β′-EtMe3Sb[Pd(dmit)2]2 [dmit = 1,3-dithiole-2-thione-4,5-dithiolate], κ-(ET)2M2(CN)3 [ET = bis(ethylenedithio)tetrathiafulvalene, M = Cu and Ag] and κ-H3(Cat-EDT-TTF)2 [ = catechol-fused ethylenedithiotetrathiafulvalene]14,15,16,17,18,19,20,21,22,23. The triangular lattices of β′-EtMe3Sb[Pd(dmit)2]2 are comprised of a pair of independent monomers (Fig. 1(a)), where each pair is termed a dimer (Fig. 1(b–d))16,24,25. If one is to assume that an electron or a hole should be located at a dimer, then β′-EtMe3Sb[Pd(dmit)2]2 should have a half-filled band, which would lead to a Mott insulator. The structures of the triangular lattices of κ-(ET)2M2(CN)3 and κ-H3(Cat-EDT-TTF)2 are similar to that of β′-EtMe3Sb[Pd(dmit)2]2. The results of magnetic and transport property measurements have revealed the absence of any ordering, even at low temperature, which suggests that a spin liquid (SL) state occurs in β′-EtMe3Sb[Pd(dmit)2]2, κ-(ET)2M2(CN)3 and κ-H3(Cat-EDT-TTF)2 14,15,16,17,18,19,20,21,22,23. However, a series of X[Pd(dmit)2]2 [X: a monovalent cation], whose 2D layer is isostructural to that of β′-EtMe3Sb[Pd(dmit)2]2, has been shown to have various ground states as shown in Table 1 16,24. Hereafter, we use the abbreviations in Table 1. Similar variety is also observed in the κ-type compounds5,18,26.

Figure 1
figure 1

Molecular structure, crystal structure and schematic views of two-dimensional layers of β′–EtMe3Sb[Pd(dmit)2]2 (Sb-salt 2). (a) Monomer of Pd(dmit)2. (b) Crystal structure as viewed along the b-axis. (c) Two equivalent conducting layers with different orientations; layer 1 and layer 2. t d, t s and t t denote the transfer integrals between neighboring dimers in the diagonal, stacking and transverse directions, respectively. (d) Schematic views of the two-dimensional layers.

Table 1 Ground states of X[Pd(dmit)2]2 salts and deviation from equilateral triangular lattices.

There is a significant difference between the molecular arrangements in the 2D layers of X[Pd(dmit)2]2 and κ-type compounds. The 2D layers of κ-type compounds always consist of isosceles or equilateral triangular lattices18, while the 2D layers in X[Pd(dmit)2]2 can be equilateral, isosceles or scalene lattices, depending on X17,25. Therefore, there is a variety of 2D layers that X[Pd(dmit)2]2 can have, whereas the 2D layers of κ-type compounds are specific. An additional important difference between them is that the inter- and intra-dimer transfer integrals are significantly smaller and larger, respectively, in X[Pd(dmit)2]2 than in κ-type compounds24. A large intra-dimer transfer integral is due to the chemical bond between monomers in a dimer, the result of which is an inversion in the energy levels of the molecular orbitals near the Fermi level ε F 27,28,29. This energy inversion favours several kinds of charge-ordered (CO) states30,31,32,33,34,35,36,37,38,39,40.

As shown in Table 1, the antiferromagnetic (AF) transition temperature, T AF, decreases as the 2D layers become shaped more like equilateral triangular lattices16,21,24,25. The trend in Table 1 suggests that the magnetic ordering in β′-EtMe3Sb[Pd(dmit)2]2 (Sb-salt 2) is suppressed by a geometrical frustration. The specific heat capacity, thermal conductivity, nuclear magnetic resonance and magnetic torque of Sb-salt 2 reveal that there is finite entropy in the very-low-temperature range, but these experimental results also reveal that there is a subtle entropy release below the liquid helium temperature (3–4 K by the specific heat capacity)14,15,41,42,43,44. Similar anomalous phenomena have been reported for κ-(ET)2Cu2(CN)3 20,45,46,47,48, and these phenomena indicate that the precursor of any ordering is hidden behind the SL states. However, the ground states of β′-Et2Me2Sb[Pd(dmit)2]2 (Sb-salt 3), monoclinic-EtMe3P[Pd(dmit)2]2 (m-salt), triclinic-EtMe3P[Pd(dmit)2]2 (t-salt) and β′-Cs[Pd(dmit)2]2 (Cs-salt) are CO states accompanied by the alternation in the inter-molecular distances which is denoted as “valence bond ordering” (=VBO)30,32,33,34,35,36,38,39,40,49. The pressure inducing the superconducting transition of the m-salt is lower than that of any other X[Pd(dmit)2]2 salt exhibiting the AF ground state24,49,50. The 1/4-filled model is more appropriate than the effective 1/2-filled model for the X[Pd(dmit)2]2 salts exhibiting the CO states38,39,40. There are two kinds of VBO in X[Pd(dmit)2]2: the lattice distortions exhibiting the alternations in the inter- and intra-dimer transfer integrals. The electron densities and the alternations in the inter- and intra-dimer transfer integrals (=amplitudes of the VBOs) in the CO states shown in Table 1 were examined from the C=C stretching modes37,38,39,40. The CO state and VBO have also been observed for κ-D3(Cat-EDT-TTF)2 and κ-(ET)2B(CN)4, respectively22,23. Using information from infrared (IR) and Raman spectra of P-salt 1, P-salt 2, Sb-salt 1 and Sb-salt 2, we report that the absence of any ordering and a small entropy releasing of a Sb-salt 2 can be ascribed to dynamical fluctuations due to the competition between different types of CO states that are accompanied by different types of VBOs.

Results

The electron densities and the amplitudes of the VBOs for the SL and AF salts were examined from the C=C stretching modes. Prior to discussing the experimental results for the SL and AF salts, we show how C=C stretching modes of monomers, dimers, tetramers and octamers are observed (Fig. 2). Hereafter, the word dimer will always refer to a dimer that has a chemical bond between monomers. We assume that the dimers, tetramers and octamers in Fig. 2 each have a centre of inversion symmetry. When a 2D layer is composed of dimers, none of the C=C stretching modes, except for the CD and BD modes, should be observable in an IR spectra. Furthermore, none of the C=C stretching modes, aside from those for the AD and DD modes, should be observable in a Raman spectra. This mutual exclusion rule can be applied not only to equilateral lattices but also to isosceles and scalene lattices when there is no VBO in a 2D layer. With regards to tetramers, the number of C=C stretching mode becomes eight (from AT1 to DT2), all of which belong to Groups A–D in Fig. 2. The total number of C=C stretching modes in an octamer is sixteen (from AO1 to DO4), all of which also belong to Groups A–D.

Figure 2
figure 2

Correlation diagram of C=C stretching modes in monomers, dimers, tetramers and octamers that have centre of inversion symmetries. The tetramers and octamers are composed of two and four dimers, respectively. The C=C stretching modes of the monomers and dimers are shown by the arrows. AD–DD: C=C stretching modes of a dimer, AT1–DT2: C=C stretching modes of a tetramer, and AO1–DO4: C=C stretching modes of an octamer. The positive and negative signs for each vibrational mode denote in-phase and out-of-phase vibrations, respectively. These C=C stretching modes are classified into Groups A–D. AR-DIR in the right column correspond to peaks in Fig. 3. CO4 denoted by the blue area is independent of any C=C stretching modes of tetramer and dimer. AIR and DIR, denoted by green area, consist of C=C stretching modes of tetramer and octamer, but these are independent of any C=C stretching modes of dimer. AIR consists of the AT2 and AO3 modes, and DIR consists of the DT2 and DO3 modes. AR, CIR, BIR and DR, denoted by orange area, consist of C=C stretching modes of dimer, tetramer and octamer. AR consists of the AD, AT1 and AO1 modes, CIR consists of the CD, CT1 and CO1 modes, BIR consists of the BD, BT1, BO1 and BO4 modes, and DR consists of the DD, DT1, DO1 and DO4 modes.

Figure 3(a)–(d) show the Raman spectra and the conductivity spectra in the IR region for P-salt 1, P-salt 2, Sb-salt 1 and Sb-salt 2. Hereafter, the IR spectra designate the conductivity spectra in the IR region. The notations of the C=C stretching modes in Fig. 3 correspond to those in the right column of Fig. 2, and the frequencies of the C=C stretching modes are summarised in Table 2. More than two C=C stretching modes were observed in the IR spectra of Sb-salt 2 (Fig. 3(a)), which indicates that its lattice was composed of tetramers or octamers. The C=C stretching modes for Sb-salt 2 can be assigned based on the IR and Raman spectra of Sb-salt 3, Cs-salt, t-salt and m-salt, where all of the dimer modes (from AD to DD) were split into multiple peaks due to the occurrence of tetramers and octamers37,38,39,40. The Raman and IR spectra in the CO states are shown in Supplementary information37,38,39,40. CO4 shown in Fig. 2 is easily distinguishable from any other C=C stretching modes because its frequency is the lowest due to the electron-molecular vibrational (e-mv) interaction involving both inter- and intra-dimer charge transfers37,40. The CO4 mode is characteristic of an octamer and independent of the C=C stretching modes belonging to a dimer and a tetramer. We have found that the CO4 mode for Sb-salt 2 in the a- and b-polarized spectra (Fig. 3(a) and (b)); however, its intensity was significantly weak in comparison to those of the Cs-salt and Sb-salt 3 exhibiting static octamers shown by Supplementary information37,40. This result indicates that there should not be any static CO state due to the octamers and the octamers exhibits fluctuation not only at 70 K but also at 5 K.

Figure 3
figure 3

IR and Raman spectra. (a) a-polarized IR spectra for Sb-salt 2. (b) b-polarized IR spectra for Sb-salt 2. (c) c*-polarized IR spectra of P-salt 1, Sb-salt 1 and Sb-salt 2. The a-polarized IR spectra of P-salt 2 is also shown in (c). (d) Raman spectra observed at 5 K. Aside from Sb-salt 2, all of the salts here have antiferromagnetic ground states. All of the IR spectra were obtained as conductivity spectra from the Kramers–Kronig transformation of the corresponding reflectance spectra.

Table 2 Frequencies of the C=C stretching modes.

DIR was observed in both the a- and b-polarised spectra at 5 K (Fig. 3(a) and (b)). Because the linewidth is large, i.e. 16 cm−1, DIR consists not only of the DT2 mode of a tetramer but also of the DO3 mode of an octamer, those which are denoted by the green area in Fig. 2. These spectral features support that both the tetramers and octamers exhibit fluctuations in the SL state. The modes of DIR were observable at 70 K, which indicates that the fluctuation remains at this temperature. The DIR mode is also observed in the c*-polarized spectra although its intensity is significantly weak. This phenomena is ascribed to the fact that the long axes of the [Pd(dmit)2]2 molecules are not exactly normal to the ab-plane37,38,39,40. We have found that DIR is commonly observed in the c*-polarised spectra of P-salt 1, P-salt 2 and Sb-salt 1. This result suggests that the fluctuation due to forming tetramer or octamer is non-negligible for the AF salts, i.e. P-salt 1, P-salt 2 and Sb-salt 1. In particular, the IR and Raman spectra of P-salt 1 exhibit several minor vibrational modes, which indicates the phase separation. To investigate the inhomogeneous charges and the inter-molecular interactions in the AF salts, further experiments including the a- and b-polarized spectra are required.

AIR was observed as a shoulder of BIR in the a-polarized spectra at 5 K (i.e. Sb-salt 2 (Fig. 3(a)). As shown in Fig. 2, AIR consists of AT2 in a tetramer and AO3 in an octamer, those which are denoted by the green area. The frequency of AIR (1342.9 cm1) was found to be significantly lower than that of AR (1359.9 cm1) in Fig. 3(d). This difference in the frequencies is due to the inter-dimer transfer integral in a tetramer being larger than that between tetramers38,39. The intensity of AIR was very weak in comparison to that of BIR. Conversely, the intensities of AIR were comparable to those of BIR in the CO states, i.e. in t-salt and m-salt38,39. The weak intensity of Sb-salt 2 indicates that it does not include any static tetramers or octamers; in other words, fluctuations in the formation of both octamers and tetramers take place in the SL state.

The linewidth of DR in Fig. 3(d) of Sb-salt 2 is larger than that of any salt that had an AF ground state. These spectral features indicate the fluctuation in the formation of the octamers and tetramers in the SL state of Sb-salt 2. When octamers, tetramers and dimers compete with one another, the frequencies of DD, DT1, DO1 and DO4 denoted as the orange area in Fig. 2 are close to each other in the Raman spectra, which leads to the broadening of DR. However, neither peak separation nor broadening was observed in the DR mode in the AF ground states. This result indicates that the fluctuation was greater in Sb-salt 2 than in P-salt 1, P-salt 2 and Sb-salt 1. The degree of fluctuation is quantitatively discussed in the Discussion section.

The vibrational modes at 1331 cm−1 in Fig. 3(a)–(c) belong to BIR. This assignment is supported by the relative intensities in the c*-polarised spectra being higher than those in the a- and b-polarised spectra37,38,39,40. The frequency of BIR is proportional to the molecular charge40. The frequencies in the AF states of P-salt 1, P-salt 2 and Sb-salt 1 reveal that all of the monomer charges are equal to −0.5, i.e. [Pd(dmit)2]0.5−. The frequency of Sb-salt 2 also indicates that it consists of [Pd(dmit)2]0.5−; however, the linewidth of Sb-salt 2 shown as ΔB in Table 2 is twice or three times as large as ΔB of the AF salts. This result indicates the inhomogeneous molecular charge in the SL state of Sb-salts 2. As shown in Supplementary information, Group B in the CO state exhibits the peak separation and the separated peaks belong to the charge-rich and charge-poor molecules37,38,39,40. ΔB in the CO state is obtained from the difference in the frequencies37,38,39,40. ΔB of Sb-salt 2 is smaller than those in the CO states of Sb-salt 3, Cs-salt and t-salt (37, 43, 35 cm−1, respectively), but a slightly larger than that in the CO state of m-salt (6 cm−1). BIR of Sb-salt 2 does not exhibit any peak separation whereas that of m-salt exhibits the peak separation due to the static tetramer (BT1 and BT2)38. The absence of any peak separation and the broad linewidth for Sb-salt 2 indicate that an inhomogeneous molecular charge distribution in the SL state is ascribed to the fluctuation due to forming octamer, tetramer and dimer. Thus, BIR in the SL state of Sb-salt 2 consists of BD, BT1, BO1 and BO4, those which are denoted as the orange area in Fig. 2.

As shown by the orange areas in Fig. 2, AR and CIR in Fig. 3 consists of the C=C stretching modes for dimer, tetramer and octamer. As described in the previous literatures and Supplementary information, the frequencies of AR and CIR (=CT1 and CO1) are insensitive to the change in the inter-molecular interaction. Thus, the inter-molecular interaction cannot be analysed from AR and CIR 37,38,39,40.

Discussion

The behaviour of the C=C stretching modes of Sb-salt 2 at 5 K indicates the competition between the octamers, tetramers and dimers. The CO states accompanied by the tetramers and octamers would be favourable if there were an inversion in the energy levels of the molecular orbitals near the Fermi level ε F 31,35,36,38,39,40. As a result of such an inversion, the highest occupied molecular orbitals (HOMOs) and the lowest unoccupied molecular orbitals (LUMOs) of the monomers in Fig. 4(a), respectively, constitute the LUMO and HOMO of the dimer, those which are denoted as HH and L + L in Fig. 4(a)27,28,29. The interchange in the energy levels happens when the dimerization is tight enough for it to seem as if there is a chemical bond between the monomers; this is called a HOMO–LUMO inversion24,28. VBO due to the electron-phonon interaction is enhanced in the HOMO of [tetramer]2−, and the charge separation due to the nearest neighbour Coulomb repulsions (=V) is enhanced in the next HOMO of [tetramer]2−  38,39. Similarly, VBO and the charge separation due to V are enhanced in the different orbirals of [octamer]4− near the Fermi level, ε F 31,32,36. As a result, VBO and V originating from the different orbitals are cooperatively enhanced in the CO state of the HOMO–LUMO inversion system. On the other hand, the CO states, spin liquid and superconductivity of the BEDT-TTF salts are ascribed to the cooperation between VBO and V in the same HOMOs8. However, there is no theoretical model on the cooperative interaction involving different orbitals. The enhancement of both VBO and V in the X[Pd(dmit)2]2 salts results in several kinds of CO states being accompanied by bond alternations, two of which are shown by Fig. 4(b) and (c) 30,37,38,39,40,51. The CO states shown by Fig. 4(b) and (c) are accompanied by tetramers and octamers, respectively. The 2D layer containing tetramers shown in Fig. 4(b) exhibits a difference in the inter-dimer transfer integrals38,39. This difference is referred to by this paper as an inter-dimer VBO. The 2D layer containing octamers shown in Fig. 4(c) exhibits a difference in the intra-dimer transfer integrals30,40; this difference is referred as an intra-dimer VBO. The 2D layer of Fig. 4(c) also exhibits the inter-dimer VBO30,40. For each CO state, the ionic molecules are next to each other due to the cooperation between VBO and V. Figure 4(d) shows the 2D layer under the assumption that X[Pd(dmit)2]2 belongs to a Mott insulator, where there is no inter-dimer or intra-dimer VBO and there is no CO. On the other hand, the effective 1/4-filled model rather than the effective 1/2-filled model is applicable to a tetramer and an octamer38,39,40. In this Discussion section, the magnitudes of the intra-dimer and inter-dimer VBOs are examined in the SL state (i.e. Sb-salt 2), and they are compared to those of the AF states (P-salt 1, P-salt 2 and Sb-salt 1). The quantitative analyses of the intra-dimer and inter-dimer VBOs can be done using the A and D groups, but cannot be done using the C group because the frequency of the CO4 is extraordinary perturbed by both inter-dimer and intra-dimer charge transfers37,40.

Figure 4
figure 4

HOMO–LUMO inversion, and charge distributions and bond alternations in a 2D layer of X[Pd(dmit)2]2. (a) Energy diagram of the monomers, normal dimer, and tight dimer. “L” and “H” represent the LUMO and HOMO of a monomer, respectively. The sum (+) and subtraction (−) between them designate bonding and antibonding interactions, respectively. (b) and (c): Two of three different CO states revealed by our previous works. (d): Layer consisting of regularly arranged [dimers]. (e): Legend of electron densities of monomers in (bd). The black, grey and white sections in the monomers of (bd) denote the electron densities in the HOMO of (b) [tetramer]2−, (c) [octamer]4− and (d) [dimer]. Different patterns of the inter-dimer bond alternations; the bold, thin and dotted lines between the dimers denote the strong, intermediate and weak inter-dimer interactions, respectively, between the HOMOs in (b) [tetramer]2− and (c) [octamer]4−. The interacting monomers that form tetramers or octamers in (b) and (c) are highlighted by the orange rectangles. Inter-dimer VBOs in (b) are formed along two of three directions in the triangular lattice. Intra-dimer charge separation in (b) is accompanied by inter-dimer VBOs. The intra-dimer interaction in a charge-rich dimer of (c) is weaker than that in a charge-poor dimer, which manifests itself as the intra-dimer VBO. Inter-dimer VBOs are also formed in (c). Intra- and inter-dimer charge separations of (c) are accompanied by inter- and intra-dimer VBOs, respectively. The combination between intra- and inter-dimer VBOs is formed along one of the three directions. There is no VBO in (d).

To begin with, we examine the intra-dimer VBO. Because the frequency of the DR mode decreases as the magnitude of dimerization increases37,40, the peak separation or broad linewidth in the DR mode indicates the inhomogeneity in the dimerization. The degree of inhomogeneity is estimated from ΔD, which is defined as being the difference in the frequencies of the DR modes of the charge-rich and charge-poor dimers (DO1 and DO4 in Fig. 2) 37,40. When the DR mode does not show any recognisable peak separation, ΔD is defined as being the linewidth of DR 40. Table 3 shows ΔD in the AF, SL and CO states. ΔD in the AF state becomes large when T AF is small. ΔD of Sb-salt 2 is larger than that of any salt containing AF ground states. These experimental results indicate that the inhomogeneity of the dimerization increases from the AF to the SL states. This conclusion is inconsistent with that of a previous report, in which the dielectric constant was found to have no obvious relationship with T AF 52. This inconsistency suggests that the anomaly in the dielectric constant should include not only the inhomogeneity due to the bond alternations but also macroscopic inhomogeneities. However, ΔD of Sb-salt 2 is smaller than ΔD of the octamers in the CO states (Sb-salt 3 and Cs-salt), and it is of an intermediate value and comparable to that of Cs-salt at 100 K and t-salt in the CO state. Cs-salt above T CO exhibits a dynamical fluctuation due to the competition between the octamer and tetramer, and t-salt in the CO state contains residual octamers in the 2D layer consisting of tetramers39,40. Therefore, the intermediate value of ΔD of Sb-salt 2 indicates the fluctuation of the intra-dimer bond alternation.

Table 3 Amplitudes of intra-dimer and inter-dimer bond alternations and charge inhomogeneity.

Next, we examine the inter-dimer VBO. The magnitude of the bond alternation in tetramers is defined as being ΔA, which is the difference in the frequencies between the AR (=AT1) and AIR (=AT2) modes38,39. The ΔA value of the octamers is reflected in the alternation between the bold line and the thin line in the rectangle in Fig. 4(c)40. ΔA of Sb-salt 2 is not larger than any of the values of ΔA in Table 3. This result suggests that the inter-dimer VBO is not evident in the SL state, which is in agreement with the fact that the AIR mode in the SL state is weak.

However, ΔA cannot be obtained from the IR and Raman spectra of the AF salts (P-salt1, P-salt 2 and Sb-salt 1), because the AIR mode is not observed clearly in the IR spectra. Nevertheless, the frequencies of DIR are slightly higher than those of DR. When the 2D layer contains tetramers, DR exhibits the symmetric vibration with respect to centre of inversion symmetry in a tetramer whereas DIR exhibits asymmetric vibration. DR induces the inter-dimer charge transfer whereas DIR does not. This relationship is comparable to that between AIR and AR. Similarly to ΔA, the magnitude of the bond alternation in tetramers is proportional to the difference in the frequencies between DR and DIR. The frequencies of DR and DIR in the AF salts were estimated from the Raman and c*-polarized spectra, respectively. The difference in the frequencies (δD) is shown in Table 3. Non-zero δD indicates the inter-dimer VBO in the AF salts. Nevertheless, both δD and ΔD are small for the AF salts. Any accurate value of DIR cannot be obtained from the c*-polarized spectra of Sb-salt 2 because its intensity is very weak and the linewidth is large. The frequency and linewidth were obtained from the a-polarized spectra. Nevertheless, no accurate values of δD (DIR DR) could be obtained for Sb-salt 2, because the linewidths of both DR and DIR are broad (DR = 9.5 and DIR = 16.4 cm1). The broad linewidth of DIR suggests that there is an inhomogeneity in the inter-dimer VBO; thus, the linewidth can be regarded as being the maximum value in the magnitude of the bond alternation. This value is shown in Table 3 as δD for Sb-salt 2. This estimation is supported by δD being almost identical to ΔA. The δD of Sb-salt 2 is significantly larger than that of any of the AF salts (P-salt1, P-salt 2 and Sb-salt 1); therefore, the magnitude of the inter-dimer VBO in the SL state is intermediate between the AF and CO states. Although the δD of the AF salts are small, the observation of the DIR mode in the c*-polarized spectra indicates the weak charge inhomogeneity due to the dynamical or static fluctuation.

Table 3 shows the inhomogeneity in the molecular charges. The difference in the molecular charges between the charge-rich and charge-poor molecules (i.e. Δρ of [Pd(dmit)2](0.5±Δρ)) can be estimated from ΔB 40. Δρ in the SL and AF states were obtained from this work, and Δρ in the CO states were obtained from the previous experiments37,38,39,40. Δρ of Sb-salt 2 is larger than those in the AF states (P-salt 1, P-salt 2 and Sb-salt 1), a slightly larger than that in the CO state of m-salt and smaller than those in the CO states of t-salt, Sb-salt 2 and Cs-salts. Δρ of Sb-salt 2 is comparable to that of Cs-salt at 100 K, where tetramers and octamers are competing with each other. This result supports the dynamical fluctuation in the SL state of Sb-salt 2.

The inter-dimer transfer integrals of X[Pd(dmit)2]2 are too small for metallic bonds; rather, inter-dimer interactions induce tetramers and octamers. As a result, ΔA (and δD) and ΔD become non-zero. In a tetramer, ΔA (and δD) increases, but ΔD remains small; in an octamer, not only does ΔA (and δD) increase but so does ΔD. δD and ΔD decrease in the AF salts. As for Sb-salt 2, both ΔA (and δD) and ΔD are intermediate between the AF state and octamer. Quantitative analyses of the electron-molecular vibration coupling modes in the SL state indicate that the dynamical fluctuations are due to the competition among three building blocks; octamers, tetramers and dimers. The broad linewidth in Group B supports the motional narrowing due to the dynamical fluctuation among the octamers, tetramers and dimers. These spectral features indicate that the solid solution consisting of three building blocks exhibits the fluctuation; i.e. 4[dimer] ↔ 2[tetramer]2− ↔ [octamer]4− ↔ 4[dimer]. No ordering in Sb-salt 2 can be ascribed to the fluctuation in the triangular lattice. In the SL state of Sb-salt 2, all of the charge distribution patterns in Fig. 4(b,c) and (d) participate in the fluctuation. Because of the fluctuation, no ordering is considered to be realised, even though the 2D layer slightly deviates from that of an equilateral triangular lattice. Without the dynamical fluctuation, a subtle deviation from the equilateral triangular lattice should produce a CO state accompanied by VBO, which was the case for m-salt and Sb-salt 332,34,35,36,37,38,40,49,50,53. The broad and small peak in the normalized C p T −1 vs T plot obtained from the specific heat measurement of Sb-salt 2 is in agreement with the fluctuation involving the tetramers and octamers because the tetramers and octamers induce the small entropy releasing43. The fluctuation is also consistent with the fact that Sb-salt 2 retains the finite entropy in the very low temperature region.

Conclusion

We have examined the inter-dimer bond alternation, intra-dimer bond alternation and inhomogeneous molecular charges of Sb-salt 2, which we believe to be a quantum SL. The vibrational spectroscopies that focused on the C=C stretching modes revealed that the dynamical fluctuation was due to the competition among the dimers, tetramers and octamers in the SL state. A HOMO–LUMO inversion was found to favour several kinds of CO states consisting of tetramers and octamers, which enhanced the dynamical fluctuation and the inhomogeneous charges. Therefore, the SL state of Sb-salt 2 is ascribed to the dynamical fluctuation. We also have found that an inhomogeneous molecular charge accompanied by bond alternation is non-negligible in the AF states.

Methods

Single crystals of P-salt 1, P-salt 2, Sb-salt 1 and Sb-salt 2 were synthesised by aerial oxidation from the acetone solution of [(CH3)4P]2[Pd(dmit)2], [(C2H5)2(CH3)2P]2[Pd(dmit)2], [(CH3)4Sb]2[Pd(dmit)2] and [(C2H5)(CH3)3Sb]2[Pd(dmit)2], respectively17. The thickness of the single-crystal of Sb-salt 2 was ca. 10 μm, and the area of the 2D plane (i.e. the crystallographic ab-plane) was ca. 200 × 200 μm2. The a- and b-polarised IR-reflectance spectra of Sb-salt 2 were measured at the Institute for Molecular Science (IMS) using a Thermo Nicolet Nexus 8700 equipped with a SpectraTech IR-Plan Microscope. The b-direction corresponds to the transverse direction in Fig. 1(c) and (d), while the a-direction is perpendicular to the transverse direction. The a-polarised IR-reflectance spectra of P-salt 2 were also measured at IMS. The c*-polarised IR-reflectance spectra of P-salt 1, Sb-salt 1 and Sb-salt 2 were measured at Beamline No.43IR in SPring-8 using Bruker IFS120HR, since the spatial resolution of the equipment (10 × 10 μm2) was suitable for measurements on the crystal edge. However, the absolute reflectivity could not be obtained. The conductivity spectra were obtained by a Kramers–Kronig transformation from the IR-reflectance spectra. The Raman spectra of P-salt 1, P-salt 2, Sb-salt 1 and Sb-salt 2 were measured at the IMS using a Ramascope (Renishaw) with the backward scattering configuration. The wavelength of the incident light was 633 nm from the He-Ne laser, which was suitable for observing the C=C stretching modes. Samples used to measure both the IR-reflectance and Raman spectra were cooled using a He-flow cryostat. The detailed experimental set-up for m-salt, t-salt, Sb-salt 3 and Cs-salt has been reported elsewhere37,38,39,40.