## Introduction

Nowadays, the world has been forced to adapt to the the Covid-19 living conditions. Thus, there has been a high demand and compulsory need for the human race to work at home/indoors with no/minimum physical interaction. This has resulted in a high dependence on the internet and wireless communications networks. Hence, the need for wireless communications with reliable link margins and high data rates. Multiple-input multiple-output (MIMO) systems have been shown to improve/enhance spectral efficiency (SE) and /or improve the link reliability of wireless networks against multi-path fading1. The idea behind MIMO systems is to improve the error performance (EP) of wireless communication networks using spatial diversity and/or enhancing SE using spatial multiplexing1.

Conventional spatial modulation (C-SM) is one of many MIMO techniques that can improve SE by the utilization of both the spatial constellation (transmit antenna indices) and the signal constellation2. In C-SM, information bits are categorized into two; the bits for transmitting antenna indices and the symbols bits2. Only a single transmit antenna per channel use is needed in C-SM. Hence, compared to other transmission schemes like Alamouti space-time block codes and vertical Bell layered space-time scheme (V-BLAST), C-SM was found to have a better EP because it precludes inter-channel interference ICI and inter-antenna synchronization3. Also, C-SM was found to be more bandwidth efficient when compared to conventional modulation with a single transmit antenna4. This is because it’s specified transmit antenna per time slot, also conveys additional information4. However, the number of physical antennas that can be employed at the transmission side limits the SE of C-SM schemes, and thus, conventional generalized spatial modulation (C-GSM) was proposed to alleviate this disadvantage5. C-GSM overcomes the C-SM disadvantage by curbing the restriction of transmit antennas being a power of two in C-SM schemes5. Thus, in C-GSM, the input data stream bits are mapped into two categories, the symbol signal bits and the spatial constellation bits (antenna pairs combination bits). In C-GSM, a unique combination/grouping of active transmit antennas per time slot is represented by an index. This unique combination/grouping relies on the input random data stream5. Thus, C-GSM is different from C-SM as two or more transmit antennas are activated per channel use as compared to one active transmit antenna per channel use in C-SM5. Hence, C-GSM enhances the overall SE by $$\log _2(N_c)$$; where $$N_c=\lfloor \log _{2}\begin{pmatrix} N_T \\ N_A\end{pmatrix} \rfloor _{2^p}$$ pairs of permissible transmit antennas for transmission, with $$\lfloor \cdot \rfloor _{2^p}$$ representing the largest integer that is less than or equal to $$(\cdot )$$, (that is, an integer p power of 2). Whereas, $$\begin{pmatrix} \cdot \\ \cdot \end{pmatrix}$$ represents the binomial coefficient of the argument. $$N_T$$ is the total number of transmit antennas for the C-GSM scheme, and $$N_A$$ has been discussed before in5. The chosen value of use for $$N_A$$ in this paper, is 2.

In the past decade, an effort to improve the EP and/or SE of SM systems, led to the derivation of conventional-quadrature spatial modulation (C-QSM) in Mesleh et al.6. C-QSM extends the spatial constellation into two dimensions (the in-phase (I) and quadrature (Q) dimensions). The I dimension is for transmitting the real part of a single amplitude/phase modulated symbol, and the Q dimension is for transmitting the imaginary part of the single amplitude/phase modulated symbol6,7. The I components are modulated into cosine carriers and Q components are modulated into sine carriers, respectively. Hence with that, C-QSM eliminates ICI, thereby enhancing the EP of SM systems8. Additionally, C-QSM enhances the SE ($$\mathfrak {m}$$) by $$\log _2(N_T)$$ bits/s/Hz compared to the C-SM, which achieves $$\mathfrak {m}=\log _2(M) + \log _2(N_T)$$ bits/s/Hz; where M is the amplitude/phase modulation order6. In Li et al.9, C-QSM is used where indexes of the designated receive antennas ($$N_R$$) are used to convey information. Thus, in Li et al.9, an amplitude/phase modulated symbol is precoded so that only a single receive antenna is activated, thereby conveying more information to the receiver. Also, in Kim10, C-QSM has been used in antenna selection schemes to improve the EP and simultaneously reduce the detection complexity. However, despite the capability of improving the EP and SE of SM wireless communication networks, C-QSM has a disadvantage of utilising many transmit antennas as compared to conventional spatial multiplexing (C-SMux) techniques, thereby governing the improvement of the EP of SM systems11.

### Motivation

Motivated to improve the EP and/or SE of SM systems with a minimal number of transmit antennas, a generalized complex quadrature spatial modulation (GCQSM) scheme was proposed in Mohaisen et al.12. The GCQSM scheme builds on C-QSM by adding the attributes of C-GSM to the C-QSM scheme. It improves the SE by transmitting two amplitude/phase modulated symbols drawn from two different constellation sets at each channel use, using unique combinations of generalized I domain antennas and Q domain antennas. The two symbols could be transmitted from the same antenna at one point, and thus the total modulation set at the transmitter becomes the Minkowski sum of the original two constellation sets12. Therefore, compared to C-SM and C-QSM, this leads to an increase in the size of the modulation set and a decrease in the minimum distance between transmitted symbols at the transmitter of GCQSM systems, which in turn results in the degradation of the EP of the GCQSM scheme13. This results in a poor EP because the detector depends on the minimum Euclidean (MED) distance between the transmitted vector symbols14.

To further enhance/improve the EP of SM systems, we propose to build on GCQSM by equipping it with hexagonal QAM (H-QAM) constellations which have the benefits of a maximised MED (M-MED) with a relatively low peak-to-average (PA) power ratio15. H-QAM systems are QAM systems with a hexagonal structure and they are densely packed with a M-MED. In15, it was found that the structure of H-QAM systems gave them an advantage of outperforming conventional-QAM (C-QAM) systems by approximately 0.6 dB. SM and QSM systems with H-QAM were introduced by Cogen et al. in16 and in17. H-QAM was found to be more energy efficient than square QAM, rectangular QAM and cross-QAM. Hence, its use was found for many applications including multi-carrier systems, MIMO systems, SM, QSM and advanced channel coding15 and17. Cogen et al.17, also solved the error floor degradation of the EP of QSM systems by rotating the H-QAM symbols. Also, in12, Mohaisen et al. investigated the rotation of symbols in GCQSM systems which also led to an improvement in the EP of GCQSM systems. In Naidoo et al.14, and Singya et al.15, the EP of SM systems was also found to depend on the M-MED between two neighbouring constellation points and the average symbol energy. Hence, we propose to use H-QAM systems as they have been proven to have a M-MED and relatively low PA which improves the EP of SM schemes, as compared to C-QAM systems.

Thus, to improve the EP of SM systems, we propose a GQSM scheme that builds on GCQSM13, by combining it with rotated H-QAM systems like Cogen et al.17. The proposed scheme adopts rotated hexagonal-8QAM (H-8QAM) and utilises generalized combinations of the I and Q domain antennas. This is because H-8QAM has a M-MED between neighbouring symbols and a relatively low PA as compared to C-8QAM18. Also, the proposed scheme uses generalized antenna combinations because they help enhance and/or improve EP and SE5. Thus, the benefits of GCQSM and rotated H-QAM systems lead to an improved EP. The H-8QAM concept could be extended to other various H-QAM constellations like H-32QAM and others, however as discussed in18, this results in a poor EP as the size of the hexagonal lattice structure increases. Hence the proposed scheme utilises only H-8QAM as it is the optimum H-8QAM compared to others of the same family18. A constellation comparison diagram between rotated H-8QAM and C-8QAM is included in "Appendix 1" of this paper. The creation of H-8QAM symbols was discussed in18. It was also well covered including the solution of the error floor in the EP of QSM systems by Cogen et al.17. This paper adopts the same way of creation and rotation of H-8QAM symbols like in17.

In summary, this paper, we propose a GQSM scheme that improves the EP of SM systems by equipping GCQSM systems with rotated H-8QAM17. The proposed scheme uses the C-QSM approach of splitting two rotated H-8QAM symbols into their respective I and Q components. The split symbols are then transmitted by generalized combinations of the I and Q domain antennas like in C-QSM. This helps with eliminating ICI and thereby further enhancing the EP of the proposed scheme, since the I and Q components of the C-QSM scheme are modulated into cosine and sine carriers8. Thus, the main core contributions of this paper are:

• Proposal of a new GQSM scheme called GSM with H-QAM (GQSM-H-8QAM), that improves the EP of GCQSM systems by equipping them with rotated H-8QAM. The H-8QAM systems have a lattice structure that has a M-MED and relatively low PA compared to C-QAM. This helps improve the EP of SM schemes.

• Derivation of an upper bound average bit error rate (ABER) expression for the GQSM-H-8QAM scheme over independent and identically distributed (i.i.d) Rayleigh frequency-flat RF-F fading channels.

• Validation of the derived analytical bound using Monte Carlo simulation results.

Finally, the paper is organised as follows. We introduce our proposed GQSM-H-8QAM system in Section "System model". It is presented in the form of a MIMO $$N_T \times N_R$$ H-8QAM system shown in Figs. 1 and 2. For this system, there exists $$\mathfrak {n}=\begin{pmatrix}N_T \\ N_A\end{pmatrix}$$ possible transmission antenna pairs of which only $$N_c$$ pairs are allowed for transmission, with $$N_c\le \mathfrak {n}$$. The receiver computational complexity analysis (CCA) based on real valued multiplications and additions of the GQSM-H-8QAM scheme is then presented in Section "MLD CCA of the proposed GQSM-H-8QAM scheme-based on real valued multiplications and additions", followed by the performance analysis of the proposed scheme in Section "Proposed error performance analysis of GQSM-H-8QAM". Section "Simulation and numerical results analysis" then dwells on the results and performance comparisons. Finally, we conclude the paper in Section "Conclusion".

### Notation

$$(\cdot )^*$$ represents the conjugate and $$|\cdot |$$ denotes the Euclidean norm operator. We use bold lowercase letters to denote column vectors and uppercase letters for matrices, respectively. Scalar quantities are then represented by regular letters. For a set of $$S\times L$$ complex matrices, we use $$\mathbb {C}^{S\times L}$$. For complex arguments, we use $$\Re (\cdot )$$ and $$\Im (\cdot )$$ to represent the real and imaginary parts, respectively. $$Q(\cdot )$$ is for representing the Gaussian Q-function, $$E\{\cdot \}$$ is for denoting the expectation operator, $$\underset{w}{{\text {argmin}}}\{{\cdot }\}$$ and $$\underset{w}{{\text {argmax}}}\{{\cdot }\}$$ represent the minimum or maximum value of an argument with respect to w, respectively. $$[\cdot ]^T$$ and $$(\cdot )^H$$ are the transpose and Hermitian operators, respectively. Finally, $$\left\Vert \cdot \right\Vert _F$$ denotes the Frobenius norm operator.

## System model

For the proposed scheme in Figs. 1 and 2, the input data stream is divided into two categories, the spatial input bits (antenna pair indices) and two rotated H-8QAM symbol bits. The first input category is the spatial information. Spatial input bits ($$\varvec{d}=\log _{2}N_c$$) are assigned to a unique generalized $$K{\rm th}$$ antenna pair of transmit antennas ($$k_1$$ and $$k_2$$), which are for the I dimension of the system. The same $$\varvec{d}$$ bits are assigned to a second but different generalized $$K{\rm th}$$ antenna pair of transmit antennas ($$k_3$$ and $$k_4$$), which are for the Q dimension of the system. An example is where $$N_T=4$$, two possible sets of antenna pairs $$\begin{bmatrix}(1,3); (1,4); (2,3); (2,4)\end{bmatrix}$$ for the I dimension and $$\begin{bmatrix}(2,4); (2,3); (1,4); (1,3)\end{bmatrix}$$ for the Q dimension are assigned the same bit indices $$\begin{bmatrix}00; 01; 10; 11\end{bmatrix}$$. A mapping table of the grouped bits (I dimension transmit antenna pairs and Q dimension transmit antenna pair combinations), is given in Tables 1 and 2. The antenna mapping for the I and Q dimensions are different in order to avoid ICI or the use of the same pair of antennas to simultaneously transmit a mixed signal of the real and imaginary parts. This enhances the EP of the proposed system.

The second category of the input data stream is the symbols bit stream. A bit stream $$\varvec{b}=\begin{bmatrix} b_1 b_2 ... b_{2r}\end{bmatrix}$$, with $$r=$$log$$_2$$ M is fed into a mapper ($$\mathfrak {G_1}$$). In mapper ($$\mathfrak {G_1}$$), the 2r input bits are mapped onto a rotated H-8QAM signal constellation set to yield two symbols $$S _{q}^{1}$$ and $$S _{q}^{2}$$, where $$q=1+\sum _{v=1}^{2r}2^{2r-v}b_v$$, $$q\in [1:M^2]$$. The creation of the H-8QAM symbol constellation and its rotation is well discussed by Cogen et al.17. This paper adopts the same way of creating H-8QAM systems. Also, Cogen et al.17, discusses the degrading effect of error floor on the EP of QSM systems when using H-QAM without rotation. Thus, Cogen et al.17, then devised a solution of using optimum rotation angles on H-QAM-QSM systems to curb the error floor on the EP of H-QAM-QSM schemes. The optimum rotation angle for the chosen H-8QAM schemes in this paper, is $$\frac{\pi }{4}$$17. The mapping of the rotated H-8QAM symbols to bits is provided in Table 3 and Fig. 10. The SE of the proposed GQSM-H-8QAM scheme is given as,

\begin{aligned} \mathfrak {m}=2\log _{2}(M) + \lfloor \log _{2}\begin{pmatrix} N_T \\ 2\end{pmatrix} \rfloor _{2^p}. \end{aligned}
(1)

The modulated symbols ($$S _{q}^{1}$$ and $$S _{q}^{2}$$) are split into their respective real parts and imaginary parts ($$S _{q}^{1I}+ \Im { S _{q}^{1Q}}$$ and $$S _{q}^{2I}+\Im { S _{q}^{2Q}}$$). The real parts of the symbols are transmitted via the I dimension using the in-phase transmit antenna pairs ($$k_1$$ and $$k_2$$). The imaginary parts of the symbols are in turn transmitted through the Q dimension using the quadrature antenna pairs ($$k_3$$ and $$k_4$$). Thus, the split symbols are transmitted by four transmit antennas concurrently per time slot, in the form of an $$N_T\times 1$$ transmit vector $$\textbf{x}$$ shown in Fig. 3. Vector $$\textbf{x}$$ has four non-zero elements, which are the two real parts ($$S _{q}^{1I}$$ and $$S _{q}^{2I}$$) and the two imaginary parts ($$S _{q}^{1Q}$$ and $$S _{q}^{2Q}$$) of the symbols. Moreover, the positions of these four nonzero elements correspond to the indices of the four chosen active transmit antennas. $$\theta _k$$ is the rotation angle applied to different transmit antenna pairs as shown in Tables 1 and 2. This is done in order to further minimise ICI that could be generated by overlapping antenna pairs14. The discussion of the selection of the antenna pairs and the optimum $$\theta _k$$ was discussed in14 and19. Hence, Tables 1 and 2 show the chosen transmit antenna pairs and the corresponding optimum $$\theta _k$$ values used in this paper according to14.

At the receiver, the signal is given by $$\textbf{y}\in \mathbb {C}^{N_R\times 1}$$ (2), as

\begin{aligned} \textbf{y}&=\sqrt{\frac{\rho }{4}}\textbf{H}\textbf{x} + \textbf{n}, \nonumber \\&=\sqrt{\frac{\rho }{4}}\bigg (\left( \textbf{h}_{k_1} S _{q}^{1I} + \textbf{h}_{k_2} S _{q}^{2I}\bigg )e^{j\theta _k} + \Im \bigg (\textbf{h}_{k_3} S _{q}^{1Q} + \textbf{h}_{k_4} S _{q}^{2Q}\bigg )e^{j\theta _k} \right) + \textbf{n}, \end{aligned}
(2)

where the fading channel matrix is represented by $$\textbf{H}\in \mathbb {C}^{N_R\times N_T}$$ and $$\textbf{n}\in \mathbb {C}^{N_R\times 1}$$ is the AWGN noise vector. $$\textbf{h}_{k_l}(1\le l\le N_T)$$ is the lth column vector of the channel gain matrix $$\textbf{H}=\begin{bmatrix}\textbf{h}_{k_1}&\textbf{h}_{k_2}&\textbf{h}_{k_3}&...&\textbf{h}_{k_{N_T}}\end{bmatrix}$$ and $$\textbf{h}_{k_l}=\begin{bmatrix}\textbf{h}_{1k_l}&\textbf{h}_{2k_l}&\textbf{h}_{3k_l}&...&\textbf{h}_{N_Rk_l}\end{bmatrix}^T$$. The elements of both $$\textbf{n}$$ and $$\textbf{H}$$ are assumed to be i.i.d Gaussian random variables (GRVs) with distribution $$CN (0,1)$$, respectively. Finally, $$\frac{\rho }{4}$$ is the average signal-to-noise ratio (SNR) per receive antenna.

The optimum maximum likelihood detection (MLD) for the proposed scheme, can be written as in (3) and it entails a joint estimation of the transmission antenna pairs indices ($$\hat{K}$$) and the sent symbols ($$S _{\hat{q}}^{1}$$ and $$S _{\hat{q}}^{2}$$)11 and12, as follows,

\begin{aligned} \begin{bmatrix} \hat{K}, S _{\hat{q}}^{1}, S _{\hat{q}}^{2}\end{bmatrix}&=\underset{K\in 1:N_c}{\underset{q\in 1:M^2}{{\text {argmin}}}}\bigg \{{\left\Vert \textbf{y}-\sqrt{\frac{\rho }{4}}\textbf{H}\hat{\textbf{x}}\right\Vert _F^2}\bigg \}, \nonumber \\&=\underset{K\in 1:N_c}{\underset{q\in 1:M^2}{{\text {argmin}}}}\bigg \{{\left\Vert \mathbf {\mathfrak {g}}\right\Vert _F^2}-2\Re \bigg \{\textbf{y}^H\mathbf {\mathfrak {g}}\bigg \}\bigg \}, \end{aligned}
(3)

where $$\mathbf {\mathfrak {g}}=\sqrt{\frac{\rho }{4}}\bigg (\left( \textbf{h}_{k_1} S _{q}^{1I} + \textbf{h}_{k_2} S _{q}^{2I}\bigg )e^{j\theta _k} + \Im \bigg (\textbf{h}_{k_3} S _{q}^{1Q} + \textbf{h}_{k_4} S _{q}^{2Q}\bigg )e^{j\theta _k} \right)$$ and $$\hat{\textbf{x}}$$ is the estimated received vector shown in Fig. 4, respectively.

## MLD CCA of the proposed GQSM-H-8QAM scheme-based on real valued multiplications and additions

The CCA of the GQSM-H-8QAM scheme is similar to that discussed in Holoubi et al.20. It is based on real value multiplications and additions. In20, it is given as $$(20N_R-1)2^{\mathfrak {m}}$$. In this section, all the incorporated CCAs of various schemes in this paper (including enhanced spectral efficiency GSM (ESE-GSM), GSM multiplexing two symbols(MIMO-GSM) and constellation reassigned GSM (GSM-CR)), have been discussed before in6 and21. As seen in Tables 4 and 5, the GQSM-H-8QAM scheme has the highest CC as compared to all the schemes included in this paper. GQSM-H-8QAM has an optimal MLD at the expense of a high CC. However, it is essential to also consider that the proposed scheme uses fewer transmit antennas as compared to various MIMO-SM schemes under the same conditions. An example is shown in Tables 9 and 10 found in "Appendix 1". Table 10 shows that the proposed scheme requires 4 transmit antennas to achieve a SE of $$\mathfrak {m}=8$$ bits/s/Hz as compared to C-SM and C-GSM with $$N_T=32$$ and 10, to achieve the same SE under the same conditions, respectively. Hence, the disadvantage of a high CC in the proposed scheme, is compensated by the advantage of using a less number of transmit antennas as compared to other various MIMO-SM schemes incorporated in this paper. Due to a high CC shown by the proposed scheme, as upcoming work, we intend to formulate low complexity detection algorithms to overcome this disadvantage.

## Proposed error performance analysis of GQSM-H-8QAM

In this section, the ABEP of GQSM-H-8QAM is formulated. The detection discussed in section "System model" follows the same approach as6,11 and12. Hence the ABEP of GQSM-H-8QAM is upper-bounded by

\begin{aligned} P_e \le \frac{1}{2^\mathfrak {m}} \sum _{q=1}^{2^\mathfrak {m}} \sum _{\hat{q}=1}^{2^\mathfrak {m}} \frac{1}{\mathfrak {m}^2}N\left( q,\hat{q}\right) P(\textbf{x}\rightarrow \hat{\textbf{x}}), \end{aligned}
(4)

where $$N\left( q,\hat{q}\right)$$ is the total number of bit errors for the associated pairwise error probability (PEP) event $$P(\textbf{x}\rightarrow \hat{\textbf{x}})$$ between the transmitted vector $$\textbf{x}$$ and the received vector $$\hat{\textbf{x}}$$. The PEP $$P(\textbf{x}\rightarrow \hat{\textbf{x}})$$ is formulated as

\begin{aligned} P(\textbf{x}\rightarrow \hat{\textbf{x}}|\textbf{H})=Q\left( \sqrt{\omega _g}\right) , \end{aligned}
(5)

where $$\omega _g=\frac{\rho }{8}\left\Vert \left( \textbf{h}_{k_1}d_1 + \textbf{h}_{k_2}d_2+ \textbf{h}_{k_3}d_3 + \textbf{h}_{k_4}d_4\right) \right\Vert _F^2=\frac{\rho }{8}\left\Vert \textbf{H}_k\textbf{G}_k\right\Vert _F^2= \frac{\rho }{8}\left\Vert \textbf{H}_k\right\Vert _F^2\left\Vert \textbf{G}_k\right\Vert _F^2= \frac{\rho }{8}\left\Vert \textbf{H}_k\right\Vert _F^2\left( (d_1)^2+(d_2)^2+(d_3)^2+(d_4)^2\right)$$, with $$\textbf{H}_k=\begin{bmatrix} \textbf{h}_{k_1}&\textbf{h}_{k_2}&\textbf{h}_{k_3}&\textbf{h}_{k_4}\end{bmatrix}$$, $$\textbf{G}_k=\begin{bmatrix}d_1&d_2&d_3&d_4\end{bmatrix}^T$$, $$d_i=\left( S _q^{1I}- S _{\hat{q}}^{1I}\right) e^{j\theta _k}$$ and $$d_{(l+2)}=\left( S _q^{lQ}- S _{\hat{q}}^{lQ}\right) e^{j\theta _k}$$, $$\{i,l\}\in [1:2]$$. Similarly, based on (11) in Koc et al.22, $$\omega _g$$ are chi-squared RVs with $$2N_R$$ degrees of freedom defined as $$\omega _g=\sum _{t=1}^{2N_R} \alpha _{\omega _{g,t}}^2$$ with $$\alpha _{\omega _{g,t}}^2\sim N(0,\sigma _{\omega _g}^2)$$ and $$\sigma _{\omega _g}^2=\frac{\rho }{8}\big ((d_1)^2+(d_2)^2+(d_3)^2+(d_4)^2\big )$$. The derivation of the PEP is found in "Appendix 2".

Upon further simplification by taking the expected value of $$\omega _g$$, the average PEP assuming $$N_R$$ receive antennas is adopted and modified from6,11 and12, to give

\begin{aligned} P(\textbf{x}\rightarrow \hat{\textbf{x}})= \gamma ^{N_R} \sum _{k=0}^{N_R-1} \begin{pmatrix} N_R-1+k\\ k\end{pmatrix} \begin{bmatrix} 1-\gamma \end{bmatrix}^k, \end{aligned}
(6)

where $$\gamma =\frac{1}{2}\left( 1-\sqrt{\frac{\sigma _{\omega _g}^2}{1+\sigma _{\omega _g}^2}}\right)$$.

Hence, taking the Taylor series of (6) and using the same approach as6 and20 at high SNRs, (6) leads to (7) when higher-order terms are neglected.

\begin{aligned} P(\textbf{x}\rightarrow \hat{\textbf{x}})= \frac{2^{2N_R-1}\Gamma (N_R+0.5)}{\sqrt{\pi (N_R)!}}\bigg (\sigma _{\omega _g}^2\bigg )^{-N_R}. \end{aligned}
(7)

The overall diversity gain ($$G_c$$) attained by a MIMO-SM scheme has been defined in23, as

\begin{aligned} -G_c=\lim _{ SNR \rightarrow \infty } \bigg \{\frac{\log \big (P_b( SNR )\big )}{\log ( SNR )}\bigg \}, \end{aligned}
(8)

where $$P_b$$ denotes overall probability of error as a function of SNR. Hence given $$P_b(SNR)=P(\textbf{x}\rightarrow \hat{\textbf{x}})$$ as given in (7), then (8) becomes

\begin{aligned} -G_c=\lim _{\rho \rightarrow \infty } \bigg \{\frac{\log \big (P(\textbf{x}\rightarrow \hat{\textbf{x}})\big )}{\log (\rho )}\bigg \}. \end{aligned}
(9)

Hence substituting (7) into (9), leads to (10).

\begin{aligned} -G_c=\lim _{\rho \rightarrow \infty } \bigg \{\frac{\log \big (\beta \rho ^{-N_R}\big )}{\log (\rho )}\bigg \}, \end{aligned}
(10)

where $$\beta =\bigg (\frac{2^{2N_R-1}\Gamma (N_R+0.5)}{\sqrt{\pi (N_R)!}}\bigg )\bigg (\frac{1}{8}\bigg )^{-N_R}\bigg ((d_1)^2+(d_2)^2+(d_3)^2+(d_4)^2\bigg )^{-N_R}$$.

Since $$\lim _{\rho \rightarrow \infty } \bigg (\frac{\log (\beta )}{\log (\rho )}\bigg )=0$$, therefore $$G_c=N_R$$. Hence the diversity gain of the proposed system is $$N_R$$.

## Simulation and numerical results analysis

This section presents simulation results for the proposed GQSM-H-8QAM system with a different number of transmit antennas and comparisons with C-SM, C-GSM, C-QSM, MIMO-GSM and ESE-GSM. The theoretical ABEP for C-SM schemes was based on (9) in Naidoo et al.4, that of C-GSM schemes was based on (5) in Pillay et al.19, and that of ESE-GSM was based on (5) and (12) in Pillay et al.19. The theoretical ABEP of C-QSM was based on (3) and (4) in Oladoyinbo et al.8, and that of MIMO-GSM was based on (16) in Wang et al.21. Another aim of this section is to validate the theoretical performance bound derived in (4). The average bit error rate BER of the proposed scheme was evaluated using Monte Carlo simulations over i.i.d RF-F fading channels with AWGN. The BER performance was assessed for various SEs ($$\mathfrak {m}=8$$ bits/s/Hz, $$\mathfrak {m}=9$$ bits/s/Hz and $$\mathfrak {m}=10$$ bits/s/Hz) as a function of the average SNR per receive antenna ($$\rho /4$$). These three SEs considered, are shown in Figs. 5, 6, 7, 9. The comparisons were made under the same conditions, which are; four receive antennas assumed in all scenarios, identical SE, optimal MLD method, and at a BER value of $$1\times 10^{-5}$$ for comparison. The formulae for evaluating the SEs of all the various scheme in Figs. 5, 6, 7, 9 are found in "Appendix 1", Table 8.

Firstly, Fig. 5 presents the BER performance curves of the GQSM-H-8QAM scheme with $$N_T=4, 6$$ and 8, respectively. In Fig. 5, the analytical results are validated by simulation results and it is seen that they are tighter as the SNR values increase. They are tight at high SNR values because of the upper-bounded ABER performance in all the configurations of the GQSM-H-8QAM scheme.

Secondly, Figs. 6 and 7 with Table 6 show the simulation results and the calculated EP gains of the proposed $$4\times 4$$ GQSM-H-8QAM scheme over $$4\times 4$$ various schemes of the same SE ($$\mathfrak {m}=8$$ bits/s/Hz). As seen from Figs. 6 and 7 together with Table 6, the proposed scheme has an EP of 0.61 dB over $$4\times 4$$ GCQSM-C-8QAM, 0.93 dB over $$4\times 4$$ GSM-CR-64QAM scheme and 0.75 dB over $$4\times 4$$ MIMO-GSM-C-8QAM scheme. Also, Wang et al.21, discussed MIMO-GSM and it was shown to outperform C-SM and C-GSM schemes of the same SE. In14, it is also seen that GSM-CR outperforms C-SM and C-GSM schemes. Thus, the proposed scheme has performance gain over all the schemes included in this paper, as it outperformed the best of those discussed in literature14,21. This might be because of the use of generalised antennas and H-8QAM with a M-MED that improved the EP of the proposed scheme.

Thirdly, Fig. 8 shows the proposed GQSM-H-8QAM scheme performance compared with the same schemes in Figs. 6 and 7 at a SE of $$\mathfrak {m}=9$$ bits/s/Hz, $$N_R=4$$ but under different number of transmit antennas. Fourthly, Fig. 9 shows the proposed GQSM-H-8QAM scheme performance compared with the same schemes in Fig. 6 but under different SEs and different number of transmit antennas. In Fig. 9, the $$8\times 4$$ GQSM-H-8QAM scheme is compared with other various $$8\times 4$$ schemes with a SE of $$\mathfrak {m}=10$$ bits/s/Hz. Figure 9 together with Table 7 also exhibits the same behavior as Figs. 6 and 7. The proposed scheme outperformed all the various schemes that were compared to it (Fig. 10).

It is also important, to note that the proposed scheme (GQSM-H-8QAM), has another advantage of using a less number of transmit antennas under conditions of fixed $$N_R$$ and fixed M at a certain SE ($$\mathfrak {m}$$) as compared to the other systems included in this paper. An example is shown in Tables 9 and 10 found in "Appendix 1", which show the number of $$N_T$$s required to achieve a certain SE at $$N_R=4$$. Table 9 shows that the GQSM-H-8QAM scheme requires 4 transmit antennas to achieve a SE of $$\mathfrak {m}=8$$ bits/s/Hz using H-8QAM as compared to C-SM which requires $$N_T=32$$, C-GSM which requires $$N_T=10$$ and ESE-GSM which requires $$N_T=8$$ to achieve the same SE under the same settings. This shows that the GQSM-H-8QAM scheme has an advantage of less hardware costs as it requires a less number of transmit antennas as compared to the other systems. The only exception seen in Tables 9 and 10, is the MIMO-GSM scheme which requires the same number of transmit antennas as the proposed scheme to achieve the same SE under identical settings as the proposed scheme. However, it is outperformed by the proposed scheme as seen from the results in Figs. 6, 7, 9.

## Conclusion

In this paper, GQSM-H-8QAM scheme was proposed to improve the EP of MIMO-SM schemes. The scheme builds on GCQSM and uses energy efficient H-8QAM instead of the C-8QAM, to improve the EP of MIMO-SM schemes. Also, in this paper, an upper bound analytical EP formula for the proposed scheme in i.i.d (RF-F) fading channels was derived. This analytical bound was validated using Monte Carlo simulations. The results showed that the derived analytical bound is increasingly tight with increasing SNR values. Furthermore, the proposed scheme outperformed GCQSM, C-SM, C-QSM, MIMO-GSM, C-GSM and other various MIMO-SM schemes incorporated in this paper. Hence, from the results obtained, it can be concluded that the proposed scheme is a viable option of improving the EP of MIMO-SM schemes. However, despite its capability of improving the EP of MIMO-SM schemes, the proposed scheme was found to have a high complexity MLD. Hence, for future work the authors intend to work on low complexity detection methods for the proposed scheme. In addition, the authors also intend to build on the proposed scheme by fusing it with uncoded space-time labelling diversity and golden code-words to improve/enhance the SE of MIMO-SM systems.