IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

1

Performance Evaluation for Widely Linear Demodulation of PAM/QAM Signals in the Presence of Rayleigh Fading and Co-channel Interference Kiran Kuchi, Member, IEEE, Vasant K. Prabhu, Life Fellow, IEEE

Abstract—In this paper, widely linear filtering (WLF) techniques are applied to the demodulation of both pulse amplitude modulation (PAM) and quadrature amplitude modulation (QAM) systems with multiple receiver antennas and multiple cochannel interferers. In the proposed implementation, the existing correlations between the real (I) and imaginary (Q) parts of the complex-valued baseband received signal are exploited using a widely linear (WL) maximum likelihood (ML) receiver. Several bounds and approximations are developed to analyze the average symbol error rate (SER) performance, and to analyze the tradeoff between the diversity advantage and interference cancellation (IC) gain in a flat Rayleigh fading channel. Two new results are shown. First, we show that the IC ability of WL receivers is independent of the modulation type used by the desired signal but the gain depends mainly on the modulation type employed by the individual interferers and the total number of antennas. Secondly, assuming that the system uses a mixture of PAM and QAM signals, we show that a WL receiver with N antennas can reject any combination of M1 PAM, M2 QAM interferers satisfying the constraint: M1 + 2M2 < 2N with an asymptotic diversity order N − M21 − M2 . In contrast, a conventional receiver whose performance is independent of the modulation characteristics of the individual interferers can reject only up to M1 + M2 interferers, where M1 + M2 < N with an asymptotic diversity order N − M1 − M2 . When the system contains either PAM, or a mixture of PAM and QAM type CCI, it is shown that WL processing offers a significant performance advantage with a moderate increase in the receiver complexity. Index Terms—Widely Linear Filtering, Minimum Mean Square Estimation, Diversity, Interference Suppression.

I. I NTRODUCTION N wireless networks, fading caused by the multi-path signal propagation and co-channel interference (CCI) generated by the frequency re-use (spectrum sharing) mechanism are among the major impairments that limit the system capacity. Multiple antenna receivers have been widely used to mitigate both these impairments to enhance link and system performance [1]– [5]. In particular, non-linear schemes such as joint detection (JD) [6], and linear minimum mean-square error (MMSE) combining (also known as optimum combining) have been previously considered. The JD scheme, in spite of its

I

Paper approved by M. Chiani, the Editor for Wireless Communication. Manuscript received April 19, 2006; revised March 11, 2007. This work was done when the first author was with The University of Texas at Arlington, USA (e-mail: [email protected]) V.K.Prabhu is with The University of Texas at Arlington, USA. Digital Object Identifier 10.1109/TCOMM.2009.060565

ability to provide full diversity gain and excellent interference cancellation (IC) performance [6], has found limited application in cellular type systems where it is difficult to reliably estimate the channel state information (CSI) of individual cochannel interferers which is a requirement for this technique. Moreover, implementation complexity for this method grows exponentially with the modulation size of signal/interference and the total number of interferers. On the other hand, the sub-optimum MMSE receiver [6] which relies on the received signal covariance information for IC is more suitable for low-complexity systems. Since this technique maximizes the signal-to-noise-plus-interference-ratio (SINR) at the output of the MMSE filter, it provides a significant advantage over traditional maximal-ratio-combining (MRC) receiver which treats interference as circular additive white Gaussian noise (AWGN). In [2], Winters et al. have shown that a MMSE receiver equipped with N antennas can fully reject M cochannel interferers with a diversity order N −M , for M < N . Since reliable estimates of CSI of the desired signal and the total noise covariance can be obtained in practice, for instance using a training sequence, the MMSE-type methods have gained widespread application. In this regard, widely linear (WL) MMSE [7] methods that jointly filter the complex and complex-conjugate copies (alternatively the real (I) and imaginary (Q) parts) of the baseband received signal [8], [9] are preferable to conventional MMSE-type methods especially in systems that encounter non-circular signals [10]. Widely linear filtering (WLF) provides the gain by exploiting the pseudo-covariance (i.e. the covariance between the complex-valued received signal and its complex-conjugate) together with conventional signal covariance information in deriving the MMSE weights. In general, systems employing real modulation alphabets such as binary phase-shift-keying (BPSK), offset-quadrature phaseshift-keying (O-QPSK), minimum-shift-keying (MSK), and Gaussian minimum-shift-keying (GMSK) modulated signals when convolved with complex-valued baseband channel can be modeled as non-circular (rotationally variant, or improper) random processes i.e., processes with non-vanishing pseudocovariance. Many different types of receivers based on WLF techniques have been proposed for these systems to enhance equalization and interference suppression performance (see e.g., [11]– [19]).

c 2009 IEEE 0090-6778/09$25.00 

2

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

Especially for pulse amplitude modulation (PAM) systems, with PAM-type interference, splitting the complex-valued received signal1 into real and imaginary parts creates two copies of signal and interference with real-valued channel gains. In [20], [21] it has been shown that a WL receiver that works with the I-Q branches provides significant IC advantage for a receiver with a single antenna only. This single antenna interference cancellation (SAIC) feature has been successfully adopted for GMSK modes of Global System for Mobile communications (GSM) standard. Recently, in [22], the SAIC concept has been further generalized for PAM systems with multiple receiver antennas. In the existing literature, WLF has not yet been applied for quadrature amplitude modulation (QAM) systems with PAM/QAM-type interference. In this paper, it is shown that both PAM and QAM receivers can benefit from WL processing in systems where interference consists of either PAM or a mixture of PAM and QAM signals. As such, the proposed receivers can be utilized to enhance performance of Enhanced Data rates for GSM Evolution (EDGE) systems which employ a combination of GMSK and 8-ary phase-shift-keying (8-PSK) signaling formats. In this paper, we first introduce WL receiver structures for PAM/QAM systems with multiple antennas and multiple interferers in a flat Rayleigh fading channel. To keep the analysis at a general level, we assume a mixed interference model that consists of both PAM and QAM CCI components. Next, we derive several bounds and approximations to determine the symbol error rate (SER). These results are used to analyze the diversity-IC tradeoff for WL systems. The results of this paper are distinct from the previous work in the following aspects: •

•

In the communications literature, WLF is typically referred to a method where the real and imaginary parts of the complex-valued baseband received signal are weighed and summed before applying conventional symbol demodulation. This approach is typically useful for PAMtype signals that employ real constellations only. In case of QAM, where the modulation signal contains both real and imaginary parts, we show that the full benefit of WLF can be obtained only if the demodulator takes into account the residual correlations between the I-Q parts of the data signal during ML symbol detection, in addition to pre-filtering of I-Q signal branches. With this type of receiver, we show that the gain of WLF is independent of the modulation type employed by the desired signal but the gain depends mainly on the modulation type employed (that is whether PAM or QAM is being used) by the individual interferers. In a flat Rayleigh fading channel, the SER calculation typically requires calculation of expectation of a certain complementary error function with respect to the eigenvalues of the interference correlation matrix (ICM). Closed form evaluation of the expectation operation requires a multiple integral that has no reported closed form solution for the general case of arbitrary number of antennas with arbitrary number of interferers [23], [24]. Chiani et al. in [25], [26], addressed this problem

1 The wireless channel is generally characterized as a complex-valued channel with non-zero real and imaginary parts

x(k )

h

y (k )

n(k ) Fig. 1.

•

Discrete baseband signal model

for the conventional case by developing several useful bounds and approximations for the case of multiple interferers with equal average power. In this paper, we generalize certain key results of [25] for receivers applying WL processing. In particular, for systems with PAM-type interference, the statistical properties of real Wishart matrices [27] are utilized to derive several closed form approximations to the SER. For channels with a mixture of PAM and QAM interferers, the SER is well approximated using the mean eigenvalues of the ICM which are obtained using Monte Carlo simulation. Our analysis shows that WL processing offers a substantial gain over systems employing conventional MMSE combining for both PAM and QAM signal detection in the presence of interference. In particular, we show that a WL receiver with N antennas can reject any combination of M1 PAM, M2 QAM interferers satisfying the constraint: M1 +2M2 < 2N , with an asymptotic diversity order N − M21 − M2 . In contrast, a conventional receiver whose performance is independent of the modulation characteristics of the individual interferers can reject only up to M1 +M2 interferers, where M1 +M2 < N , with an asymptotic diversity order N − M1 − M2 . These results convey a significant improvement in diversity advantage and/or IC gain when the system contains PAM-type interference.

The rest of the paper is organized as follows. After introducing the system model in Section II, section III discusses the structure and implementation of WL receivers for PAM and QAM cases, respectively. In Section IV, the SER with WLF is formulated. In Section V, several bounds and approximations are given. The tradeoff between diversity and IC gain is discussed in Section VI. In Section VII, the analytical results are compared with Monte Carlo simulation and Section VIII offers some concluding remarks.

II. S YSTEM M ODEL We consider widely linear receiver processing for both PAM and QAM signaling in a flat Rayleigh fading environment for a system with N antennas at the receiver, and M co-channel interferes. Throughout this paper, (.)T denotes the matrix transposition operation, and (.)† stands for the Hermitian operation. The baseband complex-valued received signal at the output of an array of antennas at time k can be represented

K. KUCHI et al.: PERFORMANCE EVALUATION FOR WIDELY LINEAR DEMODULATION OF PAM/QAM SIGNALS IN THE PRESENCE OF RAYLEIGH FADING AND CO-CHANNEL INTE

Re{n(k )}

Re{n(k )}

QAM input

Re{h}

PAM input

Re{y (k )}

⎡Re{x(k )}⎤ ~ x (k ) = ⎢ ⎥ ⎣ Im{x(k )}⎦

Re{y (k )}

⎡Re{h} − Im{h}⎤ ⎢ Im{h} Re{h} ⎥ ⎦ ⎣

x(k )

Im{y (k )}

Im{h}

Im{y (k )}

Im{n(k )}

Fig. 3.

Equivalent baseband I-Q signal model for QAM

Im{n(k )} Fig. 2.

Equivalent baseband I-Q signal model for PAM

as (see Fig 1): y(k) =

M   √ Chx(k) + Il gl xl (k) + w(k), l=1





n(k)

(1)



where C and Il are the average energies of the signal and lth interferer, respectively; x(k) and xl (k) are data samples (with real/complex values depending on whether PAM or QAM is being used) of the desired and lth interferer both with unit variance; The desired and interfering signal channel vectors h and gl are modeled as multi-variate circular complex Gaussian random vectors having independent, and identically distributed (i.i.d.) elements with E[h] = E[gl ] = 0 and E[hh† ] = E[gl gl† ] = I, where I denotes the identity matrix, and E denotes the expectation operator. The thermal noise term w(k) is also modeled as a circular complex Gaussian noise vector composed of i.i.d. elements with E[w(k)] = 0 and E[w(k)w(k)† ] = N0 I, where N0 denotes the noise variance per antenna branch. In (1), n(k) denotes the total noise which includes both thermal noise and interference. Further, in case of QAM, it is assumed that the real and imaginary parts of complex modulation signals are zero-mean, uncorrelated signals with equal variance. To apply widely linear filtering, the receiver first collects the real (I) and imaginary (Q) parts of the complex-valued baseband received signal from all antennas and stacks up the real-valued I-Q output signals in a single column vector format. This type of processing leads to different types of system models for PAM and QAM cases as shown in the following. A. I-Q Signal Model for PAM To illustrate the main ideas, we first assume that the CCI contains PAM-type signals only. The extension to the general case where interference has both PAM and QAM signals is considered in section V-C. For PAM signaling, since the modulation signal takes real values, the equivalent signal model with I-Q parts of the received signal stacked a column vector format (see Fig 2) is given by: M   √ ˜ + Il g ˜l xl (k) + w(k), ˜ y ˜(k) = C hx(k) l=1

(2)

˜ where h = [Re{h1 }, Im{h1 }, .., Re{hN }, Im{hN }]T , g ˜l = [Re{g(l,1) }, Im{g(l,1) }, .., Re{g(l,N ) }, Im{g(l,N ) }]T denote the equivalent channel vectors of desired and lth interfering signals, respectively. The noise term w(k) ˜ represents the I-Q thermal noise vector with zero-mean, i.i.d. real elements with variance N20 . For convenience, M √ let n ˜(k)  Il ˜ gl xl (k) + w(k). ˜ Further, let l=1 Rn˜ n˜  E[˜ n(k)˜ nT (k)] be the short-term noise correlation matrix. Conditioned on the channel vectors of CCI, it can be obtained as:  M  Rn˜ n˜ = E[xl (k),w(k)] Il g ˜l xl (k) + w(k) ˜ × ˜ T ⎤

M  ⎦ Il g ˜l xl (k) + w(k) ˜

l=1

l=1

=

M  l=1

Il g ˜l ˜ glT +

N0 I, 2

(3)

where E[xl (k),w(k)] denotes the expectation with respect ˜ to the random variables xl (k), w(k). ˜ Further, R˜i˜i  M T I g ˜ g ˜ denotes the correlation contribution due to PAMl l l l=1 type CCI. B. I-Q Signal Model for QAM When the desired signal has QAM (where the modulation consists of both real and imaginary parts), and since the channel is assumed to be complex2, the signal model with real and imaginary parts arranged in a column vector format leads to a multiple-input-multiple-out (MIMO) signal model of type (See Fig 3): √ ˜ x(k) + n ˜(k). (4) y ˜(k) = C H˜ ˜ takes a matrix form: ˜ = Due to I-Q split, the channelH H Re{hq } Im{hq } , q = [H1 , .., HN ]T , where Hq  −Im{hq } Re{hq } 1, 2, .., N, and the modulation takes a vector form: x ˜(k) = [Re{x(k)}, Im{x(k)}]T . Note that when the signal is split into I-Q parts, the PAM system assumes a single-input-multipleoutput (SIMO) model with twice the dimensionality compared ˜ is to the conventional case (the equivalent PAM channel h 2 In wireless medium, the propagation channel always has non-zero real and imaginary parts. Especially in the Rayleigh fading case, the real and imaginary parts of the channel are assumed to be zero-mean, i.i.d. random variables

4

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

a (2N × 1) size vector) whereas the QAM system takes a MIMO signal model where the equivalent channel is a (2N × ˜ Because of the inherent differences in the 2) size matrix H. system models, the corresponding WL receivers for PAM and QAM cases exhibit significant differences both in structure and implementation as we shall see shortly.

wI

Re{}

y (k )

Threshold Demodulator

wQ

Fig. 4.

the symbol decision metric for PAM signal detection can be formulated as [30]: minx¯(k) ˜ eT (k)R−1 e(k) (5) n ˜n ˜˜ √ ˜x(k), x where ˜ e(k)  y ˜(k) − C h¯ ¯(k) and x ˆ(k) denote the candidate and detected symbols, respectively. When calculating the distance metrics, repeated multiplications involving the matrix R−1 n ˜n ˜ can be eliminated by applying a pre-whitening filter that is obtained by using the Cholesky decomposition ˜L ˜ T where L ˜ is a lower-triangular matrix [31]: Rn˜ n˜ = L with unit elements along the principal diagonal. The matrix ˜ −1 can be used as a pre-whitening filter to simplify the L ˆ x(k)|2 , distance metric (5) as: ˜ eT (k)R−1 e(k)√= |ˆ y(k) − h¯ n ˜n ˜˜ −1 −1 ˆ = CL ˜ denote the ˜ y ˜ h where y ˆ(k) = L ˜(k) and h pre-whitened signal and channel vectors, respectively. The ˜ −1 is a (2N × 2N ) real-valued filter that whitens the filter L noise contained in the I-Q signal branches corresponding to multiple antenna elements. Practical implementation of this filter requires certain additional complexity compared to a conventional MMSE system where noise whitening would be accomplished with lower complexity using a (N × N ) complex-valued filter. Now, instead of using the direct ML decision rule which requires multiple distance calculations, a whitened-matchedˆ T can be applied to the pre-whitened signal to filter (WMF) h ˆT y obtain a scalar decision variable: u(k) = h ˆ(k). Using u(k), symbol decisions can be obtained with reduced complexity using a threshold demodulator. Fig 4 depicts a schematic diagram for the simplified WL-PAM receiver implementation, ˆ I , and wQ = h ˆ Q where h ˆI , h ˆ Q represent if we set wI = h ˆ The WMF the elements corresponding to I-Q branches of h. solution can also be obtained by formulating the problem starting with WL-MMSE estimation. Using [32], the optimum WL-MMSE solution can be shown to be: x ˆ(k) = arg

= =

To facilitate the receiver derivation, we first assume that the sum of noise plus interference has Gaussian pdf4 . Under this assumption, n ˜(k) can be modeled with multi-variate Gaus−1 T 1 e−{˜n (k)Rn˜ n˜ n˜(k)} . sian distribution: p(˜ n(k)) = √ 2N (2π)

|Rn ˜n ˜|

Weighing for Q-branch

A WL-PAM Receiver

w ˜ A. WL PAM Receiver

xˆ (k )

Im{}

III. WL R ECEIVERS In case of PAM, where a real-valued signal is to be estimated using a complex-valued received signal, traditional WL-MMSE estimation [17] i.e., weighing and summing of I-Q signal branches (where the filter weights are derived by minimizing the mean square error (MSE) metric) has been considered in the literature (See Fig 4). In this case, data demodulation is typically performed using the standard ML decision rule with the assumption that the residual noise at the output of the WL-MMSE filter has Gaussian probability density function (pdf). However, in case of QAM, where the signal of interest has both real and imaginary parts, one needs to jointly estimate the real and imaginary parts of the data signal from a complex-valued received signal. A MIMO MMSE-type receiver [28] can be used to obtain estimates of both real and imaginary parts of the data as well as the auto-and-crosscorrelation between the filtered I-Q data branches. In such case, a conventional ML demodulator that models the filtered residual noise as circular AWGN leads to a sub-optimum performance especially when the IQ crosscorrelation is non-zero (i.e., in the presence of noncircular noise). Therefore, to get the full benefit of WLF, one should define a ML decision rule that takes into account the full correlations between the I-Q signals (See Fig 5 for an implementation of WL-QAM Receiver). This type of receiver was first considered in [29] for MMSE equalization in the presence of inter-symbol-interference (ISI) and CCI. However, in the special case of single tap channel, the aforementioned WL-MMSE receiver can be implemented equivalently, and alternatively using a WL-ML receiver3 that combines the I-Q filtering and demodulation steps into a single receiver operation. This type of WL-ML decision rule can be derived if we model the total background noise as a non-circular multi-variate Gaussian random process. Using this approach, we propose a generalized WL-ML receiver that is applicable to both PAM and QAM signaling schemes with arbitrary interference type. Note that, although the MMSE is considered to be the de facto approach for WLF, the alternative WL-ML method is considered here for the purpose of exposition only.

Weighing for I-branch

ˆ −1 h ˆT L ˆ T h] ˜ −1 [1 + h √ ˜ −1 h ˜T R−1 ˜T R−1 h] C[1 + C h n ˜n ˜ n ˜n ˜

(6) (7)

˜ −1 ˜T R−1 h] Ch n ˜n ˜

repwhere the multiplicative constant [1 + resents the MMSE bias. Removing the bias term (which maximizes the SINR [32], [33]), we can see that the MMSE approach also leads to the WMF solution.

Assuming single user maximum likelihood (ML) detection, 3 The ML and MMSE techniques yield the same error rate in a single tap channel but the methods differ in both complexity and performance when the channel has ISI. 4 Note that the Gaussian approximation has been widely used for evaluating the SER for conventional optimum combining systems.

B. WL QAM Receiver Since the I-Q QAM signal model defined by (4) has vector modulation, we propose that the real and imaginary parts of the discrete QAM signal be jointly detected taking into

K. KUCHI et al.: PERFORMANCE EVALUATION FOR WIDELY LINEAR DEMODULATION OF PAM/QAM SIGNALS IN THE PRESENCE OF RAYLEIGH FADING AND CO-CHANNEL INTE Matrix-valued filter

B. PEP for WL QAM Receiver

Re{}

yˆ I ( k ) ~ W

y (k )

I-Q ML Symbol Detector

xˆ (k )

yˆ Q (k ) Im{}

Using the QAM signal model (4), the PEP for QAM can be generalized as [30]: ⎡ ⎛ ⎞⎤ T (k)H ˜ T R−1 He(k) ˜ Ce n ˜n ˜ ⎣Q ⎝ ⎠⎦ . PEPQAM = E(H ˜ ˜ T R−1 H) n ˜n ˜ 4 (10)

Fig. 5.

A WL-QAM Receiver

To simplify the analysis, we have:

account the correlation between I-Q branches. The I-Q ML decision metric for this case becomes: (k)R−1 e(k), n ˜n ˜˜

e (8) x ˆ(k) = arg minx¯(k) ˜ √ √ ˜ x(k) = C He(k) ˜ where ˜ e(k)  y ˜(k) − C H¯ +n ˜(k). The vector e(k)  x ˜(k) − x ¯(k) denotes the error between the desired and candidate I-Q data vectors. As in PAM case, the decision metric can be simplified using a preˆ x(k)|2 , where e(k) = |ˆ y(k)− H¯ whitening as: ˜ eT (k)R−1 n ˜n ˜˜ √ filter ˆ = CL ˜ is a pre-whitened channel matrix of size ˜ −1 H H (2N × 2). Fig 5 represents a schematic implementation of the ˜ =L ˜ −1 . WL-QAM receiver with pre-whitening, if we set W Note that the pre-whitening operation offers some complexity reduction, but the WMF operation in this case does not facilitate simplified threshold detection as in PAM case. This happens because threshold demodulation can only be applied for a scalar decision variable whereas a WMF operation for this case produces a vector-valued signal with correlated I-Q elements. In this case, the QAM demodulator must explicitly calculate the decision metrics for all symbol possibilities to determine the minimum distance. Therefore, the WL QAM receiver requires certain extra computational power compared to both conventional and WL-PAM detectors. Also, we would like to remark here that the equivalence among ML, WMF, and MMSE approaches can be generalized for WL QAM detection as well. T

IV. SER E VALUATION In this section, we consider the SER performance of the proposed WL receivers for both PAM and QAM signaling with arbitrary modulation size. We use the standard pairwise error probability (PEP) formulation that is similar to the commonly used union bound. After formulating the PEP for PAM and QAM systems, we proceed to highlight the SER equivalence between the two cases. In Section IV-C, several bounds and approximations are developed to obtain a closed form expression for the SER. A. PEP for WL PAM Receiver The PEP defined as the probability of transmitting a correct symbol x(k) and erroneously receiving x ¯(k) can be formulated as [30]:    2 (k)C h T R−1 h ˜ ˜ (9) d PEPPAM = E(h˜T R−1 h) Q ˜ n ˜n ˜ n ˜n ˜

where d2 (k) =

|x(k)−¯ x(k)|2 4

denotes the (normalized) squared  ∞ a2 Euclidean distance, Q(x)  √12π x e− 2 da, and E(h˜T R−1 h) ˜ n ˜n ˜ ˜ ˜ T R−1 h. denotes the expectation with respect to the pdf of h n ˜n ˜

˜T ET (k)R−1 E(k)h ˜ ˜ ˜ T R−1 He(k) eT (k)H =h n ˜n ˜ n ˜n ˜

(11)

˜ ¯ where E(k)  X(k)  − X(k) denotes the error  event matrix, Re{x(k)} −Im{x(k)} ˜ X(k)  IN ⊗ denotes5 the Im{x(k)} Re{x(k)} modulation matrix, ⊗ denotes the matrix Kronecker product ¯ [31], and X(k) represents the modulation matrix for the candidate symbol x¯(k). Note that E(k) is an orthogonal ˜ ¯ matrix because both X(k), X(k) are orthogonal matrices by definition. Therefore, ET (k)E(k) can be expressed as: ¯(k)|2 I, where ET (k)E(k) = [e2I (k) + e2Q (k)]I = |x(k) − x eI (k)  Re{x(k)} − Re{¯ x(k)} and eQ (k)  Im{x(k)} − Im{¯ x(k)}. Now, ˜T ET (k)R−1 E(k)h ˜ = h n ˜n ˜ ¯ where E(k)  √

¯ T (k)R−1 × ˜T E [e2I (k) + e2Q (k)]h n ˜n ˜ ¯ ˜ E(k)h, (12)

E(k) (e2I (k)+e2Q (k))

becomes a unitary matrix after

¯ = E(k) ¯ h. ˜ Since the individual elements normalization. Let h of the complex-valued channel vector h are assumed to be i.i.d. circular complex Gaussian random variables, the real and imaginary parts of h are also zero-mean, i.i.d. Gaussian. ˜ ∼ N(0, 1 I2N ), where the notation denotes a Therefore, h 2 multi-variate real Gaussian distribution with zero-mean and ¯ has the same distribution as variance 12 I2N . The vector h ¯ ˜ h, since E(k) represents a unitary transformation. Using this result, (12) can be expressed as: ¯ T (k)R−1 E(k) ¯ h ˜ = ˜T E [e2I (k) + e2Q (k)]h n ˜n ˜

[e2I (k) + e2Q (k)] × ¯ ¯T R−1 h. (13) h n ˜n ˜

˜ and h ¯ have Using (9),(10),(13), and using the fact that h same pdf, it can be shown that the PEP expressions for PAM and QAM cases take identical form with the exception of constellation specific scaling factors d2 (k). Hence, the PEP for either PAM or QAM signaling can be represented in universal form as:    −1 ¯ 2 T ¯ PEP = E(h¯T R−1 h) d (k)C h Rn˜ n˜ h , Q ¯ n ˜n ˜

d2 (k) =

|x(k) − x ¯(k)|2 . 4

(14)

The above result indicates that the interference suppression capability of a WL receiver is completely independent of the modulation type used by the desired signal i.e., whether PAM or QAM is being used, but the gain depends mainly on the noise correlation matrix structure. 5 The

symbol IN denotes an identity matrix of size N

6

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

C. Bounds and Approximations to the SER Using the PEP expression (14), the SER for PAM/QAM modulation can be approximated as [30]:    2 T R−1 h ¯ ¯ , (15) P ≈ Kd E(h¯T R−1 h) d C h Q ¯ min n ˜n ˜ n ˜n ˜

where dmin is the minimum Euclidean distance between signal points and Kd denotes the largest number of neighboring points that are at a distance dmin from any constellation point. Note that the above expression is exact for BPSK only; For higher order modulation, it gives asymptotically accurate results in the high signal-to-noise-plus-interference power (SINR) regime only. Next, the matrix R−1 n ˜n ˜ is expressed as: T R−1 n ˜n ˜ = U ΛU

(16)

where U is a unitary matrix and Λ is a diagonal matrix whose elements along the principal diagonal are the eigenvalues of Rn˜ n˜ , denoted by λ = [λ1 , λ2 , .., λ2N ]. Using (16), we 2N ωl2 ¯T R−1 h ¯ = d2 ω T Λω = d2 have: d2min h min min n ˜n ˜ l=1 λl , where ¯ has the same distribution as h, ¯ ω  [ω1 , .., ω2N ] = Uh since U represents a unitary transformation. Now (15) can be expressed as: ⎡ ⎛ ⎞⎤  2N 2   ω l ⎠⎦ P ≈ Kd E(ω,λ) ⎣Q ⎝d2min C . (17) λl l=1

To simplify the evaluation of expectation operation in (17,) we upper-bound (UB) the Q-function as a sum of exponentials using a tight approximation proposed in [34]: Q(x) < 2 −2x2 1 −x 2 + 14 e 3 . Using this result, and using the fact that 12 e the elements of ω are statistically independent real Gaussian random variables, we further upper-bound (17) as: ! 2N  Cω2  Kd −d2min 2λ l l + E(λ) E(ω) e P ≤ 12 l=1 ! 2N  Cω2  Kd −2d2min 3λ l l . (18) E(λ) E(ω) e 4 l=1

−αωl2

Next we use the following result from [30]: E[e ] = √ 1 2 when ωl ∼ N(0, σ 2 ). Using this result, and using (1+2ασ )

the chain rule of conditional expectation, we first evaluate the expectation with respect to ω conditioned on λ, to get: ⎡ Kd + P ≤ E(λ) ⎣ "  d2 C 2N 12 l=1 (1 + min ) 2λl ⎤ Kd ⎦ "2N  2d2 C 4 l=1 (1 + min ) 3λl ⎡ ⎛ ⎞⎤  Ki ⎝ ⎠⎦ , (19) = E(λ) ⎣ "2N  ci C (1 + ) i=1,2 l=1 λl d2

2d2

Kd min where the constants c1 = min 2 , c2 = 3 , K1 = 12 , K2 = Kd 4 are introduced for compactness. Now, let m denote the rank of the (2N × 2N ) matrix R˜i˜i . Then R˜i˜i has exactly

˜ = m positive eigenvalues represented in vector form: λ ˜ ˜ ˜ [λ1 , λ2 , .., λm ] and the remaining 2N − m eigenvalues are identically equal to zero. Therefore the eigenvalues of Rn˜ n˜ ˜l + N0 , for l = 1, .., m and can be expressed as: λl = λ 2 N0 λl = 2 , for l = m + 1, .., 2N . Using this result, we can rearrange various terms in (19) as:  Ki × P ≤ 2ci C N − m 2 (1 + N0 ) i=1,2 ⎞⎤ ⎛ ⎜ ⎜ 1 ⎜  E(λ) ˜ ⎜ ⎝ "m 1+ l=1

ci C ˜ l + N0 λ 2

⎟⎥ ⎟⎥ ⎟⎥  ⎟⎥ . (20) ⎠⎦

In the above expression, the expectation with respect to the ˜ is yet to be determined. eigenvector λ V. F URTHER A PPROXIMATIONS Recall that,for PAM-type interference, the ICM takes the ˜l g ˜lT . Since ˜ gl is an i.i.d. real Gaussian form: R˜i˜i = M l=1 Il g vector, and if we assume that the interferers have equal average power i.e., Il = I0 for l = 1, 2, .., M , R˜i˜i becomes a real Wishart matrix [27] whose joint pdf of ordered eigenvalues ˜ l } is given in the Appendix. Evaluation of expectation {λ operation in (20) requires a multiple integral that requires ˜ l } which is integration with respect to the joint pdf of {λ difficult to handle analytically. Note that this type of situation also occurs in conventional systems with optimum combining. In that case, several useful bounds and approximations have been derived (see e.g., [25], [35], [36]). In the following, we obtain new bounds and approximations to the SER of WL receivers using a generalization of Chiani’s results provided in [25] together with an approximation based on the expected value of the square root of the product of non-zero eigenvalues (pseudo-determinant) of a real Wishart matrix. A. Interference Limited Case Let us consider an interference limited system, where N0 << I0 < C0 . With this assumption, the term ˜ ci CN0 >> 1 λl + 2 for most channel realizations. Therefore, for N0 → 0, we have:   "m "m  ci C 1 + ≈ l=1 cλ˜i C . Using this result, the N0 l=1 ˜ λl +

2

l

SER expression (20) can be approximated as: ⎞⎤ ⎡ ⎛  m   ˜ K λ i l ⎠⎦ ⎣ PA = E(λ) . ˜ ⎝ 2ci C N − m 2 c C (1 + ) i N0 i=1,2 l=1 Further, using Theorem A.4 from Appendix, the expected value of square root of R˜i˜i can ' &"of the  pseudo-determinant m " + 12 ) m m Γ( n−i+1 2 ˜ 2 be expressed as: E λl = (I0 ) l=1 i=1 Γ( n−i+1 ) , 2 where m = min(2N, M ), n = max(2N, M ). Now we have: !  Ki (21) PA = ρ 2ci C 2N2−m 2ci C m ( I0 ) 2 i=1,2 (1 + N0 )

K. KUCHI et al.: PERFORMANCE EVALUATION FOR WIDELY LINEAR DEMODULATION OF PAM/QAM SIGNALS IN THE PRESENCE OF RAYLEIGH FADING AND CO-CHANNEL INTE

TABLE I ˜ 1 ], .., E[λ ˜ m )] FOR A REAL W ISHART N ON - ZERO MEAN EIGENVALUES (E[λ MATRIX , I0 = 1 M =1 1 2 3 4

N 1 2 3 4

m 2

M =2 (1.79, 0.21) (3.18, 0.82) (4.4, 1.6) (5.68, 2.32)

TABLE II N ON - ZERO MEAN EIGENVALUES FOR MIXED INTERFERENCE , I0 = 1 M1 = 0, M2 = 1 (0.5, 0.5) (1.0, 1.0) (1.5, 1.5) (2.0, 2.0)

N 1 2 3 4

M =3 (2.5, 0.5) (4.13, 1.5, 0.33) (5.68, 2.5, 0.84) (7.08, 3.5, 1.44)

"m

+ 12 ) Γ( n−i+1 2 i=1 Γ( n−i+1 ) . 2

where ρ  2 In CCI limited cases, this approximation denoted as approximation A gives asymptotically accurate results in the high signal-to-interference-powerratio (SIR, defined as C/I0 ) regime.

M1 =1, M2 =1 (1.5, 0.5) (2.66, 1.0, 0.34) (3.74, 1.5, 0.76) (4.82, 2.0, 1.18)

m ˜ I0 E[ j=1 λ j ] = E[Trace(R˜i˜i )] = mn 2 . Substituting this result in (22), we get:  1 × Ki PB2 ≤ 2ci C 2N2−m (1 + N0 ) i=1,2 ⎤

B. Generalization of Chiani’s Bounds In this section we generalize certain key results of [25] for WL systems with interference. 1) Approximation via Mean Eigenvalues: Since the eigen˜1 , λ ˜ 2 , .., λ ˜m )  the function f (λ values {λ˜l } are positive, ⎤ ⎡ ⎢ ⎢ ⎣"

m l=1

 1    1+

⎥ ˜ l when other ⎥ is concave in each λ

ci C ˜ + N0 λ l 2

⎦

1

  "m l=1 1 +

ci C ˜ l ]+ N0 E[λ

⎥ ⎥ ⎥ ⎥ ⎦

(22)

2

which is a function of individual mean eigenvalues (MEV). Since closed form evaluation of the MEV is difficult for arbitrary values of (N, M ), we resort to Monte Carlo simulation. See Table I for a list of MEV for a few cases. To average over 10,000 channel instances, the simulation took only a fraction of a second on a 2-GHz PC. It should be noted that this simulation time is negligibly small compared to the time required for brute force link level simulation. It is subsequently shown that this approximation denoted as approximation B1 is within 1.0 dB of the SER obtained via simulation for all cases considered in this paper. 2) Approximation via Equal Mean Eigenvalues: Next, we obtain a closed form expression by further upperbounding the SER given by approximation B1. Using a result from Section V-B of [25], it can ⎤ ⎡ be shown that the ˜ 2 ], .., E[λ ˜ m ])  ⎣ ) " ˜ 1 ], E[λ function f (E[λ m

1

l=1 (1+

ci C N0 ) ] 2

˜ + E[λ l

⎦

attains its maximum  value when the individual MEV are m ˜ j=1 E[λj ] ˜l ] = equal i.e., E[λ , for l = 1, 2, .., m. Using m this result, the second part of (22) can be upper-bounded 1 ≤

as: ) " m 1 ci C  m , where 2 ˜ + E[λ l

 1+

1



ci C N0 I0 2 +n 2

m 2

⎥ ⎥ ⎥. ⎦

(23)

This bound is further referred to as approximation B2. Note that this bound matches approximation B1 for the case of single interferer with arbitrary N . C. Mixed Interference Scenario

variables are fixed. The function is neither globally convex nor concave. Yet, we apply Jensen’s type inequality as suggested in [25] and [35] to approximate the SER as:  1 PB1 ≈ × Ki 2ci C 2N2−m (1 + N0 ) i=1,2 ⎤

l=1 (1+

M1 =2, M2 =1 (2.3, 0.7) (3.78, 1.54, 0.56, 0.1) (5.18, 2.38, 1.0, 0.38) (6.44, 3.2, 1.6, 0.74)

N0 ) ] 2

1+

m

ci Cm N0  m ˜ ] E[λ + j j=1 2

To determine the performance in a mixed interference scenario, we assume that the total CCI consists of M1 PAM , M2 QAM interferers with equal average power. In this case,√ the I-Q noise vector can be represented √  M1 M1 +M2 ˜ Gl x g ˜l xl (k) + I0 l=M ˜l (k) + as: n ˜ (k) = I0 l=1 1 +1 w(k), ˜ where the equivalent channel for the lth QAM ˜ l = [G(l,1) , .., G(l,N ) ]T , G(l,q)  CCI is given by: G   Re{g(l,q) } Im{g(l,q) } , q = 1, 2, .., N, and x ˜l (k) = −Im{g(l,q) } Re{g(l,q) } [Re{xl (k)}, Im{xl (k)}]T . As in (3), the ICM can be calculated as: M  M 1 1 +M2  ˜ lx R˜˜ = I0 E G g ˜l xl (k) + ˜l (k) × ii

l=M1 +1

l=1

M 1 

gl xl (k) + ˜

=

I0

l=1

g ˜l g ˜lT

T ⎤

˜ lx G ˜l (k)

⎦ (24)

l=M1 +1

l=1

M 1 

M 1 +M2

1 + 2

M 1 +M2

! T ˜ ˜ Gl Gl .

(25)

l=M1 +1

In (25), the first term represents the correlation contribution due to PAM-type CCI which has real Wishart distribution whereas the second term which is introduced by the QAM interferers has complex Wishart distribution [25]6. But the total ICM does not belong to the class of Wishart matrices. For this case, we have not been able to determine the pdf of eigenvalues, but, since approximations B1 and B2 alleviate the need for calculating the eigenvalue distribution, the SER can be calculated using (22), (23) with reasonable accuracy. Also, to utilize approximation B1, the MEV are found using Monte  M2 ˜ ˜T of I-Q split, the eigenvalues of the matrix l=1 Gl Gl occur in pairs. Excluding the multiplicity, these eigenvalues are same as that of the eigenvalues of the conventional ICM that is defined for the sum of all QAM interferers without I-Q split which follow complex-Wishart distribution. 6 Because

8

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

TABLE III D IVERSITY-IC TRADEOFF FOR WL RECEIVERS WITH N = 4 M1 2

(M1 , M2 ), M1 + 2M2 < 2N (7,0),(5,1), (3,2), (1,3) (6,0),(4,1), (2,2),(0,3) (5,0),(3,1),(1,2) (4,0),(2,1),(0,2) (3,0),(1,1) (2,0),(0,1) (1,0)

Carlo simulation. See Table II for a list of MEV for a few mixed interference scenarios. Further, since approximation A is specific to real Wishart distribution, it cannot be applied for the present case directly. Nevertheless, a coarse approximation to the SER is obtained by substituting each QAM interferer with two PAM interferers with statistically independent channels while the total average power is redistributed equally among all interferers. With this assumption, the system has a total of M1 + 2M2 PAM interferers. Then approximation A given by (21) can be used to obtain an estimate of the SER with the following parameters: m = min(2N, M1 + 2M2 ), and n = max(2N, M1 + 2M2 ). Note that the QAM-PAM substitution does not change the rank of the ICM, but the substitution alters the eigenvalue distribution which causes certain inaccuracy when approximation A (which depends on the expected value of square root of the pseudo-determinant ICM) is applied. Nevertheless, this approximation is useful in high SIR cases where the rank of the ICM mainly dictates the SER performance. Through simulation, it is subsequently shown that this method leads to an UB that is within 1.0 dB of simulation in the high SIR regime. VI. D IVERSITY-IC T RADEOFF For high signal-to-noise-power-ratios (SNR, defined as NC0 ), the SER given in Section V-A can be further upper-bounded as: !  Ki . (26) PA < ρ m m ( 2cNi0C )N − 2 ( 2cIi0C ) 2 i=1,2 Let us consider the general case with N antennas, M1 PAM and M2 QAM interferers. When R˜i˜i is rank deficient i.e., for M1 + 2M2 < 2N , we have: m = min(2N, M1 + 2M2 ) = M2 + 2M2 . The asymptotic diversity order for this system is: dW L = N − M2 − M21 , which is given by exponent of NC0 . Therefore, at high SNR, the SER is dictated by the thermal noise level rather than interference power level. In fact, from the SER expression (26) we can see that receiver provides a zero error rate as N0 → 0 irrespective of interference power level. This result implies that the WL receiver fully eliminates M1 PAM and M2 QAM interferers, when M1 + 2M2 < 2N with a diversity benefit dW L = N − M2 − M21 . In the opposite case when R˜i˜i has full rank (m = 2N ), i.e. when M1 + 2M2 ≥ 2N , the SER is limited by the interference power rather than thermal noise. In this type of system, one can define the diversity order with respect to the SIR metric. In this case, the WL receiver provides full diversity N (which is given by the exponent of IC0 in (26)) with a limited IC gain.

WL Receiver (Simulation) Conv Receiver (Simulation) Approximation A Approximation B2 Approximation B1

−1

10

SER

dW L = N − M2 − 0.5 1 1.5 2 2.5 3 3.5

0

10

−2

10

−3

10

−4

10

Fig. 6.

0

5

10

15 SIR

20

25

30

Single Antenna: QPSK desired, single BPSK CCI

Here, the multiplicative factor ρ represents the SER reduction compared to a conventional system. To highlight the performance differences between conventional MMSE and WL detection, let us assume that M = M1 + M2 be the total number of interferers in the system. In [2], it has been shown that a conventional receiver can fully reject all M interferers with an asymptotic diversity order dConv = N − M , for M < N . This result suggests that a conventional receiver cannot provide any IC gain in the absence of multiple antennas, whereas according to our SER analysis, a single antenna WL receiver fully suppresses a single PAM interferer. For N = 2, while the conventional receiver can reject a single interferer, the WL receiver is capable of suppressing a maximum of three PAM interferers or, a single PAM plus a single QAM CCI. Further, to illustrate the tradeoff between diversity advantage and IC gain, in the second column of Table III, we show all possible combinations (M1 , M2 ) that gives a given diversity order dW L satisfying the constraint: M1 + 2M2 < 2N . When the total number of interferers to be cancelled is fixed, the WL receiver provides a higher diversity gain than the conventional method. As an example, consider N = 4 with M1 = 3 PAM interferers (and M2 = 0). Then we have: dConv = 1 and dW L = 2.5. Clearly, we see that WL receivers could offer good diversityIC tradeoff with a fewer number of antennas compared to the conventional case. VII. N UMERICAL AND S IMULATION R ESULTS In this section, we compare the SER obtained via Monte Carlo simulation with various bounds approximations derived in Section V. To assess the gain of WL receivers, the simulated SER performance with conventional MMSE detection is shown for reference. Results are provided for BPSK and QPSK systems with single/dual antenna reception with multiple interferers. The results are averaged over 100,000 fading channel realizations and 150 data symbols are transmitted for each realization. To investigate the impact of both interference and thermal noise, the SER results are reported in two formats. In one case, the SER is plotted as function of SIR for a

K. KUCHI et al.: PERFORMANCE EVALUATION FOR WIDELY LINEAR DEMODULATION OF PAM/QAM SIGNALS IN THE PRESENCE OF RAYLEIGH FADING AND CO-CHANNEL INTE 0

−1

10

10

WL Receiver (Simulation) Conv Receiver (Simulation) Approximation A Approximation B2 Approximation B1 −2

−1

10

SER

SER

10

−3

−2

10

10

WL Receiver (Simulation) Conv Receiver (Simulation) Approximation A Approximation B2 Approximation B1 −4

−3

10

Fig. 7.

10 0

5

10

15

20 SNR

25

30

35

40

Fig. 9.

Single Antenna: BPSK desired, single BPSK CCI (SIR=0 dB)

−4

−2

0

2 SNR

4

6

8

Two Antennas: BPSK desired, 2-Interferers (PAM+QAM)

0

10

WL Receiver (Simulation) Conv Receiver (Simulation) Approximation A Approximation B2 Approximation B1

−1

SER

10

−2

10

−3

10

−4

10

Fig. 8.

0

4

8

12 SIR

16

20

24

Single Antenna: BPSK desired, two BPSK interferers

fixed value of NI00 at 20 dB. This scenario corresponds to an interference limited system. In the second case, the error rate is plotted as a function of SNR for a given SIR. In Fig 6, we show the SER as function of SIR when a QPSK signal is corrupted by a single BPSK interferer. The WL receiver shows significant SER reduction over conventional MMSE. Since the ICM is rank deficient, and since we are considering an interference limited situation, the gain of WLF for this case is mainly dictated by the parameter NI00 which is set at 20 dB. Note that this gain increases (or decreases) with higher (or lower) values of NI00 . In this case, both approximation A and approximation B2 provide accurate SER results at high SIR but approximation A becomes somewhat loose UB at low SIR. Since the IC capability is independent of modulation type of the desired signal, in the rest of the cases we consider SER for BPSK detection only. Also, note that for N = 1, approximation B1 coincides with approximation B2. Fig 7 shows SER (plotted as a function of SNR) for single antenna BPSK detection in the presence of a single BPSK CCI. Here, the SER for the conventional receiver reaches an error floor (due to lack of degrees of freedom for full

IC) whereas the WL receiver shows full IC gain with an asymptotic diversity order equal to 0.5. Approximation B2 (as well as B1) closely follows the simulation for the whole range of SNR values but approximation A becomes a loose UB as this scenario violates the assumption N0 << I0 << C. Next, in Fig 8, we show the SER (as a function of SIR) for a single antenna receiver with two BPSK interferers. Since the ICM has full rank, the WL receiver showed a limited IC gain of approximately 2.5 dB. Here, approximation A closely matched simulation at high SIR whereas approximation B2 overestimated the SER by about 3.0 dB; Approximation B1, however, outperformed approximation B2 by about 2.0 dB and it is within 1.5 dB of simulated SER for a wide range of SIR values. For N = 2, the WL receiver can reject a maximum of 3PAM interferers or two mixed interferers (one PAM plus one QAM). In Fig 9, we show the SER for the mixed interferer case. In this case, approximation B2 is within 2.0 dB of simulation whereas approximation B1 has better accuracy which is within 1.0 dB of simulation. Also, note that approximation A which is based on QAM-PAM substitution closely matched approximation B1 at high SIR; However, at low SIR, since the assumption N0 << I0 << C0 is not met, it becomes a loose UB. VIII. C ONCLUSION In this paper, we introduce WL receiver structures for both PAM and QAM systems with multiple receiver antennas and multiple co-channel interferers. In the proposed implementation, the real and imaginary parts of the complex-valued baseband received signal are jointly filtered using a WLML receiver that applies a pre-whitening filter to whiten the interference across I-Q dimensions before applying ML demodulation. In the presence of PAM-type interference, we show that this type of receiver provides a significant IC gain compared to conventional MMSE systems with a moderate increase in receiver complexity. We clarified the computational and implementation differences between PAM and QAM systems with WLF. It is shown that the WL QAM receiver

10

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 57, NO. 1, JANUARY 2009

requires a more complex ML demodulator since it takes into account the correlations between the I-Q data branches during ML demodulation. In contrast, both conventional and WLPAM demodulators can be implemented more efficiently with standard threshold demodulators. When fading is symmetrically distributed between the I-Q parts (as in a Rayleigh fading model), we show that the IC gain offered by WL processing is independent of the modulation type used by the desired signal but the gain depends mainly on the modulation characteristics of the interferers. In a system with N antennas and I-Q split, the receiver has 2N observations. The WL receiver is said to have full IC capability (that is complete interference removal) only when the ICM is rank deficient i.e. when m < 2N . The asymptotic diversity order for this system is shown to be N − m 2 . This condition implies that when the system employs a mixture of PAM and QAM signals, a WL receiver can reject any combination of M1 PAM, M2 QAM interferers satisfying the constraint: M1 + 2M2 < 2N , with a diversity order N − M2 − M21 . The analysis shows that WL receivers outperform conventional MMSE methods both in diversity and IC aspects when the system contains PAM-type CCI. We obtained several bounds and approximations to the SER for both PAM and QAM WL receivers with arbitrary number of antennas and arbitrary number of interferers of equal average power. The pseudo-determinant based approximation A is asymptotically accurate in the presence of PAM-type CCI when N0 << I0 << C. For the considered scenarios, the trace based approximation B2 is typically within 2.0 dB of simulation when the ICM is rank deficient but it becomes a loose UB when the ICM has full rank. However, the approximation B1 which is based on the mean eigenvalues served as a good approximation for all cases including the mixed interference scenario. This approximation is within 1.0 dB of simulation in all considered scenarios.

Proof: See [27]. Theorem A.2: The expected value of products of the square of a Wishart matrix is: &" roots  'of the eigenvalues n−i+1 + 12 ) m 2 ˜ i = (2β) m2 "m Γ( n−i+1 λ . E i=1 i=1 Γ( ) 2 Proof: A generalization of Theorem 3.2.15 of [27] leads to the stated result.

A PPENDIX R EAL W ISHART D ISTRIBUTION In the following we use several results from [27] to describe the statistical properties of real Wishart matrices. Definition A random matrix of form A = ZT Z = n T k=1 Zk Zk is called a real Wishart matrix, denoted as: Wm (n, βI)), if the elements of Z is of form: Z = [Z1 , Z2 , .., Zn ]T where each individual vector Zi = [zi,1 , .., zi,m ]T is a real-valued i.i.d. Gaussian random vector: N(0, βI), and n ≥ m. Theorem A.1: The probability distribution of the ordered ˜ m , (λ ˜1 > λ ˜ 2 > .. > λ ˜ m > 0) of A = ˜1 , .., λ eigenvalues λ Wm (n, βI), (n ≥ m) is 2

m m (n−m−1) −1  m ˜  π 2 ˜ 2 i=1 λi 2β × e λ mn i m n (2β) 2 Γm ( 2 )Γm ( 2 ) i=1

m  ˜i − λ ˜j ) (λ

(27)

i
where Γm (.) denotes the multivariate gamma function defined m(m−1) "m 1 4 as: Γ(x) = m (a) = (π) i=1 Γ[a − 2 (i − 1)],  ∞ Γx−1 −t t e dt x > 0 is the gamma function. 0

¯ = ZZT for n > m is called Definition The singular matrix A an anti-Wishart matrix [37]. Theorem A.3: The first m ordered eigenvalues of anti˜ 1 , .., λ ˜ m ) and the remaining ¯ are equal to (λ Wishart matrix A n − m eigenvalues are identically equal to zero. These nonzero eigenvalues are same as the ordered eigenvalues of the Wishart matrix A = ZT Z: Wn (m, βI). Proof: See [25, Theorem 4]. Theorem A.4: The expected value of the product of square roots &of non-zero eigenvalues (pseudo-determinant) m " "m  ˜ ' + 12 ) m Γ( n−i+1 2 m = λl = (I0 ) 2 i=1 Γ( n−i+1 of R˜i˜i is: E l=1 ) 2 min(2N, M ), n = max(2N, M ). Proof: From Section II-A, for equal power case, the M interference T ICM can be obtained as: R˜i˜i = I g ˜ g ˜ . Recall that l=1 0 l l the I-Q channel vector of lth interferer ˜ gl is modeled as a column vector of length 2N whose elements are assumed to be i.i.d. zero-mean, real Gaussian random variables with variance 12 . When the number of interferers M is greater than or equal to 2N , R˜i˜i becomes a real-valued Wishart matrix denoted as: W2N (M, I20 I) with full rank. However, when the number of interferers M is less then 2N , the ICM becomes rank deficient. In this case R˜i˜i has anti-Wishart distribution. Combining the Wishart and anti-Wishart cases, and applying Theorems A.2, Theorem A.4, we can express the expected value of the product of square root of non-zero eigenvalues ˜l of R˜˜ as λ ii m  m n−i+1  m  Γ( + 12 ) 2 ˜ l = (I0 ) 2 , m = min(2N, M ), λ E Γ( n−i+1 ) 2 i=1 l=1 n = max(2N, M )

(28)

where m denotes the rank (that is the number of non-zero eigenvalues of) of R˜i˜i . R EFERENCES [1] J. Winters, “Optimum combining in digital mobile radio with cochannel interference," IEEE J. Select. Areas Commun., pp. 528-39, 1984. [2] J. Winters, R. Gitlin, and J. Salz, “The impact of antenna diversity on the capacity of wireless communication systems," IEEE Trans. Commun., vol. 42, pp. 1740-51, 1994. [3] J. H. Winters and J. Salz, “Upper bounds on bit error rate of optimum combining in wireless systems," IEEE Trans. Commun., pp. 1619-1624, Dec. 1998. [4] S. Ariyavisitakul, J. Winters, and I. Lee, “Optimum space-time processors with dispersive interference: Unified analysis and required filter span," IEEE Trans. Commun., vol. 47, pp. 1073-83, 1999. [5] Y. Li, J. Winters, and N. Sollenberger, “Spatio temporal equalization for IS-136 TDMA systems with rapid dispersive fading and cochannel interference," IEEE Trans. Veh. Technol., vol. 48, pp. 1182-94, 1999. [6] S. J. Grant and J. K. Cavers, “Performance enhancement through joint detection of cochannel signals using diversity arrays," IEEE Trans. Commun., vol. 46, pp. 1038-49, Aug. 1998. [7] B. Picinbono and P. Chevalier, “Widely linear estimation with complex data," IEEE Trans. Signal Processing, vol. 43, pp. 2030-2033, Aug. 1995.

K. KUCHI et al.: PERFORMANCE EVALUATION FOR WIDELY LINEAR DEMODULATION OF PAM/QAM SIGNALS IN THE PRESENCE OF RAYLEIGH FADING AND CO-CHANNEL INTE

[8] W. Brown and R. Crawe, “Conjugate linear filtering," IEEE Trans. Inform. Theory, vol. 15, pp. 462-465, 1969. [9] W. Gardner, Cyclostationarity in Communications and Signal Processing. IEEE Press, 1994. [10] F. Neeser and J. Massey, “Proper complex random processes with application to information theory," IEEE Trans. Inform. Theory, vol. 39, pp. 1293-1302, 1993. [11] Y. Yoon and H. Leib, “Maximizing SNR in improper complex noise and application to CDMA," IEEE Commun. Lett., vol. 1, pp. 5-8, Jan. 1997. [12] G. Gelli, L. Paura, and A. Ragozini, “Widely linear multiuser detection," IEEE Commun. Lett., vol. 4, pp. 187-9, Jun. 2000. [13] S. Buzzi, M. Loops, and A. Tulino, “A new class of multiuser CDMA receivers based on minimum mean-output-energy strategy," in ISIT, 2000. [14] A. Lampe and M. Breiling, “Asymptotic analysis of widely linear MMSE multiuser detection-complex vs real modulation," in Inform. Theory Workshop, 2001, pp. 55-57. [15] A. Lampe, R. Schober, W. H. Gerstacker, and J. Huber, “A novel iterative multiuser detector for complex modulation schemes," IEEE J. Select. Areas Commun., vol. 20, pp. 339-50, Feb. 2002. [16] W. H. Gerstacker and R. Schober, “Equalization concepts for EDGE," IEEE Trans. Wireless Commun., vol. 1, pp. 190-99, Jan. 2002. [17] W. H. Gerstacker, R. Schober, and A. Lampe, “Receivers with widely linear processing for frequency-selective channels," IEEE Trans. Commun., vol. 51, pp. 1512-22, 2003. [18] W. H. Gerstacker, F. Obernosterer, R. Schober, A. Lehmann, A. Lampe, and P. Gunerben, “Equalization concepts for Alamouti space-time block code," IEEE Trans. Commun., vol. 52, pp. 1178-90, 2004. [19] D. Darsena, G. Gelli, L. Paura, and F. Verde, “Widely linear equalization and blind channel identification for interference-contaminated multicarrier systems," IEEE Trans. Signal Processing, vol. 53, pp. 1163-77, 2005. [20] H. Trigui and D. Slock, “Cochannel interference cancellation within the current GSM standard," in Proc. Universal Personal Commun., Oct. 1998, pp. 511-15. [21] W. Gardner and C. Reid, “Making the most out of spectral redundancy in GSM: Cheap CCI suppression," in Proc. Asilomar Signals, Systems Computers, Oct. 2001, pp. 883-889. [22] P. Chevalier and F. Pipon, “New insights into optimal widely linear array receivers for demodulation of BPSK,MSK, and GMSK corrupted by noncircular interferers-Application to SAIC," IEEE Trans. Signal Processing, vol. 54, pp. 870-883, Mar. 2006. [23] A. Shah and A. H. Haimovich, “Performance analysis of optimum combining in wireless communication with rayleigh fading and cochannel interference," IEEE Trans. Commun., vol. 46, pp. 473-479, 1998. [24] A. Shah, A. H. Haimovich, M. K. Simon, and M. S. Alouini, “Exact bit error probability for optimum combining with a rayleigh fading Gaussain cochannel interferer," IEEE Trans. Commun., vol. 48, pp. 908-912, 2000. [25] M. Chiani, M. Win, A. Zanella, R. Mallik, and J. Winters, “Bounds and approximations for optimum combining of signals in the presence of multiple cochannel interferers," IEEE Trans. Commun., vol. 51, pp. 296-307, Feb. 2003. [26] M. Chiani, M. Z. Win, A. Zanella, and J. H. Winters, “A laguerre polynomial based bound on the symbol error probability for adaptive antennas with optimum combining," IEEE Trans. Wireless Commun., vol. 3, pp. 12-16, Jan. 2004. [27] R. Muirhead, Aspects of Multivariate Statistical Theory. Wiley Series in Probability and Mathematical Statistics, 1982. [28] N. Al-Dhahir and A. Sayed, “The finite length multi-input multi-output MMSE-DFE," IEEE Trans. Signal Processing, vol. 48, pp. 2921-36, Oct. 2000. [29] K. Kuchi, G. Mattellini, and V. Prabhu, “Generalization of widely linear filtering concepts for equalization and interference suppression for

PAM/QAM systems," in Proc. Asilomar Signals, Systems Computers, Oct. 2005, pp. 1287-1291. [30] J. Proakis, Digital Communications. Mc Graw Hill, 2000. [31] R. Horn and C. Johnson, Matrix Analysis. Cambridge University Press, 1990. [32] J. Cioffi, EE:379 Stanford Class Notes. [Online]. Available: http://www.stanford.edu/class/ee379a/ [33] N. Al-Dhahir, “MMSE decision-feedback equalizers: Finite length results," IEEE Trans. Commun., vol. 41, pp. 961-75, 1995. [34] M. Chiani, D. Dardari, and M. K. Simon, “New exponential bounds and approximations for the computation of error probability in fading channels," IEEE Trans. Wireless Commun., vol. 2, pp. 840-45, 2003. [35] T. D. Pham and K. G. Balmain, “Multipath performance of adaptive antennas with multiple interferers and correlated rayleigh fadings," IEEE Trans. Veh. Technol., vol. 48, pp. 342-352, 1999. [36] E. Villier, “Performance analysis of optimum combining with multiple interferers in flat rayleigh fading," IEEE Trans. Commun., vol. 46, pp. 1503-1510, 1999. [37] R. K. Mallik, “Pseudo-wishart distribution and its application to MIMO systems," IEEE Trans. Inform. Theory, vol. 49, pp. 2761-2769, Oct. 2003.

Kiran Kuchi was born in Nellore, India, in 1974. He received B.Tech degree in electronics and communications engineering from the Sri Venkateswara University, Tirupati, India, in 1995, M.S. and Ph.D. degrees, both in electrical engineering from the University of Texas, Arlington, USA, in 1997 and 2006, respectively. From 1997-2000, he was with Motorola Labs, Fort Worth, Texas where he worked on transmit diversity/space-time coding concepts for CDMA-2000 systems. From 2000-2008 he was with Nokia, Irving, Texas. At Nokia, he made significant contributions to baseband transceiver design for GSM/EDGE base station and mobile phone products. Presently he is with Centre of Excellence in Wireless Technology (CEWiT), Chennai, India, where he is involved in research and development activities related to broadband wireless standards.

Vasant K. Prabhu was born in Kumta, India, on May 13, 1939. He received the B.Sc. (Hons.) degree from the Karnatak University, Dharwar, India, and the B.E. (Dist.) degree in electrical communication engineering from the Indian Institute of Science, Bangalore, India. He received the S.M. and Sc.D. from the Massachusetts Institute of Technology (MIT), Cambridge, in 1963 and 1966, respectively. While at MIT, he was a Member of Research Laboratory of Electronics, first as a Research Assistant and then as an instructor in Electrical Engineering. From 1966 to 1991, he was a Member of Technical Staff at AT&T Bell Laboratories, where he made numerous contributions. Since 1991, he has been a Professor of Electrical Engineering at the University of Texas, Arlington. Dr. Prabhu is a member of Tau Beta Pi, Eta Kappa Nu, and Sigma Xi. He received the IEEE Centennial Medal in 1985. In 1998, he was awarded the Robert Q. Lee Award for Excellence in Engineering Teaching from the University of Texas, Arlington.