DESIGN OF DISTRIBUTED CHANNEL OPTIMIZED MULTIPLE DESCRIPTION VECTOR QUANTIZER Mehrdad Valipour, Farshad Lahouti Wireless Multimedia Communications Laboratory School of Electrical and Computer Engineering, University of Tehran ABSTRACT In this paper, a distributed channel optimized multiple description vector quantization (DCMDVQ) scheme is presented. A minimum mean square error decoder, utilizing side information, is proposed for effective reconstruction of multiple descriptions transmitted over noisy channels with packet loss. The DCMDVQ is designed using a deterministic annealing approach. Simulation results indicate a noticeable gain in comparison to classic MDVQ or channel optimized MDVQ, while being only 2 dB away from the rate-distortion bound for multiple descriptions with side information. Index Terms— Channel optimized multiple descriptions, Distributed source coding, Noisy channels with packet loss.
1. INTRODUCTION The robustness of multiple descriptions (MD) against packet loss (PL) makes it a viable solution for multimedia communications over networks. MD is generating several descriptions of a source sample, and transmitting them over multiple channels. Recently aiming at transferring the codec complexity from the encoder to decoder, distributed video coding (DVC) is introduced for wireless up-link applications, e.g. [1]. Therefore, distributed multiple descriptions or MD with side information (SI) is a vital element of such prospective systems. According to the Slepian-Wolf (SW) theorem, separate lossless compression of two discrete correlated sources, could be as optimal as their joint encoding [2]. For jointly Gaussian sources, the Wyner-Ziv theorem states that, lossy compression of a source when another correlated source is available only at the decoder could be as optimal as it were also available at the encoder [3]. A WZ source encoder (SE) may be constructed with a properly designed quantizer followed by an SW encoder [4]. In [5] an achievable rate region for MD when SI is available at the decoder is proposed; and in [6] it is shown that the obtained region in the case of jointly Gaussian sources is the set of all achievable rate distortion points. Vaishampayan in [7] designed a multiple description scalar quantizer composed of a scalar quantizer followed by an index assignment (IA) table for robust communications over channels with packet loss. The design of a MD vector quantizer using deterministic annealing (DA) [8] is presented in [9], and based ----------------------------------------------------------------------------------This work has been supported in part by the Iran Telecommunication Research Center.
on simulated annealing in [10]. Recently, the design of multiple descriptions over noisy channels with and without PL has received noticeable attention, e.g. [11][12]. In [5], a predictive multiple description scheme is introduced for a source with memory using a WZ encoder as a predictor, and a classic multiple descriptions. In [13], in addition to two descriptions of a source sample, extra information, generated by a WZ encoder, is transmitted for enhanced reconstruction in presence of packet loss. In both schemes, the MD encoder is set up based on a classic IA, where the availability of SI at the decoder is ignored during the design. To the best of our knowledge, this paper is the first on designing multiple description vector quantizer (MDVQ) with side information available at the decoder. An optimal minimum mean square error (MMSE) MDVQ decoder, utilizing the side information, is presented for efficient reconstruction over noisy channels with PL. Based on this decoder, a distributed channel optimized M-description vector quantizer (DCMDVQ) is designed based on a deterministic annealing approach. Simulation results indicate a noticeable gain in comparison to classic MDVQ or channel optimized MDVQ, while being only 2 dB away from the rate-distortion bound for multiple descriptions with side information. This paper is organized as follows. In section 2, the system model is introduced. The design of DCMDVQ is presented in section 3. Rate-distortion of MD with side information is reviewed in section 4. Section 5 includes the numerical results, and finally the paper is concluded in section 6. 2. SYSTEM MODEL The block diagram of a DCMDVQ is demonstrated in figure 1. First, a source sample (symbol) is quantized, and then using an IA table, one or more quantizer cells are mapped into one IA table cell. The descriptions are transmitted over independent noisy channels with packet loss. The bit error rate and packet loss probability of the channel are denoted by and , respectively. Then using soft information of the received descriptions and the side information available at the decoder, an MMSE source decoder reconstructs the source samples. 3. DESIGN OF DCMDVQ In this section, the source encoder and the source decoder of the proposed distributed channel optimized M-description vector quantizer is introduced. Next, the average distortion is described for communications over noisy channels with PL and
|
| , ,
, |
1 | ,
4
| ,
|
|
where | ,
| ,
|
|
5
| ,
|
6 |
Θ
7
|
where | Figure 1. System Structure of a DCMDVQ
| ,
a procedure for designing DCMDVQ using a deterministic annealing algorithm is proposed. 3.1. Source encoder and source decoder An MDVQ is composed of two parts: a vector quantizer and an index assignment table. First an -dimensional source sample is mapped into one of vector quantizer cells using the probability distribution function | for each Θ 1, , , where, Otherwise
,
| ,
|
,
2
, ,
1⁄
Θ
|
|
9
3.2. Evaluating Distortion When the mean square error is the distortion criterion, the overall distortion is separated into two parts: channel distortion due to noise and packet loss, and source encoder distortion . We have (the proof is omitted for brevity) 10
,
(1)
indicates the quantizer cell (Voronoi region). Let denotes the probability corresponding to the quantizer cell. The mapping from quantizer cells to descriptions' indices is determined by the IA table using the probability distribution function | , Θ , , Θ , Θ , 1 for each where ∑ | The descriptions are transmitted over independent and memoryless noisy channels with packet loss [14]. The received sequence at the input of the MMSE source decoder is denoted , , Θ , where is the output of Θ by channel. The sequence , , 0,1 , the indicates the status of descriptions at the decoder, i.e., 0 description. The probability specifies the loss of the , which is written as for simplicity, indicates the channels status probability. The channel transition probability | , is equal to:
|
| |
8
where
1 0
|
|
|
1
,
1 0
3
where , and . is hamming distance between Let denote the side information available at the decoder. The estimated symbol , | by an MMSE source encoder is given by (the proof is omitted for brevity),
, |
|
| , |
,
11
| 12
, |
The SE design objective is to find | , Θ Θ such that is minimized, when ∑ Θ , 1 for Θ .
|
3.3. Source encoder Design Deterministic annealing [8] is an optimization technique which is initialized with a probabilistic solution, and then the level of randomness is gradually decreased until a nonrandom solution is obtained. Specifically, for DCMDVQ design, the quantizer cells are assigned to IA table cells based on the | . In each step an IA is designed for distribution minimizing the distortion subject to a specific randomness level | of the IA table. This level of randomness is decreased until a nonrandom IA table is obtained, where, | Θ
,
log
|
13
The IA table, obtained in one step, is used for the initialization of the next step. Therefore, the goal of the intermediate optimization step is as follows,
|
arg min ∑ Subject to |
|
1,
14
Θ
Using the Lagrangian multiplier method, the equation (14) is written as follows, |
|
arg min
15 |
1
Θ
exp ∑
′
,
exp
⁄ ′,
16
⁄
′
where ,
17
| ,
,
|
|
, |
The DCMDVQ design procedure, using deterministic annealing algorithm, is summarized as follows: 1.
2. 3. 4. 5.
| , and initialize IA table Select a vector quantizer | . Start with a sufficient high . Compute the distortion using relations (10)-(12). Update the IA table | using (16)-(17). Compute the distortion using (10)-(12). If the obtained average distortion improves less than a specific threshold, or the number of iterations exceeds a specific level, reduce . If is greater than a certain small threshold, go to step 2; otherwise the algorithm is terminated. 4. RATE DISTORTION OF MD WITH SI ,
Suppose
~
0,
is a jointly Gaussian
random variable pair. When the channels are noise less, and the source decoder uses the side information , the set of all achievable distortion region [6] for a given rate pair , satisfying [5] , , (bits) is given by the triple .2 .2
18 19
.2
1
√Π
√∆
20
where Π ∆
/
1 ⁄
1
/
′ , ′ and ′ are 1,0 , 0,1 and 1,1 , respectively. As the channels are assumed independent, we have: ′ 1 25 , ′ , ′ 1 where is a normalization factor. Note that increasing or reduces , therefore, the minimum average , , , distortion is not equal to the following:
where
where and , Θ are Lagrangian multipliers. By | , and setting equal to zero; derivation with respect to | is obtained as follows (The proof is omitted for brevity). |
We denote the RHS of the equations (18)-(20) as , and , respectively. For a given rate , , , pair , and assuming that at least one of the descriptions , is received at the decoder (in accordance with the assumptions in rate distortion (R-D) bound), the minimum average distortion is obtained as follows: (24) ′ ′ ′ min , , ,
21 22 23
′
′
,
,
,
(26)
5. PERFORMANCE EVALUATION In this section, the performance of the proposed distributed channel optimized M-description vector quantizer over channels subject to noise and packet loss is investigated and compared with the rate distortion bound. Also, the effect of the source encoder and decoder design on the average distortion in presence of side information is studied. The source and side information samples are i.i.d. Gaussian . A two distributed with unit variance and a covariance of description system is used during the simulation. A quantizer with 1 and 256 levels is used. The size of descriptions , 1,2 is 8. During the design quantizer cells are mapped to IA table cells. This mapping is optimized for minimizing the average distortion over noisy channels with PL. In general, more than one quantizer cells may be mapped to one IA table and computed by simulation are cell. The values of , denoted by , and , respectively; and is ′ ′ equal to ′ . and the design of DCMDVQ on the The effect of performance are investigated in table 1. Channels are noise less with a packet loss probability equal to 0.1. The average distortion when side information is not considered in source encoder design, but is exploited at the MMSE source decoder, . When the side information is also is referred to as , not utilized at the decoder, the average distortion is denoted by . For comparison with R-D bound, it is assumed that , each description is compressed and decompressed independently using an ideal SW encoder and decoder, respectively. Of course, each SW decoder exploits the side is , information. Therefore, the transmission rate pair equal to | , | . As evident, when is smaller than 0.95, the performance of the system is only about 2 dB and away from the rate distortion bound. Comparison of demonstrates that a considerable gain is achieved , when the DCMDVQ is used at the encoder. Note that the MD is still a channel optimized one. source coder used for , The MDSQ of [7] results in an average distortion of -13.66 dB
and DCMDVQ design (
Table 1. The effect of
-
,
0 0.2 0.4 0.6 0.8 0.9 0.95 0.999
2.79 2.75 2.65 2.50 2.26 2.20 2.27 1.90
2.79 2.75 2.65 2.51 2.27 2.24 2.09 1.96
-18.132 -18.170 -18.243 -18.293 -19.563 -21.830 -24.722 -37.570
-20.318 -20.263 -20.204 -20.509 -21.551 -24.016 -26.710 -42.288
2.186 2.093 1.961 2.216 1.988 2.186 1.988 4.718
-18.132 -18.165 -18.226 -18.256 -18.568 -19.033 -19.592 -28.230
Table 2. Performance of DCMDVQ over noisy channels 3 0.1 0.8, distortion in [dB]) ( 0.1 0.01 0.001 0.0001 -8.097 -12.460 -15.021 -14.987 -8.099 -12.761 -14.744 -15.413 -10.946 -18.862 -21.231 -21.824 -10.216 -16.720 -19.039 -19.486
= 0.3 0.2 0.1 0.05 0.01 0.001
2.266 2.279 2.297 2.321 2.441 2.940
2.266 2.274 2.271 2.362 2.602 2.876
-17.557 -18.371 -19.600 -20.866 -23.449 -30.061
-19.296 -20.184 -21.660 -23.400 -27.859 -39.398
1.738 1.813 2.060 2.533 4.410 9.338
and -16.54 dB for 0 and 0.9, respectively. The performance of DCMDVQ is presented in table 2, where the effect of noisy channels is investigated. The packet loss is equal to 0.8. The probability is equal to 0.1, and transmission rate for each description is equal to 3. Note that . the design objective is to minimize The effect of packet loss probability on the performance of DCMDVQ is studied in table 3. When the packet loss probability is more than 0.05, the gap of the average distortion from the rate distortion bound is about 2 dB. As evident, the gap from the bound is increased when the PL probability is decreased. This is due to the fact that in this case, the descriptions may be transmitted at lower rates. For example, when there is not any packet loss, the descriptions could be transmitted at a rate of , | instead of | | without any degradation in the distortion. 6. CONCLUSIONS A channel optimized M-description vector quantizer is designed in the presence of side information at the decoder using a deterministic annealing algorithm. An optimal MMSE source decoder for the proposed system is suggested. The
0, distortion in [dB]) ,
0 -0.005 -0.017 -0.037 -0.995 -2.797 -5.130 -9.340
,
-18.132 -18.132 -18.132 -18.132 -18.132 -18.132 -18.132 -18.132
-
,
0 -0.038 -0.111 -0.161 -1.431 -3.698 -6.590 -19.438
effects of the covariance of the source and side information, the bit error rates, and packet loss probabilities of channels on the average distortion are investigated. Also, the simulation result is compared with the rate-distortion bound. 7. REFERENCES [1] [2]
Table 3. Effect of packet loss on the performance of DCMDVQ ( 0 0.8, distortion in [dB])
0.1
[3] [4]
[5] [6] [7] [8] [9] [10] [11]
[12]
[13]
[14]
B. Girod, A. Aaron, S. Rane, and D. Rebollo-Monedero, “Distributed video coding,” Proc. IEEE, vol. 93, no. 1, pp. 71–83, Jan. 2005. D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 471–480, July 1973. A. D. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Trans. Inform. Theory, vol. IT-22, pp. 1–10, Jan. 1976. D. Rebollo-Monedero, R. Zhang, and B. Girod, “Design of optimal quantizers for distributed source coding,” in Proc. IEEE Data Compression Conf. (DCC), Snowbird, UT, Mar. 2003, pp. 13–22. A. Sehgal and N. Ahuja, “Robust predictive coding and the Wyner-Ziv problem,” in Proc. IEEE Data Compression Conf., 2003, pp. 103–112. S. Diggavi and V. Vaishampayan, “On multiple description source coding with decoder side information,” in Proc. IEEE Information Theory Workshop,San Antonio, TX, Oct. 2004. V. A. Vaishampayan, “Design of multiple description scalar quantizers,” IEEE Trans. Inform. Theory, vol. 39, pp. 821–834, May 1993. K. Rose, “Deterministic annealing for clustering, compression, classification, regression, and related optimization problems,” Proc. IEEE, vol. 86, no. 11, pp. 2210–2239, Nov. 1998. P. Koulgi, S. Regunathan, and K. Rose, “Multiple description quantization by deterministic annealing,” IEEE Transactions on Information Theory, vol. 49, pp. 2067 – 2075, 2003. P. Yahampath, "On index assignment and the design of multiple description quantizers," in IEEE Int. Conf. on Acoust., Speech and Signal Process., Proc., Montreal, 2005, vol. 4, pp. 597-600. Y. Zhou and W.-Y. Chan, “Multiple Description Quantizer Design Using a Channel Optimized Quantizer Approach,” in Proc. 38th Annual Conf. on Information Sciences and Systems, Princeton, NJ, March 2004. M. Valipour, F. Lahouti, “Multiple Descriptions with Symbol Based Turbo Codes over Noisy Channels with Packet Loss: Design and Performance Analysis," to appear in IEEE Journal of Selected Topics in Signal Processing, April 2008. M. Wu, A. Vetro, and C.W. Chen, "Multiple-Description Image Coding with Distributed Source Codeing and Side Information," SPIE Multimedia Systems and Applications VII, Vol. 5600, pp. 120-127, October 2004. F. Lahouti, A. K. Khandani, “Soft reconstruction of speech in the presence of noise and packet loss,” IEEE Trans. Audio, Speech and Language Process., Vol. 15, No. 1, pp. 44 – 56, Jan. 2007.
