Wyner-Ziv Quantization and Transform Coding of Noisy Sources at High Rates David Rebollo-Monedero, Shantanu Rane and Bernd Girod
Information Systems Lab. Dept. of Electrical Eng. Stanford University
Outline Introduction
and previous work
Characterize
quantizers for Wyner-Ziv distributed coding of noisy sources at high rates
Extend
the principles of transform coding to Wyner-Ziv coding of noisy sources
Apply
the new quantization and transform coding theory to Wyner-Ziv coding of a noisy image
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
2
Wyner-Ziv Coding of Noisy Sources
Source Data
Noisy Observation
X
Noisy Channel
Lossless Coding
Quantization Index
Q
Z q( z )
Reconstruction
SlepianWolf Encoder
SlepianWolf Decoder
Q
Xˆ xˆ (q, y )
Y
Known statistics Rate Distortion Cost Objective: optimal design of q(z) and ^ x(q,y)
Reconstructed Source Data
Quantization
Side Information
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
3
Quantizers for Wyner-Ziv Coding
Quantizer q(z) cannot depend on x or y y Statistical dependence among X, Y and Z exploited in design y Minimize performance loss with respect to optimal q(z,y)
Mapping of different cells into common quantization index may help performance xˆ ( q, y )
Example Z=X
pX|Y(x|y) x Possibly disconnected quantization region Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
4
State of the Art Clean
source coding with side information at the decoder
y Information-theoretic rate-distortion bounds [Slepian, Wolf, 73] [Wyner, Ziv, 76] [Zamir, 96]
y Practical lossless (Slepian-Wolf) coding [Pradhan, Ramchandran, 99] [García-Frías, Zhao, 01] [Aaron, Girod, 02]
y Practical lossy (Wyner-Ziv) coding using quantizers and transforms [Fleming, Zhao, Effros, 01] [Pradhan, Ramchandran, 01] [Gastpar et al., 01] [Rebollo, Zhang, Girod, 03] [Rebollo, Aaron, Girod, 03] Noisy source coding without side information [Dobrushin, Tsybakov, 62] [Wolf, Ziv, 70] [Ephraim, Gray, 88] Noisy
source coding with side information
y Information-theoretic rate-distortion bounds [Yamamoto, Itoh, 80] [Flynn, Gray, 87] [Witsenhausen, 80]
y Fixed-rate coding [Gubner, 93] [Lam, Reibman, 93] Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
5
Wyner-Ziv Quantization and Transform Coding of Noisy Sources at High Rates
Assume y Availability of ideal Slepian-Wolf coders y Gersho’s conjecture (true for one dimension) y High rates
Previous work in [Rebollo, Aaron, Girod, 03] for clean sources (Z=X) y Theoretic characterization of quantizers at high rates ` Lattice quantizers without index repetition asymptotically optimal ` Performance close to case in which Y available
y DCT optimal transform of source data if conditionally covariance stationary given the side information y In the Gaussian case, side information may also be transformed with no performance loss
Extension to noisy source coding case (Z≠X) Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
6
High-Rate Wyner-Ziv Quantization of a Noisy Source Suppose
y Estimation additively separable: y Traditional Benett’s assumptions and Gersho’s conjecture for conventional quantization apply to Then,
at high rates an asymptotically optimal implementation of a quantizer exists such that y lattice quantizer without index re-use y (Mn moment of inertia) y Same performance as conditional quantizer
Z
q( z)
Q
xˆ(q, y)
Xˆ
Z
xZ ( z )
XZ
q ( xZ )
Q
xˆZ (q, y )
Xˆ Z
Xˆ
xY ( y )
Y WZ Quantization of a Noisy Source
Y An Optimal Implementation at High Rates for Additively Separable Estimation
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
7
Example of Additively Separable Estimation X,
Y and Z zero-mean, jointly Gaussian random scalars Optimal estimation is linear, thus additively separable
Estimation
function used in WZ coding does not depend on Y but does take into account its statistical dependence
If
Y and Z uncorrelated, then
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
8
X Z′ 1
Noisy Observation
X Z1 Z
xZ ( z )
XZ2 XZn
U
T
X Z′ 2 X Z′ n
q1′ q2′ qn′
Q1′
Q1′ SWC
Q2′
Q2′ SWC
Qn′
Qn′ SWC
xˆZ′ 1 xˆZ′ 2 xˆZ′ n
Xˆ Z′ 1
Xˆ Z 1
Xˆ Z′ 2
Xˆ Z 2
Xˆ Z′ n
U
Xˆ Z n
Side Information Y Estimation
Xˆ
Reconstructed Source Vector
Transform of Noisy Observation
xY ( y )
Transform coding of regarded as a clean source, studied in [Rebollo, Aaron, Girod, 03] (rotated, scaled Z-lattice quantizer)
Assume
y Additively separable estimation: y High-rate approximation for (clean) WZ coding at each band Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
9
Optimality of Transform of Noisy Observation Theoretical
results for clean transform Wyner-Ziv coding hold if hypotheses apply to instead of X y Under certain conditions, U is Karhunen-Loève Transform (KLT) for y Conditions satisfied in the Gaussian case y Uniform quantizer with common width in all bands, no index re-use y If wide-sense stationary as n→∞, for each y, then DCT asymptotically optimal choice for U
Same
performance as if optimal estimate had access to side information
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
10
X Z′ 1
Noisy Observation
X Z1
Z
xZ ( z )
XZ2 XZn
U
T
X Z′ 2 X Z′ n
q1′ q2′
qn′
Q1′
Q1′ SWC
Q2′
Q2′ SWC
Qn′
Qn′ SWC
xˆZ′ 1 xˆZ′ 2 xˆZ′ n
Y1′
Y2′
Xˆ Z′ 1
Xˆ Z 1
Xˆ Z′ 2
Xˆ Z 2
Xˆ Z′ n
U
Xˆ Z n
Yn′
y′( y )
Xˆ
Reconstructed Vector
Transform of Side Information
xY ( y ) Y Side Information
Reduce dimension or alphabet of side information at each band: Small performance loss by using Yi’ at each band i instead of Y Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
11
Optimality of Transform of Side Information
Concept of sufficient statistic y Let X (observation) and Θ (data) be r. v. y A statistic for Θ from X is a function of the observation T=t(X) y T sufficient if and only if Θ↔T↔X, or equivalently, I(Θ;T)=I(Θ;X)
Assume hypotheses for transformation of a noisy observation (additive separability and high-rate approximation at each band)
Then y A sufficient statistic for from can be used, instead of , for Slepian-Wolf decoding and reconstruction at each band i, with no loss of performance (asymptotically, at high rates) y In the Gaussian case the best linear MSE estimate is a sufficient statistic
Transformation applied to XZ
Estimation of XZ from side information
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
12
Experimental Setting
Wyner-Ziv transform coding of a noisy image y X = 8×8 blocks of pixels of first 25 frames of ‘foreman’ QCIF video sequence (n=64) y V, W white Gaussian noise with variances σV2= σW2=25 per sample y Z=X+V, Y=X+W, with X, V and W independent
Cases compared 1. Side information Y available at the encoder estimator, Wyner-Ziv transform coding of the estimate (Y only used to improve estimate) 2. Noisy Wyner-Ziv transform coding as proposed in this work 3. Wyner-Ziv transform coding of Z directly, X estimated at the decoder 4. As proposed in this work, but without the use of the side information in the reconstruction functions inside each band
All estimators constrained to be linear, e.g.,
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
13
Experimental Results 38.25 PSNR of best affine estimate = 38.2406 38.2 38.15
PSNR (dB)
38.1 38.05 38 Conditional estimation and WZ transform coding (Case 1) 37.95
Noisy WZ transform coding of Z (Case 2) Direct WZ transform coding of Z (Case 3)
37.9
Noisy WZ w/o side-info in reconstruction (Case 4) 37.85
1.5
2
2.5 Rate (bpp)
3
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
14
Conclusions Assume
y High rates y Conditional expectation of the unseen data X given the side information Y and the observation Z additively separable Wyner-Ziv
quantizers of noisy sources
y Quantizers can be decomposed into estimators and lattice quantizers for clean sources y Same performance as if side information were available Wyner-Ziv
transform coding of noisy sources
y We propose decomposition into estimator and transform coder for a clean source y Under certain conditions DCT optimal y Side information can be replaced by a sufficient statistic Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
15
Wyner-Ziv Quantization and Transform Coding of Noisy Sources at High Rates David Rebollo-Monedero, Shantanu Rane and Bernd Girod
Information Systems Lab. Dept. of Electrical Eng. Stanford University
X
Wyner-Ziv Encoder
Wyner-Ziv Decoder
Reconstructed Source Data
Source Data
Wyner-Ziv Coding
Xˆ
Y
Side Information
Rate-distortion theory for distributed source coding suggests small performance loss [Slepian, Wolf, 73] [Wyner, Ziv, 76] [Zamir, 96]
Many applications, for instance video coding [Aaron, Zhang, Girod, 02] Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
17
High Rate Quantization SNRIN=10 R=1.16
dB
bit
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
18
High Rate Quantization Gaussian
scalar case
22
Wyner-Ziv Bound Conditional q(x|y)
20
noisy version of X
2 σ SNR = X = 5 dB IN σ Z2 σ X2
SNR OUT =
Distributed q(x)
18
SNR=OUT SNR [dB] σ 2X/D[dB] out
Y
16
14 12
D 10
8
[Rebollo, Zhang, Girod, 03]
6
0
0.5
1
1.5
2
2.5
R [bit]
R [bit]
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
19
Conditional Quantization of a Noisy Source MSE
distortion is used and Distortion Estimation
Quantization of a Noisy Source
Quantization of a Clean Source
Rate Estimation
Z
q( z | y)
Q
at the encoder xˆ(q, y)
Xˆ
Y General Conditional Quantization
Z
E[ X y, z]
X
q( x | y)
Q
xˆ(q, y)
Xˆ
Y An Optimal Implementation at Any Rate
Rebollo, Rane, Girod: Wyner-Ziv Quantization and Transform Coding of Noisy Sources
20
