Transforms for High-Rate Distributed Source Coding David Rebollo-Monedero, Anne Aaron and Bernd Girod
Information Systems Lab Dept. of Electrical Eng. Stanford University
Outline Characterize
quantizers for distributed source coding at
high rates Use
principles of conventional transform coding in distributed source coding
Apply
new quantization and transformation theory to distributed video coder
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
2
Wyner-Ziv Coding Lossless Coding
Reconstruction
Quantization Index Q
X q( x )
SlepianWolf Encoder
SlepianWolf Decoder
Q
Xˆ xˆ (q, y )
Y
Rate-distortion theory for distributed source coding suggests small performance loss
Reconstructed Source Vector
Source Vector
Quantization
Side Information Vector
[Slepian, Wolf, 73] [Wyner, Ziv, 76] [Zamir, 96]
Rate Distortion D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
3
Quantizers for Distributed Coding
Quantizater q(x) cannot depend on y, but statistical dependence between X and Y exploited Mapping of different cells into common quantization index may help performance In [Rebollo, Zhang, Girod, 03] y Rate measure r(q,y) introduced to extend Lloyd algorithm to Slepian-Wolf coding y Quantizers found in experiments were uniform y Performance close to case in which Y available
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
4
State of the Art Lossless
distributed source coding
[Pradhan, Ramchandran, 99] [García-Frías, Zhao, 01] [Aaron, Girod, 02] Quantization
for distributed source coding
y Extension of Lloyd algorithm [Fleming, Zhao, Effros, 01] y Further extension for Slepian-Wolf coding [Rebollo, Zhang, Girod, 03] Transforms
for distributed source coding [Gastpar et al., 03]
y Conditional covariance matrix constant with side info y Not in the context of a practical coding system with quantizers for distributed source coding
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
5
High-Rate Quantization - Theorem Assume
traditional high rate results for PDF of X given {Y=y}, for each y y Bennett’s assumptions (imply well behaved PDFs) y Gersho’s conjecture (true if n=1) y Optimal family of lattice quantizers q(x|y) on x for each y
Then,
there exists asymptotically optimal q(x) for high rate
y Lattice quantizer, no index repetition y (Mn normalized moment of inertia, M1=1/12) y No performance loss by not using Y in quantization y No performance loss by not using Y in reconstruction (but still used in SW decoder!)
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
6
Quantization Index Q
X q( x )
SlepianWolf Encoder
SlepianWolf Decoder
Q
Xˆ xˆxˆ(q(q, )y )
Y Q
X q( x | y)
Cond. Encoder
Cond. Decoder
Q
Reconstructed Source Vector
Source Vector
High-Rate Quantization Performance
Side Information Xˆ
xˆ (q, y )
Y D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
7
X 1′
Source Vector
X1
X2
Xn
UT
X 2′ X n′
q1′ q2′ qn′
Q1′
Q1′ SWC
Q2′
Q2′ SWC
Qn′
Qn′ SWC
xˆ1′ xˆ2′
Xˆ 1
Xˆ 2′
Xˆ 2
Xˆ n′
xˆn′
Y
Xˆ 1′
U
Xˆ n
Reconstructed Source Vector
Transform of Source Data
Side Information Vector
Orthonormal transformation Rate Distortion Goal: minimum performance loss w.r.t. joint coding D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
8
Transform of Source Data - Theorem Define
y Covariance of error of best non-linear estimate y If constant with y, then it is just Assume
y High-rate approximation for each band i y Normalized PDF of transformed components constant with U y Variance of conditional distribution of Xi´ given Y changes very little with Y Then,
optimal rate-distortion performance achieved when
y Uniform quantizer common width in all bands y U is Karhunen-Loève Transform (KLT) for
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
9
Transform of Source Data - Corollary If
X and Y jointly Gaussian y Only high rate approximation necessary y Other hypotheses hold exactly, KLT indeed optimal
If
(Xi|{Y=y})i wide sense stationary as n→∞, for each y y Only high rate approximation and PDF invariance necessary y Discrete Cosine Transform (DCT) asymptotically optimal choice for U
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
10
X 1′
Source Vector
X1
X2
Xn
UT
X 2′ X n′
q1′ q2′ qn′
Q1′
Q1′
xˆ1′
SWC
Q2′
Q2′
xˆ2′
SWC
Qn′
Qn′
Xˆ 1
Xˆ 2′
Xˆ 2
Y1′ Y2′ Xˆ n′
xˆn′
SWC
Xˆ 1′
U
Xˆ n
Reconstructed Source Vector
Transform of Side Information
Yn′
VT Ym
Y1
Side Information Vector
Goal: minimum performance loss by using Yi’ at each branch instead of Y D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
11
Transform of Side Information - Theorem Assume
y X and Y jointly Gaussian y High rate approximation Then
y Optimal transformation of side info is
Source transformation
Estimation of source vector from side info
y No loss in rate or distortion w.r.t. using entire vector Y
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
12
Wyner-Ziv DCT Video Coder X WZ frames (even)
X Inverse Transform
Transform
Xi’ Xi’
Scalar Quantizer
Turbo Encoder
Turbo Decoder
Reconstruction
Request bits
For each transform band i
Buffer
Yi’ Transform
Y Key frames (odd)
K
Conventional Intraframe coding
Interpolation/ Extrapolation Conventional Intraframe decoding
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
K 13
Mother Sequence: Pixel vs DCT
First 100 frames of QCIF Mother and Daughter sequence Key frames – odd WZ frames – even Side information generated from motioncompensated interpolation (MC-I) or extrapolation (MC-E) Compared to DCTbased intraframe coding and H.263+ I-B-I-B coding Similar step size in all bands D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
14
Conclusions High-rate
quantization for distributed coding
y Lattice quantizers without index repetition asymptotically optimal y Operational Wyner-Ziv rate loss vanishes as D → 0 Transforms
for distributed coding
y Transformation of the source vector ` KLT of source vector determined by ` Optimal in the Gaussian case ` DCT optimal if source process conditionally stationary
y Transformation of the side information, Gaussian case ` Transformed estimate of source data given side information ` No loss in rate or distortion performance Experiments
show important performance improvement
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
15
Transforms for High-Rate Distributed Source Coding David Rebollo-Monedero, Anne Aaron and Bernd Girod
Information Systems Lab Dept. of Electrical Eng. Stanford University
X
Wyner-Ziv Encoder
Wyner-Ziv Decoder
Reconstructed Source Data
Source Data
Wyner-Ziv Coding
Xˆ
Y
Side Information
Rate-distortion theory for distributed source coding suggests small performance loss [Slepian, Wolf, 73] [Wyner, Ziv, 76] [Zamir, 96]
Many applications, for instance video coding [Aaron, Zhang, Girod, 02] D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
17
High-Rate Quantization Gaussian
scalar case
22
Wyner-Ziv Bound Conditional q(x|y)
20
noisy version of X
2 σ SNR = X = 5 dB IN σ Z2 σ X2
SNR OUT =
Distributed q(x)
18
SNR=OUT SNR [dB] σ 2X/D[dB] out
Y
16
14 12
D 10
8
[Rebollo, Zhang, Girod, 03]
6
0
0.5
1
1.5
2
2.5
R [bit]
R [bit]
D. Rebollo, A. Aaron, B. Girod: Transforms for High-Rate Distributed Source Coding
18
