JE – 799
*JE799*
VI Semester B.E. (E&C) Degree Examination, June/July 2013 (Y2K6 Scheme) EC 604 : INFORMATION THEORY AND CODING Time : 3 Hours
Max. Marks : 100
Instruction : Answer any five full questions choosing atleast two questions from each Part. PART – A 1. a) Define nth extension of zero memory information source clearly, in terms source symbols and probability distribution. If H(S) is entropy of zero memory source and H(Sn) is the entropy of its nth extension. State and prove the (2+1+5) relation between H(S) and H(Sn). b) Let X represent the outcome of tossing fair coin two time. What is the entropy X ?
4
c) A Markoff Source is given below
Find the entropy of each state and the source entropy. 2. a) The table below shows 5 binary code. Identify (with reason) the instantaneous code/codes. Source
Code
Code
Code
Code
Code
Symbols
A
B
C
D
E
S1
00
0
0
0
0
S2
01
100
10
100
10
S3
10
110
110
110
110
S4
11
111
111
11
11
8
8
P.T.O.
JE – 799
*JE799*
-2-
b) Consider a discrete memoryless source with alphabet {s0, s1, s2} and probabilities {0.7, 0.15, 0.15}. Let the source be extended to order 2. Apply the Huffman algorithm to the resulting extended source and find efficiency of the code and average code word length.
6
c) Consider a zero memory source S = [s1, s2, s3, s4] with symbol probabilities 1 1 1⎤ ⎡ P = ⎢ 1 , , , ⎥ . Show that it is impossible to encode the symbols from ⎣ 2 4 8 8⎦ this source into a uniquely decodable binary code with average length L less
than 1
3 bits/symbol. Develop such a code. 4
6
3. a) Given Joint probability matrix of discrete channel as ⎡ 0 .2 ⎢ ⎢ 0 .1 P( X, Y ) = ⎢ 0 ⎢ ⎢0 .04 ⎢⎣ 0
0
0 .2
0 ⎤ ⎥ 0 .01 0 .01 0 .01⎥ 0 .02 0 .02 0 ⎥ ⎥ 0 .04 0 .01 0 .06 ⎥ 0 .06 0 .02 0 .02 ⎥⎦
determine H(X), H(X,Y), H (X/Y) and I (X;5).
12
b) Define a deterministic channel and draw its channel diagram. Show that for such channel I (A; B) = H (B), where A and B are I/P and O/P alphabets of the channel. 4. a) A continuous RV X is constrained to an average value
1 α
8
and is defined for
x≥0
i) Find the density function that maximizes H (x). ii) Find the maximum value of H(X).
8
b) i) State and explain Shannon-Hartley law. ii) Derive the expression for limit of the channel capacity. iii) Discuss the trade off between SNR and BW.
12
*JE799*
-3-
JE – 799
PART – B 5. a) Define the generator matrix G and parity check matrix H of an (n,k) LBC, and prove that HGT=0.
5
b) Define syndrome of an (n,k) LBC, and show that i) The syndrome depends only on the error pattern, and not on the transmitted code word. ii) All error patterns that differ at most by a code word have the same syndrome.
6
⎡0 1 1 1 0 0⎤ ⎢ ⎥ c) Given the generator matrix of a (6, 3) LBC, as G = ⎢ 1 0 1 0 1 0 ⎥ ⎢ ⎥ ⎢⎣ 1 1 0 0 0 1⎥⎦
Design the encoder circuit of (6,3) LBC and explain its operation with an example.
9
6. a) Design the decoder circuit for (6,3) LBC whose generator matrix is as given at Q.5 (c).
6
b) Check whether the following polynomials could be generator polynomial of a cyclic code for given ‘n’ and ‘k’. i) (6,3) cyclic code, g(x) = x + x2 + x3 ii) (6,3) cyclic code, g(x) = 1+ x + x5 iii) (7,4) cyclic code, g(x) = 1 + x + x7 reason your answer.
6
c) Let the generator polynomial of (7, 4) cyclic code be g(x) = 1 + x + x3. i) Compute the parity check polynomial of (7,4) cyclic code. ii) If received word is R = (0010110), check if R is in error or not.
8
7. a) The generator polynomial for a (7, 4) cyclic code is g(x) = 1+ x + x3. i) Draw the syndrome calculator diagram and explain its operation if the received word R = (0001011). Show the contents of syndrome register as the recived word R is shifted into the circuit. 10 b) Write short notes on : i) BCH codes ii) Reed-Soloman codes.
(4+6)
JE – 799
-4-
*JE799*
8. a) A convolutional encoder has a single shift register with two stages, two mod-2 adders and an output multiplexer. The generator sequence of the encoder are as follows g(1) = [1, 1, 1] and g(2) = [ 1, 0, 1] i) Draw the block diagram ii) Determine the encoder output using time domain approach, for the message sequence 10111 iii) Construct the state diagram for the encoder.
(2+3+6)
b) Construct the code tree for the convolutional encoder given below. Trace the path through the tree that corresponds to the message sequence 10111.
9
______________
