Discuss the various ideas used by Shannon to prove that information can be sent reliably over a channel-at all rates upto the channel capacity.

Differentiate between conditional and relative entropy.

2. Derive an expression to show the relation between entropy and mutual information. Also substantiate it, with the help of a Venn diagram.

List any four properties of channel capacity.

Explain two examples of channel capacity with neat sketches.

4. Prove that the capacity of a Gaussian channel with power constraint P and noise variance N is log(1 bits per transmission.

5. Discuss how the Reed-Solomon codes used in concatenated form and are very useful in correcting burst errors.

6. Explain in detail the standard array and syndrome decoding.

7. Explain the reason, why for MPSK modulation the bandwidth efficiency increases with higher dimensional signalling but for MFSK it decreases.

8. Suppose that a function is band limited to the spectrum of the function is O for all frequencies greater than then prove that the function is completely determined by samples of the function spaced 1/2W seconds apart.

9. Let X and Y represent random variables with probability distributions and respectively. They are not independent.

Find the marginal entropy of X and also the mutual information of X with itself.

Find the joint entropy Y).

Find the mutual information Y).

10. Write short notes on any two of the following:

Bandwidth Efficiency Plane

Shannon Limit

BCH Codes