Table of contents

Shannon Theory of Communication

From entropy and coding to channel capacity and reliable communication

Read each section in order. Every title can be opened as a TheoryTrace document.

• Cover
• Copyright
• How to read this book
• Introduction
• Chapter 1: Communication as a Mathematical Problem
• Chapter 2: Probability for Communication Theory
• Chapter 3: Information and Self-Information
• Chapter 4: Entropy of Discrete Sources
• Chapter 5: Joint, Conditional, and Mutual Information
• Chapter 6: Relative Entropy and Information Inequalities
• Chapter 7: Discrete Memoryless Sources
• Chapter 8: Typical Sequences and the Asymptotic Equipartition Property
• Chapter 9: Lossless Source Coding
• Chapter 10: Huffman Coding and Arithmetic Coding
• Chapter 11: The Source Coding Theorem
• Chapter 12: Discrete Channels and Transition Probabilities
• Chapter 13: Channel Capacity
• Chapter 14: Error Probability, Decision Rules, and Decoding
• Chapter 15: The Noisy Channel Coding Theorem
• Chapter 16: Error-Correcting Codes in Shannon Theory
• Chapter 17: Continuous Channels and Differential Entropy
• Chapter 18: The Gaussian Channel
• Chapter 19: Bandwidth, Power, and Spectral Efficiency
• Chapter 20: Rate-Distortion Theory
• Chapter 21: Entropy Rates and Sources with Memory
• Chapter 22: Modern Perspectives and Applications
• Conclusion