|
|
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Is it possible to communicate reliably from one point to
another if we only have a noisy communication channel? How can the
information content of a random variable be measured? This course
will discuss the remarkable theorems of Claude Shannon, starting
from the source coding theorem, which motivates the entropy as the
measure of information, and culminating in the noisy channel
coding theorem. Along the way we will study simple examples of
codes for data compression and error correction. |
| Pages | 1 |
| Parts | |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 1: Definitions. Marginal Probability, Conditional
probability, Bayes' theorem, joint entropy, contitional
entropy, relative entropy, convex function, jensen's
inequality |
| Pages | 2 |
| Parts | |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 2: Why is entropy a fundamental measure of
information content? Assymptotic Equipartition Principle,
Shannon's source coding theorem, Chebyshev's inequality |
| Pages | 3 |
| Parts | |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 3: Practical Source Coding: data compression with
symbol codes. codes, expected length, Kraft-McMillan
inequality, optimal source code lengths, huffman codes,
convex function, Jensen's inequality, the relative entropy or
Kullback-Leibler distance, Gibb's inequality. |
| Pages | 3 |
| Parts | |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 4: Arithmetic Coding. |
| Pages | 2 |
| Parts | |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 5: Channel Coding |
| Pages | 4 |
| Parts | The Big Picture
Review of probabilty theory and conditional, joint and
mutual information
Communication over a noisy channel
The channel coding theorem
Inference problems |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 6: Channel Coding |
| Pages | 2 |
| Parts | The Theorem
Proof of the channel coding theorem |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Part 7 and 8: Codes |
| Pages | 5 |
| Parts | Description of Established Codes
Decoding by variational free energy minimization
The Gaussian Channel
Capacity of Gaussian Channel |
| |
| |
| Authors | David MacKay |
| Language | English |
| Type | public |
| Url | |
| Summary | Bayes |
| Pages | 3 |
| Parts | Inference |
| |
generated by OrgaDoc 0.8
|