You cannot do inference without making assumptions.
An edition of Information Theory, Inference & Learning Algorithms
(2003)
Information Theory, Inference & Learning Algorithms
1st edition
by David J.C. MacKay
- 1 Rating
- 14 Want to read
- 0 Currently reading
- 1 Have read
Share
×Close
Book Jacket:
This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks.
Publisher Description:
This textbook offers comprehensive coverage of Shannon's theory of information as well as the theory of neural networks and probabilistic data modelling. It includes explanations of Shannon's important source encoding theorem and noisy channel theorem as well as descriptions of practical data compression systems. Many examples and exercises make the book ideal for students to use as a class textbook, or as a resource for researchers who need to work with neural networks or state-of-the-art error-correcting codes.
Check nearby libraries
Buy this book
Previews available in: Undetermined English
Subjects
Information theory, Inference, Machine Learning, Bayesian, Aprendizado computacional, Information, Théorie de l', Inferenz, Statistische analyse, Information Theory, Toepassingen, Maschinelles Lernen, Informationstheorie, Teoria da informação, Informatietheorie, Algoritmen, Algorithms, Teoria da informacao, Information, Theorie de l', Inferenz <künstliche intelligenz>, Maschinelles lernen, Inferenz (künstliche intelligenz), Q360 .m23 2003, 003/.54, Dat 708f, Qh 210, Sk 880, St 130, St 300Showing 3 featured editions. View all 3 editions?
| Edition | Availability |
|---|---|
1
Information Theory, Inference and Learning Algorithms
2004, University of Cambridge ESOL Examinations, TBS
in English
0521644445 9780521644440
|
zzzz
Libraries near you:
WorldCat
|
2
INFORMATION THEORY, INFERENCE, AND LEARNING ALGORITHMS.
2003, CAMBRIDGE UNIV PRESS, Cambridge University Press
in Undetermined
0521642981 9780521642989
|
zzzz
Libraries near you:
WorldCat
|
3
Information Theory, Inference & Learning Algorithms
2003, Cambridge University Press
Hardcover
in English
- 1st edition
0521642981 9780521642989
|
aaaa
Libraries near you:
WorldCat
|
Book Details
First Sentence
"In this chapter we discuss how to measure the information content of the outcome of a random experiment."
Table of Contents
Preface.
Page v
Chapter 1.
Introduction to information theory
Page 3
Chapter 2.
Probability, entropy, and inference
Page 22
Chapter 3.
More about inference
Page 48
Part I.
Data Compression
Page 65
Chapter 4.
The source coding theorem
Page 67
Chapter 5.
Symbol codes
Page 91
Chapter 6.
Stream codes
Page 110
Chapter 7.
Codes for integers
Page 132
Part II.
Noisy-Channel Coding
Page 137
Chapter 8.
Correlated random variables
Page 138
Chapter 9.
Communication over a noisy channel
Page 146
Chapter 10.
The noisy-channel coding theorem
Page 162
Chapter 11.
Error-correcting codes and real channels
Page 177
Part III.
Further Topics in Information Theory
Page 191
Chapter 12.
Hash codes: codes for efficient information retrieval
Page 193
Chapter 13.
Binary codes
Page 206
Chapter 14.
Very good linear codes exist
Page 229
Chapter 15.
Further exercises on information theory
Page 233
Chapter 16.
Message passing
Page 241
Chapter 17.
Communication over constrained noiseless channels
Page 248
Chapter 18.
An aside: crosswords and codebreaking
Page 260
Chapter 19.
Why have sex? Information acquisition and evolution
Page 269
Part IV.
Probabilities and Inference
Page 281
Chapter 20.
An example inference task: clustering
Page 284
Chapter 21.
Exact inference by complete enumeration
Page 293
Chapter 22.
Maximum likelihood and clustering
Page 300
Chapter 23.
Useful probability distributions
Page 311
Chapter 24.
Exact marginalization
Page 319
Chapter 25.
Exact marginalization in trellises
Page 324
Chapter 26.
Exact marginalization in graphs
Page 334
Chapter 27.
Laplace's method
Page 341
Chapter 28.
Model comparison and Occam's razor
Page 343
Chapter 29.
Monte Carlo methods
Page 357
Chapter 30.
Efficient Monte Carlo methods
Page 387
Chapter 31.
Ising models
Page 400
Chapter 32.
Exact Monte Carlo sampling
Page 413
Chapter 33.
Variational methods
Page 422
Chapter 34.
Independent component analysis and latent variable modelling
Page 437
Chapter 35.
Random inference topics
Page 445
Chapter 36.
Decision theory
Page 451
Chapter 37.
Bayesian inference and sampling theory
Page 457
Part V.
Neural Networks
Page 467
Chapter 38.
Introduction to neural networks
Page 468
Chapter 39.
The single neuron as a classifier
Page 471
Chapter 40.
Capacity of a single neuron
Page 483
Chapter 41.
Learning as inference
Page 492
Chapter 42.
Hopfield networks
Page 505
Chapter 43.
Boltzmann machines
Page 522
Chapter 44.
Supervised learning in multilayer networks
Page 527
Chapter 45.
Gaussian processes
Page 535
Chapter 46.
Deconvolution
Page 549
Part VI.
Sparse Graph Codes
Page 555
Chapter 47.
Low-density parity-check codes
Page 557
Chapter 48.
Convolutional codes and turbo codes
Page 574
Chapter 49.
Repeat-accumulate codes
Page 582
Chapter 50.
Digital fountain codes
Page 589
Part VII.
Appendices
Page 597
Appendix A.
Notation
Page 598
Appendix B.
Some physics
Page 601
Appendix C.
Some mathematics
Page 605
Bibliography.
Page 613
Index.
Page 620
Edition Notes
Full text is online.
Classifications
The Physical Object
ID Numbers
Excerpts
Page 26,
added by David.
A central theme of the book.
Links outside Open Library
Community Reviews (0)
Feedback?
| December 19, 2023 | Edited by ImportBot | import existing book |
| November 8, 2023 | Edited by raybb | merge authors |
| November 8, 2023 | Edited by raybb | Merge works |
| June 23, 2015 | Edited by David | Update covers |
| December 10, 2009 | Created by WorkBot | add works page |