Mathematics of Information

Starting with the Shannon-Wiener approach to mathematical information theory, allowing a mathematical "measurement" of an amount of information, the book begins by defining the terms message and information and axiomatically assigning an amount of information to a probability. The second part explores countable probability spaces, leading to the definition of Shannon entropy based on the average amount of information; three classical applications of Shannon entropy in statistical physics, mathem