# Information theory

Main Article
Discussion
Related Articles  [?]
Bibliography  [?]
The Shannon approach was based on the fundamental notion that sequences of symbols had more content in the communication between parties if they were more rare. This ultimately produced mathematical characterizations of information as the inverse of randomness, also called entropy. Shannon's mathematical foundation consisted of a number that could be calculated for the information content of a sequence of symbols in a language given the frequencies of the use of symbols within that language. In particular, ${\displaystyle \scriptstyle H=K\sum _{i}P(i)ln(i)}$ for all symbols i in the language. This forms the basis for most of the efficient coding approaches to communications and for most of the historical and current compression algorithms and approaches as well as setting the theoretical limits for compression based on the selection of symbol sets.