It is not. If a probability measure on $mathbb{R}$ is absolutely continuous and has density $f$, then "entropy" usually refers to the differential entropy, defined in the Wikipedia page falagar linked to. If the probability measure has discrete support, entropy is defined by an analogous formula, given in this Wikipedia page. In the most classical treatments, these are the only situations covered at all.
However, both of these are special cases of the more general notion of relative entropy that Helge and robin girard pointed out: in the continuous case the reference measure ($nu$ in Helge's notation, $mu$ in robin's notation) is Lebesgue measure, and in the discrete case the reference measure is counting measure on the support.
No comments:
Post a Comment