SciTech.blog
SciTech.blog
Intuitively, entropy is the measure of in how many states a system can be. But how can we arrive from this intuition at \( H(X)= -\sum_i p_i \log p_i \)?