ssjalakazam
Programmer
I am trying to learn how to calculate information gain, and have hit a brick wall. Gain(Y,X) = entropy(Y) - entropy(Y|X)
The first term, entropy, is easy.
But, the entropy(Y|X) is the problem...
so entropy(Y|X) = SUM (prob[x] * entropy(Y|X = x)), over all values of Y.
But what is entropy(Y|X=x)? How do you find it? I have seen nothing online that explains this, and it seems crucial to calculating information gain.
The first term, entropy, is easy.
But, the entropy(Y|X) is the problem...
so entropy(Y|X) = SUM (prob[x] * entropy(Y|X = x)), over all values of Y.
But what is entropy(Y|X=x)? How do you find it? I have seen nothing online that explains this, and it seems crucial to calculating information gain.