我正在自学机器学习。
关于信息获取的一个简单问题。 如何根据pic的数据计算信息增益? 我不明白。
有人可以解释如何从第一行获得0.992385吗?
非常感谢!
答案 0 :(得分:2)
设g(x)= -x * log(x)/ log(2)
总人数是48842。
H(财富,关系)= g [52/48842] + g [111/48842] + g [309/48842] + g [1093/48842] + g [1238/48842] + g [1276/48842] + g [1454/48842] + g [4816/48842] + g [7470/48842] + g [8846/48842] + g [10870/48842] + g [11307/48842] = 2.7835
H(财富)= g(差/总)+ g(富/总)= g [0.239282] + g [0.760718] = 0.793844
H(关系)= g(丈夫/总数)+ g(Not_in_familly / total)+ ... = g [0.0308341] + g [0.0477253] + g [0.10493] + g [0.155215] + g [0.257627] + g [0.403669] = 2.15508
H(财富|关系)= H(财富,关系) - H(关系)= 2.7835 - 2.15508 = 0.628421
IG = H(财富) - H(财富|关系)= H(财富)+ H(关系) - H(财富,关系) = 0.165423
这是用Mathematica编写的源代码。如果您认为需要使用其他语言查看源代码,请使用您的首选语言在下面发表评论。如果我有时间,我会打字。 - 干杯,汉斯
Mathematica中的源代码
(* ============================================= ====== *)
m = {{10870, 8846}, {11307, 1276}, {1454, 52}, {7470, 111},
{4816, 309}, {1238, 1093}};
iTot = Total[ Flatten[m]];
h[x_] := -x * Log[2, x];
fHAll = Sum[ h[ m[[i, j]]/ iTot ], {i, 6}, {j, 2}] // N;
fHWealth = h[ Total[ m[[All, 1]]]/iTot] + h[ Total[ m[[All, 2]]]/iTot] // N ;
fHRelation = Sum[
h[ Total[ m[[i ]]]/iTot] , {i, Length[m]}] // N;
fWealthGivenRelation = fHAll - fHRelation;
Print[" H(relation, wealth) = ", fHAll];
Print[" H(relation) = ", fHRelation];
Print[" H(wealth) = ", fHWealth];
Print[" H(wealth | relation) = ", fWealthGivenRelation];
Print[" IG = MI = ", fHWealth - fWealthGivenRelation, " = ",
fHWealth + fHRelation - fHAll];
(* =================== output ==================== *)
H(relation, wealth) = 2.7835
H(relation) = 2.15508
H(wealth) = 0.793844
H(wealth | relation) = 0.628421
IG = MI = 0.165423 = 0.165423
哎呀,我没有回答你的主要问题。这是答案。
H(财富关系=丈夫) = g(10870 /(10870 + 8846))+ g(8846 /(10870 + 8846)) = 0.992385