Question

想知道是否有更好的方法来获得2D numpy数组的概率。也许使用一些numpy的内置函数。

为简单起见，我们假设有一个示例数组：

[['apple','pie'],
['apple','juice'],
['orange','pie'],
['strawberry','cream'],
['strawberry','candy']]

想获得如下概率：

['apple' 'juice'] --> 0.4 * 0.5 = 0.2
['apple' 'pie']  --> 0.4 * 0.5 = 0.2
['orange' 'pie'] --> 0.2 * 1.0 = 0.2
['strawberry' 'candy'] --> 0.4 * 0.5 = 0.2
['strawberry' 'cream'] --> 0.4 * 0.5 = 0.2

哪里有果汁＆＃39;因为第二个词的概率为0.2。由于苹果的概率为2/5 * 1/2（果汁）。

另一方面，＆＃39;馅饼＆＃39;作为第二个词，概率为0.4。来自＆＃39; apple＆＃39;的概率组合。和＆＃39; orange＆＃39;。

我解决问题的方法是向数组添加3个新列，第1列，第2列的概率和最终概率。按第1列对数组进行分组，然后按第2列进行分组，并相应地更新概率。

以下是我的代码：

a = np.array([['apple','pie'],['apple','juice'],['orange','pie'],['strawberry','cream'],['strawberry','candy']])

ans = []
unique, counts = np.unique(a.T[0], return_counts=True)                      ## TRANSPOSE a, AND GET unique
myCounter = zip(unique,counts)
num_rows = sum(counts)
a = np.c_[a,np.zeros(num_rows),np.zeros(num_rows),np.zeros(num_rows)]       ## ADD 3 COLUMNS to a

groups = []
## GATHER GROUPS BASE ON COLUMN 0
for _unique, _count in myCounter:
    index = a[:,0] == _unique                                               ## WHERE COLUMN 0 MATCH _unique
    curr_a = a[index]
    for j in range(len(curr_a)):
        curr_a[j][2] = _count/num_rows
    groups.append(curr_a)

## GATHER UNIQUENESS FROM COLUMN 1, PER GROUP
for g in groups:
    unique, counts = np.unique(g.T[1], return_counts=True)
    myCounter = zip(unique, counts)
    num_rows = sum(counts)

    for _unique, _count in myCounter:
        index = g[:, 1] == _unique
        curr_g = g[index]
        for j in range(len(curr_g)):
            curr_g[j][3] = _count / num_rows
            curr_g[j][4] = float(curr_g[j][2]) * float(curr_g[j][3])        ## COMPUTE FINAL PROBABILITY
        ans.append(curr_g[j])

for an in ans:
    print(an)

输出：

['apple' 'juice' '0.4' '0.5' '0.2']
['apple' 'pie' '0.4' '0.5' '0.2']
['orange' 'pie' '0.2' '1.0' '0.2']
['strawberry' 'candy' '0.4' '0.5' '0.2']
['strawberry' 'cream' '0.4' '0.5' '0.2']

想知道是否有更好的短/快方式使用numpy或其他手段。添加列不是必需的，这只是我的方式。其他方法也是可以接受的。

Answer 1

根据您给出的概率分布的定义，您可以使用pandas来做同样的事情，即

import pandas as pd
a = np.array([['apple','pie'],['apple','juice'],['orange','pie'],['strawberry','cream'],['strawberry','candy']])

df = pd.DataFrame(a)
# Find the frequency of first word and divide by the total number of rows
df[2]=df[0].map(df[0].value_counts())/df.shape[0]
# Divide 1 by the total repetion 
df[3]=1/(df[0].map(df[0].value_counts()))
# Multiply the probabilities 
df[4]= df[2]*df[3]

输出：

            0      1    2    3    4
0       apple    pie  0.4  0.5  0.2
1       apple  juice  0.4  0.5  0.2
2      orange    pie  0.2  1.0  0.2
3  strawberry  cream  0.4  0.5  0.2
4  strawberry  candy  0.4  0.5  0.2

如果您想以列表的形式使用，可以使用df.values.tolist()

如果您不想要列，那么

df = pd.DataFrame(a)
df[2]=((df[0].map(df[0].value_counts())/df.shape[0]) * (1/(df[0].map(df[0].value_counts()))))

输出：

           0      1    2
0       apple    pie  0.2
1       apple  juice  0.2
2      orange    pie  0.2
3  strawberry  cream  0.2
4  strawberry  candy  0.2

对于组合概率print(df.groupby(1)[2].sum())

candy    0.2
cream    0.2
juice    0.2
pie      0.4

从2D numpy获得联合概率的最佳方法

1 个答案: