Question

这个问题与my own question on Cross Validated有关，虽然这个问题专注于在Python中找到特定的解决方案，因此我将其发布在此处。

我试图根据事件的发生频率对事件进行分类。我看到的数据集大致如下：

month_year,geographic_zone,event_type,count_of_occurrences '2016-01',1,'A',50 '2016-01',1,'B',20 '2016-01',2,'A',10 '2016-01',2,'B',18 '2016-02',1,'A',62 '2016-02',1,'B',29 '2016-02',2,'A',14 '2016-02',2,'B',22 '2016-03',1,'A',59 '2016-03',1,'B',27 '2016-03',2,'A',16 '2016-03',2,'B',23

每月为n区域和m事件类型（在此简化案例中为2和2）收集数据。我得到了这些事件在该时间和地点发生的频率。

鉴于[month_year, geographic_zone]，我想预测未来发生这些事件的可能性。我不确定如何使用count_of_occurrences列来训练分类器。问题是我不知道看不见的数据的事件数，所以我不能使用clf.predict([month_year, geographic_zone, count_of_occurrences])之类的东西来查询模型。也许概率分类器更适合？

这是我当前代码的简化版本，包括我挣扎的评论：

from sklearn import svm
from sklearn.model_selection import train_test_split

X = [
    # [month_year, geographic_zone, count_of_occurrences] after encoding
    [1, 1, 50],
    [1, 1, 20],
    [1, 2, 10],
    [1, 2, 18],
    [2, 1, 62],
    [2, 1, 29],
    [2, 2, 14],
    [2, 2, 22],
    [3, 1, 59],
    [3, 1, 27],
    [3, 2, 16],
    [3, 2, 23],
]

# event_types, 1=A, 2=B
y = [
  1, 2, 1, 2,
  1, 2, 1, 2,
  1, 2, 1, 2,
]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20)

clf = svm.SVC(probability=True)

# I am fitting the model using the count_of_occurrences feature, however
# I won't have knowledge about this value for unseen data all I will really
# know is the month_year and geographic_zone for which I want to make predictions
clf.fit(X_train, y_train)

print(clf.predict_proba(X_test))

如何在分类器中使用出现/频率计数？

Answer 1

您可以将相应数量的事件放入训练集，让模型自己计算它们的相对概率。因此，在数据预处理和模型利用期间不需要使用count_of_occurences： - ）。

顺便说一句，不是直接问题，但如果您的数据是季节性的，那么您不应忘记将月份和年份拆分为单独的功能。

from sklearn import svm


data = [
    # year, month, geo, type, count
    [2016, 1, 1, 'A', 50],
    [2016, 1, 1, 'B', 20],
    [2016, 1, 2, 'A', 10],
    [2016, 1, 2, 'B', 18],
    [2016, 2, 1, 'A', 62],
    [2016, 2, 1, 'B', 29],
    [2016, 2, 2, 'A', 14],
    [2016, 2, 2, 'B', 22],
    [2016, 3, 1, 'A', 59],
    [2016, 3, 1, 'B', 27],
    [2016, 3, 2, 'A', 16],
    [2016, 3, 2, 'B', 23],
]

X = []
y = []

for year, month, geo, t, count in data:
    for i in range(count):
        X.append([year, month, geo])
        y.append(t)

clf = svm.SVC(probability=True)

clf.fit(X, y)

test = [
    [year, month, geo]
    for year in [2016, 2017]
    for month in range(1, 13)
    for geo in [1, 2]
]

prediction = clf.predict_proba(test)

for (year, month, geo), proba in zip(test, prediction):
    s = " ".join("%s=%.2f" % (cls, p)
                 for cls, p in zip(clf.classes_, proba))
    print("%d-%02d geo=%d: %s" % (year, month, geo, s))

结果：

2016-01 geo=1: A=0.69 B=0.31
2016-01 geo=2: A=0.39 B=0.61
2016-02 geo=1: A=0.69 B=0.31
2016-02 geo=2: A=0.39 B=0.61
2016-03 geo=1: A=0.69 B=0.31
2016-03 geo=2: A=0.39 B=0.61
2016-04 geo=1: A=0.65 B=0.35
2016-04 geo=2: A=0.43 B=0.57
2016-05 geo=1: A=0.59 B=0.41
2016-05 geo=2: A=0.50 B=0.50
2016-06 geo=1: A=0.55 B=0.45
2016-06 geo=2: A=0.54 B=0.46
2016-07 geo=1: A=0.55 B=0.45
2016-07 geo=2: A=0.54 B=0.46
2016-08 geo=1: A=0.55 B=0.45
2016-08 geo=2: A=0.54 B=0.46
2016-09 geo=1: A=0.55 B=0.45
2016-09 geo=2: A=0.55 B=0.45
2016-10 geo=1: A=0.55 B=0.45
2016-10 geo=2: A=0.55 B=0.45
2016-11 geo=1: A=0.55 B=0.45
2016-11 geo=2: A=0.55 B=0.45
2016-12 geo=1: A=0.55 B=0.45
2016-12 geo=2: A=0.55 B=0.45
2017-01 geo=1: A=0.65 B=0.35
2017-01 geo=2: A=0.43 B=0.57
2017-02 geo=1: A=0.65 B=0.35
2017-02 geo=2: A=0.43 B=0.57
2017-03 geo=1: A=0.65 B=0.35
2017-03 geo=2: A=0.43 B=0.57
2017-04 geo=1: A=0.62 B=0.38
2017-04 geo=2: A=0.46 B=0.54
2017-05 geo=1: A=0.58 B=0.42
2017-05 geo=2: A=0.51 B=0.49
2017-06 geo=1: A=0.55 B=0.45
2017-06 geo=2: A=0.54 B=0.46
2017-07 geo=1: A=0.55 B=0.45
2017-07 geo=2: A=0.54 B=0.46
2017-08 geo=1: A=0.55 B=0.45
2017-08 geo=2: A=0.54 B=0.46
2017-09 geo=1: A=0.55 B=0.45
2017-09 geo=2: A=0.55 B=0.45
2017-10 geo=1: A=0.55 B=0.45
2017-10 geo=2: A=0.55 B=0.45
2017-11 geo=1: A=0.55 B=0.45
2017-11 geo=2: A=0.55 B=0.45
2017-12 geo=1: A=0.55 B=0.45
2017-12 geo=2: A=0.55 B=0.45

基于出现频率的SVM分类

1 个答案: