Question

我有一个带有某些值的列（ip）的Pandas Dataframe和另一个不在此DataFrame中的Pandas Series以及这些值的集合。我想在DataFrame中创建一个列，如果给定的行在我的Pandas系列中有ip（black_ip），则该列为。

import pandas as pd

dict = {'ip': {0: 103022, 1: 114221, 2: 47902, 3: 23550, 4: 84644}, 'os': {0: 23, 1: 19, 2: 17, 3: 13, 4: 19}}

df = pd.DataFrame(dict)

df
     ip  os
0  103022  23
1  114221  19
2   47902  17
3   23550  13
4   84644  19

blacklist = pd.Series([103022, 23550])

blacklist

0    103022
1     23550

我的问题是：如何在df中创建一个新列，以便在黑名单中给定ip时显示1，否则为零？

对不起，如果这太愚蠢了，我还是编程新手。非常感谢提前！

Answer 1

isin使用astype：

df['new'] = df['ip'].isin(blacklist).astype(np.int8)

也可以将列转换为categorical s：

df['new'] = pd.Categorical(df['ip'].isin(blacklist).astype(np.int8))

print (df)
       ip  os  new
0  103022  23    1
1  114221  19    0
2   47902  17    0
3   23550  13    1
4   84644  19    0

感兴趣的是大DataFrame转换为Categorical而不是节省内存：

df = pd.concat([df] * 10000, ignore_index=True)

df['new1'] = pd.Categorical(df['ip'].isin(blacklist).astype(np.int8))
df['new2'] = df['ip'].isin(blacklist).astype(np.int8)
df['new3'] = df['ip'].isin(blacklist)
print (df.memory_usage())
Index        80
ip       400000
os       400000
new1      50096
new2      50000
new3      50000
dtype: int64

<强>计时：

np.random.seed(4545)

N = 10000
df = pd.DataFrame(np.random.randint(1000,size=N), columns=['ip'])
print (len(df))
10000

blacklist = pd.Series(np.random.randint(500,size=int(N/100)))
print (len(blacklist))
100

In [320]: %timeit df['ip'].isin(blacklist).astype(np.int8)
465 µs ± 21.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [321]: %timeit pd.Categorical(df['ip'].isin(blacklist).astype(np.int8))
915 µs ± 49.9 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [322]: %timeit pd.Categorical(df['ip'], categories = blacklist.unique()).notnull().astype(int)
1.59 ms ± 20.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [323]: %timeit df['new_column'] = [1 if x in blacklist.values else 0 for x in df.ip]
81.8 ms ± 2.72 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Answer 2

缓慢但简单易读的方法：

另一种方法是使用list comprehension创建新列，如果ip值在blacklist，则设置为1，否则为0 < / p>

df['new_column'] = [1 if x in blacklist.values else 0 for x in df.ip]

>>> df
       ip  os  new_column
0  103022  23           1
1  114221  19           0
2   47902  17           0
3   23550  13           1
4   84644  19           0

编辑：在Categorical 上构建更快的方法：如果您想要最大化速度，以下内容会非常快，但速度不如.isin非分类方法。它建立在@jezrael建议的pd.Categorical的基础上，但利用它分配类别的能力：

df['new_column'] = pd.Categorical(df['ip'], 
          categories = blacklist.unique()).notnull().astype(int)

<强>时序：

import numpy as np
import pandas as pd
np.random.seed(4545)
N = 10000
df = pd.DataFrame(np.random.randint(1000,size=N), columns=['ip'])
blacklist = pd.Series(np.random.randint(500,size=int(N/100)))

%timeit df['ip'].isin(blacklist).astype(np.int8)
# 453 µs ± 8.81 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit pd.Categorical(df['ip'].isin(blacklist).astype(np.int8))
# 892 µs ± 17.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit pd.Categorical(df['ip'], categories = \
              blacklist.unique()).notnull().astype(int)
# 565 µs ± 32.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

如果列匹配某些值，如何在Pandas Dataframe中创建虚拟变量？

2 个答案: