假设我有一个包含此数据的表(数据帧):
| user | food |
|:--------:|:-------------:|
| 'A' | 'meat' |
| 'A' | 'carrot' |
| 'A' | 'candy' |
| 'B' | 'meat' |
| 'B' | 'carrot' |
| 'C' | 'meat' |
| 'C' | 'carrot' |
代码:
df = pd.DataFrame({
"user":["A", "A", "A", "B", "B", "C", "C"],
"food":['meat', 'carrot', 'candy', 'meat', 'carrot', 'meat', 'carrot']
})
我想要建立的是一张桌子,每张食物告诉我拥有它们的用户数量:
| food 1 | food 2 | num users |
|:----------:|:-------------:|:----------:|
| 'meat' | 'carrot' | 3 |
| 'meat' | 'candy' | 1 |
| 'carrot' | 'candy' | 1 |
有办法做到这一点吗?
答案 0 :(得分:3)
您可以先使用get_dummies
:
df = pd.get_dummies(df.set_index('user'), prefix='', prefix_sep='').max(level=0)
print (df)
candy carrot meat
user
A 1 1 1
B 0 1 1
C 0 1 1
然后按list comprehension
计算:
from itertools import combinations
L = [(x[0], x[1],(df[list(x)] == 1).all(1).sum()) for x in list(combinations(df.columns, 2))]
print (L)
[('candy', 'carrot', 1), ('candy', 'meat', 1), ('carrot', 'meat', 3)]
df = pd.DataFrame(L, columns=['food 1','food 2','num users'])
print (df)
food 1 food 2 num users
0 candy carrot 1
1 candy meat 1
2 carrot meat 3
答案 1 :(得分:1)
你可以试试这个:
food_pairs = [("meat", "carrot"), ("meat", "candy")]
food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}
out = pd.DataFrame(
((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
columns=["food1", "food2", "num users"]
)
超过1000次试验的平均运行时间为0.00256s
。
测试可伸缩性代码:
import itertools
import math
import pandas as pd
from random import shuffle
from timeit import time
SIZE_OF_PAIRS = 2
NUM_FOODS = 50
NUM_USERS = 1000
NUM_RECORDS = 100000
foods = (list(range(NUM_FOODS)) * (math.ceil(NUM_RECORDS/NUM_FOODS)))[:NUM_RECORDS]
users = (list(range(NUM_USERS)) * (math.ceil(NUM_RECORDS/NUM_USERS)))[:NUM_RECORDS]
shuffle(foods)
shuffle(users)
df = pd.DataFrame({"user": users, "food": foods})
food_pairs = pd.Series([*itertools.combinations(df.food.unique(), SIZE_OF_PAIRS)])
start = time.time()
food_to_users = {food: set(df.user[df.food == food].unique()) for food in df.food.unique()}
out = pd.DataFrame(
((*pair, len(set.intersection(*(food_to_users[food] for food in pair)))) for pair in food_pairs),
columns=[*["food" + str(i) for i in range(SIZE_OF_PAIRS)], "num users"]
)
print("Time taken: {}s".format(time.time() - start))