熊猫新手,如果解决方案非常明显,那就很抱歉。
我有一个数据框(见下文),其中包含不同的电影场景和该电影场景的环境
import pandas as pd
data = [{'movie' : 'movie_X', 'scene' : '1', 'environment' : 'home'},
{'movie' : 'movie_X', 'scene' : '2', 'environment' : 'car'},
{'movie' : 'movie_X', 'scene' : '3', 'environment' : 'home'},
{'movie' : 'movie_Y', 'scene' : '1', 'environment' : 'home'},
{'movie' : 'movie_Y', 'scene' : '2', 'environment' : 'office'},
{'movie' : 'movie_Z', 'scene' : '1', 'environment' : 'boat'},
{'movie' : 'movie_Z', 'scene' : '2', 'environment' : 'beach'},
{'movie' : 'movie_Z', 'scene' : '3', 'environment' : 'home' }]
myDF = pd.DataFrame(data)
在这种情况下,电影有多种类型。我有一本字典(下面)描述了每部电影属于哪种类型
genreDict = {'movie_X' : ['romance', 'action'],
'movie_Y' : ['comedy', 'romance', 'action'],
'movie_Z' : ['horror', 'thriller', 'romance']}
我想通过这本字典对myDF进行分组,特别是能够告诉某个特定类型的特定环境出现的次数(例如,在类型恐怖中,' boat'被计算过一次,'海滩'被计算一次,' home'被计算一次)。什么是最好和最有效的方式来解决这个问题?我已经尝试将字典映射到数据帧,然后按列表分组:
myDF['genres'] = myDF['movie'].map(genreDict)
返回:
movie scene environment genres
0 movie_X 1 home [romance, action]
1 movie_X 2 car [romance, action]
2 movie_X 3 home [romance, action]
3 movie_Y 1 home [comedy, romance, action]
4 movie_Y 2 office [comedy, romance, action]
5 movie_Z 1 boat [horror, thriller, romance]
6 movie_Z 2 beach [horror, thriller, romance]
7 movie_Z 3 home [horror, thriller, romance]
然而,我得到一个错误,说列表不可用。希望你们都能提供帮助:)
答案 0 :(得分:3)
非标量对象通常会导致熊猫出现问题。除此之外,您还需要整理数据,以便下一步更容易(表格结构上的主要操作通常在整洁的数据集上定义)。你需要一个数据集,你不能连续列出所有类型,但每个类型都有自己的行。
这是实现这一目标的可能方法之一:
genre_df = pd.DataFrame(myDF['movie'].map(genreDict).tolist())
df = myDF.join(genre_df.stack().rename('genre').reset_index(level=1, drop=True))
df
Out:
environment movie scene genre
0 home movie_X 1 romance
0 home movie_X 1 action
1 car movie_X 2 romance
1 car movie_X 2 action
2 home movie_X 3 romance
2 home movie_X 3 action
3 home movie_Y 1 comedy
3 home movie_Y 1 romance
3 home movie_Y 1 action
4 office movie_Y 2 comedy
4 office movie_Y 2 romance
4 office movie_Y 2 action
5 boat movie_Z 1 horror
5 boat movie_Z 1 thriller
5 boat movie_Z 1 romance
6 beach movie_Z 2 horror
6 beach movie_Z 2 thriller
6 beach movie_Z 2 romance
7 home movie_Z 3 horror
7 home movie_Z 3 thriller
7 home movie_Z 3 romance
一旦有了这样的结构,就可以更轻松地对数据进行分组或交叉制表:
df.groupby('genre').size()
Out:
genre
action 5
comedy 2
horror 3
romance 8
thriller 3
dtype: int64
pd.crosstab(df['genre'], df['environment'])
Out:
environment beach boat car home office
genre
action 0 0 1 3 1
comedy 0 0 0 1 1
horror 1 1 0 1 0
romance 1 1 1 4 1
thriller 1 1 0 1 0
这是Hadley Wickham的精彩读物:Tidy Data。
答案 1 :(得分:1)
如果更快的数据框使用numpy
来lists
numpy.repeat
,numpy.concatenate
和Index.values
重复行:
#get length of lists in column genres
l = myDF['genres'].str.len()
#convert column to numpy array
vals = myDF['genres'].values
#repeat index by lenghts
idx = np.repeat(myDF.index, l)
#expand rows by duplicated index values
myDF = myDF.loc[idx]
#flattening lists column
myDF['genres'] = np.concatenate(vals)
#default monotonic index (0,1,2...)
myDF = myDF.reset_index(drop=True)
print (myDF)
environment movie scene genres
0 home movie_X 1 romance
1 home movie_X 1 action
2 car movie_X 2 romance
3 car movie_X 2 action
4 home movie_X 3 romance
5 home movie_X 3 action
6 home movie_Y 1 comedy
7 home movie_Y 1 romance
8 home movie_Y 1 action
9 office movie_Y 2 comedy
10 office movie_Y 2 romance
11 office movie_Y 2 action
12 boat movie_Z 1 horror
13 boat movie_Z 1 thriller
14 boat movie_Z 1 romance
15 beach movie_Z 2 horror
16 beach movie_Z 2 thriller
17 beach movie_Z 2 romance
18 home movie_Z 3 horror
19 home movie_Z 3 thriller
20 home movie_Z 3 romance
df1 = df.groupby(['genres','environment']).size().reset_index(name='count')
print (df1)
genres environment count
0 action car 1
1 action home 3
2 action office 1
3 comedy home 1
4 comedy office 1
5 horror beach 1
6 horror boat 1
7 horror home 1
8 romance beach 1
9 romance boat 1
10 romance car 1
11 romance home 4
12 romance office 1
13 thriller beach 1
14 thriller boat 1
15 thriller home 1