我有一个excel电子表格,我经常更新(每天2-3次)。此更新需要运行索引匹配以从另一个电子表格中的表中提取值,并将它们写入第一个中的列。这些值会覆盖旧值,而不会创建新列。
我想使用pandas自动执行此过程(和xlwings将数据写入电子表格,但我对该部分没有任何问题)。第一步是用pandas复制excel的INDEXMATCH()。总的来说,函数应该:
获取要作为索引的列的字符串标题,要写入的列以及包含用于匹配读取和放大的值的列的参数。写专栏
迭代写入列;在每次迭代中,在读取列中搜索其对应的匹配列值与写入列的匹配列值匹配的值
如果没有匹配值,请写入NaN或' #N / A'到数据帧(区分0和不匹配很重要)
我希望在pandas中有一个原生的vlookup / indexmatch功能,但我唯一能找到的就是加入或合并数据帧,这不是我想要做的事情 - 我想要覆盖个人数据框中的值,并以任意索引顺序执行。
我设法让它使用特定于脚本的功能非常难看,但我认为尝试将该功能用于其他用途会很有用。经过一些清理和重写后,我得到了以下内容:
##Index Match in Python with pandas
#Remember that dataframes start at 0, excel starts at 1
#This only works if both DFs have the same indices (integers, strings, whatever)
import numpy as np
import pandas as pd
#sample dataframes
d = {'Match Column' : [0.,1.,2.,3.,4.,7.,'string'],
'Read Column' : ['zero','one','two','three','four','seven','string']}
dfRead = pd.DataFrame(d)
d2 = {'Match Column' : [0.,1.,2.,3.,4.,5.,6.,7.,'8'],
'Write Column' : [0,0,0,0,0,0,0,0,'0']}
dfWrite = pd.DataFrame(d2)
#test arguments
ReadColumn = 'Read Column'
WriteColumn = 'Write Column'
ReadMatchColumn = 'Match Column'
WriteMatchColumn = 'Match Column'
def indexmatch(dfRead, dfWrite, ReadColumn, WriteColumn, ReadMatchColumn, WriteMatchColumn, skiprows=0):
#convert the string inputs to a column number for each dataframe
RCNum = np.where(dfRead.columns == ReadColumn)[0][0]
WCNum = np.where(dfWrite.columns == WriteColumn)[0][0]
RMCNum = np.where(dfRead.columns == ReadMatchColumn)[0][0]
WMCNum = np.where(dfWrite.columns == WriteMatchColumn)[0][0]
for i in range(skiprows,len(dfWrite.index),1):
match = dfWrite.loc[dfWrite.index[i]][WMCNum] #the value we're using to match the columns
try:
matchind = dfRead.index[np.where(dfRead[ReadMatchColumn] == match)[0][0]]
value = dfRead.fillna('#N/A').loc[matchind][RCNum] #replaces DF NaN values with excel's #N/A, optional method
dfWrite.set_value(dfWrite.index[i],WriteColumn,value)
except KeyError:
dfWrite.set_value(dfWrite.index[i],WriteColumn,np.nan) #if there is no match, write NaN to the 'cell'
except IndexError:
dfWrite.set_value(dfWrite.index[i],WriteColumn,np.nan)
这样做有效,但它并不漂亮,并且它并不能说明何时将列与另一个数据帧的索引匹配(例如,将数据帧与数据透视表匹配)数据帧)。
是否有更强大,更简洁的方法呢?
根据要求,预期的输入和输出:
In [2]: dfRead
Out[2]:
Match Column Read Column
0 0 zero
1 1 one
2 2 two
3 3 three
4 4 four
5 7 seven
6 string string
In [3]: dfWrite
Out[3]:
Match Column Write Column
0 0 0
1 1 0
2 2 0
3 3 0
4 4 0
5 5 0
6 6 0
7 7 0
8 8 0
In [4]: indexmatch(dfRead, dfWrite, 'Read Column', 'Write Column', 'Match Column', 'Match Column')
In [5]: dfWrite
Out[7]:
Match Column Write Column
0 0 zero
1 1 one
2 2 two
3 3 three
4 4 four
5 5 NaN
6 6 NaN
7 7 seven
8 8 NaN
答案 0 :(得分:1)
pd.Series.map
将把一个系列视为一个参数,将它视为输入带有索引作为键的字典。
在这里应用,看起来像
dfWrite['Write Column'] = dfWrite['Match Column'].map(dfRead.set_index('Match Column')['Read Column'])
dfWrite
Out[409]:
Match Column Write Column
0 0 zero
1 1 one
2 2 two
3 3 three
4 4 four
5 5 NaN
6 6 NaN
7 7 seven
8 8 NaN
给予相同的输出
indexmatch(dfRead, dfWrite, 'Read Column', 'Write Column', 'Match Column', 'Match Column')
dfWrite
Out[413]:
Match Column Write Column
0 0 zero
1 1 one
2 2 two
3 3 three
4 4 four
5 5 NaN
6 6 NaN
7 7 seven
8 8 NaN
要匹配dfRead
的索引,请跳过.set_index(...)
步骤。要匹配dfWrite
的索引,请将dfWrite['Match Column'].map
替换为dfWrite.index.to_series().map
答案 1 :(得分:0)
您也可以使用merge
功能:
dfWrite = pd.merge(left=dfWrite.ix[:,['Match Column']], right=dfRead, on='Match Column', how='left')
dfWrite.rename(columns={'Read Column':'Write Column'}, inplace=True)