Question

我正在尝试合并文件夹中的多个csv文件。

它们看起来像这样（实际上有两个以上的df）：

DF1

DF2

LCC  acres_2
2    4
3    2
4    40
5    6
6    7

我想将所有数据帧放入一个列表中，然后将它们与reduce合并。要做到这一点，他们需要具有相同的索引。

我正在尝试这段代码：

combined = []
reindex = [2,3,4,5,6]

folder = r'C:\path_to_files'

for f in os.listdir(folder):

    #read each file
    df = pd.read_csv(os.path.join(folder,f))

    #check for duplicates - returns empty lists
    print df[df.index.duplicated()]

    #reindex
    df.set_index([df.columns[0]], inplace=True)
    df=df.reindex(reindex, fill_value=0)

    #append
    combined.append(df)


#merge on 'LCC' column
final = reduce(lambda left, right: pd.merge(left, right, on=['LCC'], how='outer'), combined)

但仍会返回：

Traceback (most recent call last):

  File "<ipython-input-31-45f925f6d48d>", line 9, in <module>
    df=df.reindex(reindex, fill_value=0)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2741, in reindex
    **kwargs)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\generic.py", line 2229, in reindex
    fill_value, copy).__finalize__(self)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2687, in _reindex_axes
    fill_value, limit, tolerance)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\frame.py", line 2698, in _reindex_index
    allow_dups=False)

File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\generic.py", line 2341, in _reindex_with_indexers
    copy=copy)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\core\internals.py", line 3586, in reindex_indexer
    self.axes[axis]._can_reindex(indexer)

  File "C:\Users\spotter\AppData\Local\Continuum\Anaconda2_2\lib\site-packages\pandas\indexes\base.py", line 2293, in _can_reindex
    raise ValueError("cannot reindex from a duplicate axis")

ValueError: cannot reindex from a duplicate axis

Answer 1

在将第一列设置为索引后，您需要检查索引的重复项。

#set index by first column
df.set_index([df.columns[0]], inplace=True)

#check for duplicates - returns NO empty lists
print df[df.index.duplicated()]

#reindex
df=df.reindex(reindex, fill_value=0)

或者检查第一列中的重复项而不是索引，同时参数keep=False返回所有重复项（如有必要）：

#check duplicates in first column
print df[df.iloc[:, 0].duplicated(keep=False)]

#set index + reindex
df.set_index([df.columns[0]], inplace=True)
df=df.reindex(reindex, fill_value=0)

无法重新索引重复的轴

1 个答案: