Question

上次我在本网站上询问有关使用glob.glob()使用Python批量处理文件夹中的csv文件时，我得到了帮助。我这次尝试使用它来转置文件夹中的所有csv文件。下面的脚本只处理最后一个文件并停止。我做错了什么？

import csv
import os
import glob

directory = raw_input ("INPUT Folder")
output = raw_input("OUTPUT Folder:")
in_files = os.path.join(directory, '*.csv')

for in_file in glob.glob(in_files):
    with open(in_file) as input_file:
        reader = csv.reader(input_file)
        cols = []
        for row in reader:
            cols.append(row)
            filename = os.path.splitext(os.path.basename(in_file))[0] + '.csv'

with open (os.path.join(output, filename), 'wb') as output_file:
    writer = csv.writer(output_file)
    for i in range(len(max(cols, key=len))):
        writer.writerow ([(c[i] if i<len(c) else '') for c in cols])

Answer 1

你需要缩进＆＃34;输出＆＃34;代码的一部分，以便它为for in_file循环的每次迭代运行一次：

import csv
import os
import glob

directory = raw_input ("INPUT Folder")
output = raw_input("OUTPUT Folder:")
in_files = os.path.join(directory, '*.csv')

for in_file in glob.glob(in_files):
    with open(in_file) as input_file:
        reader = csv.reader(input_file)
        cols = []
        for row in reader:
            cols.append(row)

    # "outdent" this code so it only needs to run once for each in_file
    filename = os.path.splitext(os.path.basename(in_file))[0] + '.csv'

    # Indent this to the same level as the rest of the "for in_file" loop!
    with open (os.path.join(output, filename), 'wb') as output_file:
        writer = csv.writer(output_file)
        for i in range(len(max(cols, key=len))):
            writer.writerow ([(c[i] if i<len(c) else '') for c in cols])

在您的版本中，代码仅在for in_file循环完成后运行一次，因此仅输出从该循环的最后一次迭代中遗留的cols个数据。

我还＆＃34;缩进＆＃34; filename = ...级别的for in_file语句，因为每个in_file只需执行一次，而row每in_file只需执行一次。< / p>

Answer 2

使用pandas进行数据操作可以获得很多里程：

import os
import pandas as pd

for filename in os.listdir('.'):
    # We save an augmented filename later, 
    # so using splitext is useful for more
    # than just checking the extension.
    prefix, ext = os.path.splitext(filename)
    if ext.lower() != '.csv':
        continue
    # Load the data into a dataframe
    df = pd.DataFrame.from_csv(filename, 
                               header=None, 
                               index_col=None, 
                               parse_dates=False)
    # Transpose is easy, but you could do TONS
    # of data processing here. pandas is awesome.
    df_transposed = df.T
    # Save to a new file with an augmented name 
    df_transposed.to_csv(prefix+'_T'+ext, header=True, index=False)

如果你需要挖掘子文件夹，os.walk版本没有太大的不同。

Answer 3

这是一个有效的工作：

不得不谷歌一小时，但在python33上工作和测试

import csv
import os
import glob

directory = 'C:\Python33\csv'
output = 'C:\Python33\csv2'
in_files = os.path.join(directory, '*.csv')

for in_file in glob.glob(in_files):
    with open(in_file) as input_file:
        reader = csv.reader(input_file)
        cols = []
        for row in reader:
            cols.append(row)

    # "outdent" this code so it only needs to run once for each in_file
    filename = os.path.splitext(os.path.basename(in_file))[0] + '.csv'

    # Indent this to the same level as the rest of the "for in_file" loop!
    with open (os.path.join(output, filename), 'w') as output_file:
        writer = csv.writer(output_file)
        for i in range(len(max(cols, key=len))):
            writer.writerow ([(c[i] if i<len(c) else '') for c in cols])

Answer 4

in_files只返回该格式的单个结果。尝试返回一个列表：

in_files = [f for f in os.listdir(directory) if f.endswith('.csv')]

转置文件夹中的所有csv文件

4 个答案: