Question

对于50个csv文件，我只需要为一个特定列计算均值和标准差。然后，我需要创建一个包含50行和2列的数组，以便在每一行中都有一个csv文件的std和均值。我陷入试图获得第一个csv的df平均值和标准差的水平。

这就是我所拥有的：

import numpy as np
import pandas as pd
import glob
i=0
path ="C:\\Users\\sharon\\Desktop\\mathematical finance\\sadna"
all_files = glob.glob(path + "/*.csv")
arr = np.zeros((50,2))
for filename in all_files:
    df = pd.read_csv(filename,encoding="utf-8")
    df=df.loc[2:470,'Unnamed: 3']
    Mean=df.mean() #DOES NOT WORK
    Std=df.std(axis=1)  # What?...
    arr[i,:]=(Mean,Std)

编辑：

使用以下代码解决问题：

import numpy as np
import pandas as pd
import glob
path ="C:\\Users\\sharon\\Desktop\\mathematical finance\\sadna"
all_files = glob.glob(path + "/*.csv")
df_list = [(pd.read_csv(f, encoding = "utf-8", header = None,usecols = [3], nrows = 470, ).assign(filename = f)) for f in all_files]
final_df = pd.concat(df_list)
final_df[3]= final_df[3].apply(pd.to_numeric, errors='coerce')
agg_df = final_df.groupby(['filename']).agg(['mean', 'std'])

Answer 1

请考虑使用list comprehension在与concat串联在一起的所有CSV文件中构建单个数据帧。确保使用read_csv所需的参数。然后汇总所需的统计信息。最后，使用to_numpy将数据帧值转换为numpy数组：

path = "C:\\Users\\sharon\\Desktop\\mathematical finance\\sadna"
all_files = glob.glob(path + "/*.csv")

# SPECIFY NO HEADERS, SPECIFIC COLUMN AND NUMBER OF ROWS
df_list = [(pd.read_csv(f, encoding = "utf-8", 
                        header = None,
                        usecols = [4], 
                        nrows = 469)
              .assign(filename = f)
           ) for f in all_files]

# COMPILE LARGE DATA FRAME
final_df = pd.concat(df_list, ignore_index=True)

# AGGREGATE BY filename
agg_df = final_df.groupby(['filename']).agg(['mean', 'std'])

# CONVERT TO NUMPY ARRAY
arr = agg_df.to_numpy()

数据框列的均值和标准差

1 个答案: