Question

我有一个带有两个分隔符（;）和（,）的CSV，它看起来像这样：

vin;vorgangid;eventkm;D_8_lamsoni_w_time;D_8_lamsoni_w_value
V345578;295234545;13;-1000.0,-980.0;7.9921875,11.984375
V346670;329781064;13;-960.0,-940.0;7.9921875,11.984375

我想将其导入到pandas数据框中，（;）充当列分隔符，（,）作为list或{{1的分隔符使用array作为数据类型。到目前为止，我正在使用这种方法，但我确信那里有一些更容易的东西。

float

Answer 1

除了这里有更多特定熊猫的其他优秀答案之外，还应该注意到Python本身在字符串处理方面非常强大。您可以将';'替换为','放在StringIO对象中，并从那里正常工作：

In [8]: import pandas as pd

In [9]: from cStringIO import StringIO

In [10]: pd.read_csv(StringIO(''.join(l.replace(';', ',') for l in open('stuff.csv'))))
Out[10]: 
                   vin  vorgangid  eventkm  D_8_lamsoni_w_time  \
V345578 295234545   13    -1000.0   -980.0            7.992188   
V346670 329781064   13     -960.0   -940.0            7.992188   

                   D_8_lamsoni_w_value  
V345578 295234545            11.984375  
V346670 329781064            11.984375

Answer 2

首先使用;作为分隔符读取CSV：

df = pd.read_csv(filename, sep=';')

<强>更新

In [67]: num_cols = df.columns.difference(['vin','vorgangid','eventkm'])

In [68]: num_cols
Out[68]: Index(['D_8_lamsoni_w_time', 'D_8_lamsoni_w_value'], dtype='object')

In [69]: df[num_cols] = (df[num_cols].apply(lambda x: x.str.split(',', expand=True)
   ....:                                               .stack()
   ....:                                               .astype(float)
   ....:                                               .unstack()
   ....:                                               .values.tolist())
   ....:                )

In [70]: df
Out[70]:
       vin  vorgangid  eventkm D_8_lamsoni_w_time     D_8_lamsoni_w_value
0  V345578  295234545       13  [-1000.0, -980.0]  [7.9921875, 11.984375]
1  V346670  329781064       13   [-960.0, -940.0]  [7.9921875, 11.984375]

In [71]: type(df.loc[0, 'D_8_lamsoni_w_value'][0])
Out[71]: float

OLD回答：

现在我们可以将数字拆分成＆＃34;数字＆＃34;列：

In [20]: df[['D_8_lamsoni_w_time',  'D_8_lamsoni_w_value']] = \
    df[['D_8_lamsoni_w_time',  'D_8_lamsoni_w_value']].apply(lambda x: x.str.split(','))
In [21]: df
Out[21]:
       vin  vorgangid  eventkm D_8_lamsoni_w_time     D_8_lamsoni_w_value
0  V345578  295234545       13  [-1000.0, -980.0]  [7.9921875, 11.984375]
1  V346670  329781064       13   [-960.0, -940.0]  [7.9921875, 11.984375]

Answer 3

您可以在read_csv中使用参数converters并定义用于拆分的自定义函数：

def f(x):
    return [float(i) for i in x.split(',')]

#after testing replace io.StringIO(temp) to filename
df = pd.read_csv(io.StringIO(temp), 
                 sep=";", 
                 converters={'D_8_lamsoni_w_time':f, 'D_8_lamsoni_w_value':f})
print (df)
       vin  vorgangid  eventkm D_8_lamsoni_w_time     D_8_lamsoni_w_value
0  V345578  295234545       13  [-1000.0, -980.0]  [7.9921875, 11.984375]
1  V346670  329781064       13   [-960.0, -940.0]  [7.9921875, 11.984375]

在NaN和4.列中使用5.的另一个解决方案：

您可以将read_csv与分隔符;一起使用，然后将str.split应用于iloc选择的4.和5.列并转换每个值在list到float：

import pandas as pd
import numpy as np
import io

temp=u"""vin;vorgangid;eventkm;D_8_lamsoni_w_time;D_8_lamsoni_w_value
V345578;295234545;13;-1000.0,-980.0;7.9921875,11.984375
V346670;329781064;13;-960.0,-940.0;7.9921875,11.984375"""
#after testing replace io.StringIO(temp) to filename
df = pd.read_csv(io.StringIO(temp), sep=";")

print (df)
       vin  vorgangid  eventkm D_8_lamsoni_w_time  D_8_lamsoni_w_value
0  V345578  295234545       13     -1000.0,-980.0  7.9921875,11.984375
1  V346670  329781064       13      -960.0,-940.0  7.9921875,11.984375

#split 4.th and 5th column and convert to numpy array
df.iloc[:,3] = df.iloc[:,3].str.split(',').apply(lambda x: [float(i) for i in x])
df.iloc[:,4] = df.iloc[:,4].str.split(',').apply(lambda x: [float(i) for i in x])
print (df)
       vin  vorgangid  eventkm D_8_lamsoni_w_time     D_8_lamsoni_w_value
0  V345578  295234545       13  [-1000.0, -980.0]  [7.9921875, 11.984375]
1  V346670  329781064       13   [-960.0, -940.0]  [7.9921875, 11.984375]

如果需要numpy arrays而不是lists：

#split 4.th and 5th column and convert to numpy array
df.iloc[:,3] = df.iloc[:,3].str.split(',').apply(lambda x: np.array([float(i) for i in x]))
df.iloc[:,4] = df.iloc[:,4].str.split(',').apply(lambda x: np.array([float(i) for i in x]))
print (df)
       vin  vorgangid  eventkm D_8_lamsoni_w_time     D_8_lamsoni_w_value
0  V345578  295234545       13  [-1000.0, -980.0]  [7.9921875, 11.984375]
1  V346670  329781064       13   [-960.0, -940.0]  [7.9921875, 11.984375]

print (type(df.iloc[0,3]))
<class 'numpy.ndarray'>

我尝试改进你的解决方案：

a=0;
csv_import=pd.read_csv(folder+FileName, ';')
for col in csv_import.columns:
    a += 1
    if type(csv_import.ix[0, col])== str and a>3:
        # string to list of strings
        csv_import[col]=csv_import[col].apply(lambda x: [float(y) for y in x.split(',')])

使用两个分隔符将CSV导入到pandas

3 个答案: