Question

csv文件已发送给我/我无法重新分隔列

239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00

我替换了字符串字母以涵盖敏感信息，但问题很明显。

这是我的csv中的一个示例“问题行”。它应按如下方式分为8列：

col1: 239845723
col2: 28374
col3: 2384234
col4: AEVNE EFU 5 GN OR WNV
col5: Owinv Vnwo Badvw 5 VIN
col6: Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).
col7: 2011-07-13 00:00:00
col8: 2011-07-13 00:00:00

如您所见，第6列是出现问题的地方，因为字符串中有逗号，导致pandas分隔并错误地创建新列。我怎么解决这个问题？我认为正则表达式会有所帮助，也许可以通过以下设置。任何帮助表示赞赏！

    csvfile = open(filetrace) 
    reader = csv.reader(csvfile)
    new_list=[]
    for line in reader:
        for i in line:
            #not sure

Answer 1

无需前往正则表达式，请使用分隔符＆＃39;来读取csv，您可以提取最后两个日期并将其存储在列表中。然后使用''填充日期，然后加入所需的列并删除其余列。实施例

如果您有csv文件：

239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00
239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE.).,2011-07-13 00:00:00,2011-07-13 00:00:00
239845723,28374,2384234,AEVNE EFU 5 GN OR WNV,Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).,2011-07-13 00:00:00,2011-07-13 00:00:00

然后

df = pd.read_csv('good.txt',delimiter=',',header=None)
# Get the Dates from all the DataFrame 
dates = [[item] for i in df.values for item in i if '2011-' in str(item)]
# Merge two Dates for each column
dates = pd.DataFrame([x+y for x,y in zip(dates[0::2], dates[1::2])])
# Remove the dates present 
df = df.replace({'2011-': np.nan}, regex=True).replace(np.nan,'')

#Get the columns you want to merge 
cols = df.columns[4:]
# Merge the columns 
df[4] = df[cols].astype(str).apply(lambda x: ','.join(x), axis=1)
df[4] = df[4].replace('\,+$', '',regex=True)
#Drop the Columns 
df = df.drop(df.columns[5:],axis=1)
#Concat the dates 
df = pd.concat([df,dates],axis=1)

输出：print（df）

           0      1        2                      3  \
0  239845723  28374  2384234  AEVNE EFU 5 GN OR WNV   
1  239845723  28374  2384234  AEVNE EFU 5 GN OR WNV   
2  239845723  28374  2384234  AEVNE EFU 5 GN OR WNV   

                                                   4                    0  \
0  Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera ...  2011-07-13 00:00:00   
1  Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera ...  2011-07-13 00:00:00   
2  Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse...  2011-07-13 00:00:00   

                     1  
0  2011-07-13 00:00:00  
1  2011-07-13 00:00:00  
2  2011-07-13 00:00:00

第4栏的输出：

['Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee, 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).',

 'Owinv Vnwo Badvw 5 VIN,Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd bdfwd dsf (sdv seves 4-6), sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE.).',

'Owinv Vnwo Badvw 5 VIN sebsbe sve(sevsev esvse 7-10) fsesef fesevsesv PaVvin (1 evesve vEV VEWee 2 for WVEee VEWE. paper tuff as sWEFEWoon as VEWeew.).']

如果要更改列索引

df.columns = [i for i in range(df.shape[1])]

希望有所帮助

Answer 2

因此，在不知道文件或数据的具体情况的情况下，如果数据一致（并且在第6列末尾有句点），我可以提供可以工作的正则表达式解决方案。我们可以不使用csv模块而只使用正则表达式模块。

import re

# make the regex pattern here
pattern = r"([\d\.]*),([\d\.]*),([\d\.]*),([^,]*),([^,]*),(.*\.?),([\d\-\s:]*),([\d\-\s:]*)"

# open the file with 'with' so you don't have to worry about closing it
with open(filetrace) as f:
    for line in f:  # iterate through the lines
        values = re.findall(pattern, line)[0]  # re.findall returns a list 
                                               # literal of a tuple
        # record the values somewhere

values这里是一个包含原始csv中每个列的值的8元组，只需使用/存储它们即可。

祝你好运！

Answer 3

由于您确切知道需要多少列，并且只有一个有问题的列，因此我们可以将前几个从左侧拆分而从右侧拆分。换句话说，您不需要regex

将文件读入单个字符串

csvfile = open(filetrace).read()

制作pd.Series

s = pd.Series(csvfile.split('\n'))

拆分s并将其限制为5个拆分，应为6列

df = s.str.split(',', 5, expand=True)

现在将右侧限制为2个分割

df = df.iloc[:, :-1].join(df.iloc[-1].str.rsplit(',', 2, expand=True))

从s

开始的另一种方式

left = s.str.split(',', 5)
right = left.str[-1].str.rsplit(',', 2)

df = pd.DataFrame(left.str[:-1].add(right).tolist())

我跑了这个并采用转置使其更容易在屏幕上阅读

df.T



                                                   0
0                                          239845723
1                                              28374
2                                            2384234
3                              AEVNE EFU 5 GN OR WNV
4                             Owinv Vnwo Badvw 5 VIN
5  Ginq 2 jnwve wef evera wve 6 vwe as fgsb bfd b...
6                                2011-07-13 00:00:00
7                                2011-07-13 00:00:00

如何在导入带有额外逗号的pandas的csv文件时使用正则表达式作为分隔符？

3 个答案: