Pandas：从df中的列中的值中删除第一个和最后一个元素

Question

更新的解决方案：

我的数据'|'分隔了一些列，即它不是严格csv。我已将其导入为csv并尝试剥离特定列中额外的'|'值。我的数据如下：

    import pandas as pd
from io import StringIO



dfy = pd.read_csv('Thesis/CRSP/CampaignFin14/pacs14.txt', header=0)

#Replace '|' in cells with series.str methods
for col in dfy:
    if dfy[col].dtype == 'object':
        dfy[col] = dfy[col].str.replace('|', '')

        dfy.head()



  |2014|  |4111920141231643319|  |C00206136|  |N00029285|  1000  05/15/2014  \
0   2014  |4021120141205164809|  |C00307397|  |N00026722|  5000  10/22/2013   
1   2014  |4053020141213944220|  |C00009985|  |N00030676|     4  03/26/2014   
2   2014  |4063020141216281752|  |C00104299|  |N00032088|  1000  05/06/2014   
3   2014  |4061920141215566782|  |C00164145|  |N00034277|  2500  05/22/2014   
4   2014  |4102420141226480432|  |C00439216|  |N00036023|  1000  09/29/2014   

由于某种原因，循环没有取出|

以下有效，但我想一次完成所有列。

dfy['cycle'] = \
dfy['cycle'].map(lambda x: str(x)[:-1])
dfy.head()

    cycle   cid     amount  date    realcode    type    di  feccandid
0   |2014   |N00029285|     1000    05/15/2014  |E1600|     |24K|   |D|     |H8TX22107|
1   |2014   |N00026722|     5000    10/22/2013  |G4600|     |24K|   |D|     |H4TX28046|
2   |2014   |N00030676|     4   03/26/2014  |C2100|     |24Z|   |D|     |H0MO07113|

这是我使用.csv，sep=导入时的数据。

    cycle   cid     amount  date    realcode    type    di  feccandid
0   |2014|  |N00029285|     1000    05/15/2014  |E1600|     |24K|   |D|     |H8TX22107|
1   |2014|  |N00026722|     5000    10/22/2013  |G4600|     |24K|   |D|     |H4TX28046|
2   |2014|  |N00030676|     4   03/26/2014  |C2100|     |24Z|   |D|     |H0MO07113|

这就是.txt中的样子：

|2014|,|4111920141231643319|,|C00206136|,|N00029285|,1000,05/15/2014,|E1600|,|24K|,|D|,|H8TX22107|
|2014|,|4021120141205164809|,|C00307397|,|N00026722|,5000,10/22/2013,|G4600|,|24K|,|D|,|H4TX28046|
|2014|,|4053020141213944220|,|C00009985|,|N00030676|,4,03/26/2014,|C2100|,|24Z|,|D|,|H0MO07113|
|2014|,|4063020141216281752|,|C00104299|,|N00032088|,1000,05/06/2014,|F1100|,|24K|,|D|,|H0OH06189|
|2014|,|4061920141215566782|,|C00164145|,|N00034277|,2500,05/22/2014,|F3100|,|24K|,|D|,|H2NY22139|

以下是我rawdata

的链接

Answer 1

您可以预处理内存中的文件以从行中删除所有|并将其传递给Pandas。

import io
import pandas as pd

with open('Thesis/CRSP/CampaignFin14/pacs14.txt', 'r') as fi:
    content = ''
    for line in fi:
        content += line.replace('|', '')

block = io.StringIO(content)
dfy2 = pd.read_csv(block, skipinitialspace=True, delim_whitespace=True)

这里首先从带有io.StringIO()的字符串创建一个缓冲区，然后将其传递给作为第一个参数文件名或缓冲区接受的pd.read_csv。

Answer 2

继续阅读csv并使用Series str

等str.replace操作

import pandas as pd
from cStringIO import StringIO

# Fake csv text for example
textcsv = '''
cycle,cid,amount,date,realcode,type,di,feccandid
|2014|,|N00029285|,1000,05/15/2014,|E1600|,|24K|,|D|,|H8TX22107|
|2014|,|N00026722|,5000,10/22/2013,|G4600|,|24K|,|D|,|H4TX28046|
|2014|,|N00030676|,4   ,03/26/2014,|C2100|,|24Z|,|D|,|H0MO07113|
'''
# Read in fake csv
# normally you would use: dfy = pd.read_csv('/path/to/file.csv')
dfy = pd.read_csv(StringIO(textcsv))

# Replace '|' in cells with series.str methods
for col in dfy:
    if dfy[col].dtype == 'object':
        dfy[col] = dfy[col].str.replace('|', '')

print day

  cycle        cid  amount        date realcode type di  feccandid
0  2014  N00029285    1000  05/15/2014    E1600  24K  D  H8TX22107
1  2014  N00026722    5000  10/22/2013    G4600  24K  D  H4TX28046
2  2014  N00030676       4  03/26/2014    C2100  24Z  D  H0MO07113