Question

我有url，其中包含我要加载到pandas dataframe中的文本文件。但是在顶部有一些元数据，我在解析时无法跳过它并返回错误。

ParserError: Error tokenizing data. C error: Expected 1 fields in line 3, saw 2

这是我的代码：

import pandas as pd
data = pd.read_csv('https://fred.stlouisfed.org/data/PERMIT.txt')

当顶部没有元数据时，此代码通常适用于我。如何在加载时跳过元数据？

txt文件的开头如下：

Title:               New Private Housing Units Authorized by Building Permits
Series ID:           PERMIT
Source:              U.S. Bureau of the Census, U.S. Department of Housing and Urban Development
Release:             New Residential Construction
Seasonal Adjustment: Seasonally Adjusted Annual Rate
Frequency:           Monthly
Units:               Thousands of Units
Date Range:          1960-01-01 to 2018-03-01
Last Updated:        2018-04-24 7:01 AM CDT
Notes:               Starting with the 2005-02-16 release, the series reflects an increase
                     in the universe of permit-issuing places from 19,000 to 20,000 places.

DATE        VALUE
1960-01-01   1092
1960-02-01   1088
1960-03-01    955
1960-04-01   1016
1960-05-01   1052
1960-06-01    958
1960-07-01    999
1960-08-01    994

Answer 1

使用skiprows参数跳过元数据。在您的情况下，您有12行：

number = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20] 
godlike = []

def total_num(num):
    for x in num:
        if x % 2 == 0:
           godlike.append(x)
    print(godlike)

if __name__ == "__main__":
    total_num(number)

或者，告诉data = pd.read_csv('https://fred.stlouisfed.org/data/PERMIT.txt', skiprows=12, sep='\s+') >>> data.head() DATE VALUE 0 1960-01-01 1092 1 1960-02-01 1088 2 1960-03-01 955 3 1960-04-01 1016 4 1960-05-01 1052标题所在的read_csv参数（第11行）：

header

如果您不知道要跳过多少行，您可以实施this answer中使用的策略

Answer 2

如果您想保留标题：

import pandas as pd
from pandas.io.common import StringIO as sio
import requests as req

url = 'https://fred.stlouisfed.org/data/PERMIT.txt'
res = req.request('get', url).content.decode()

h, b = res.replace('\r', '').split('\n\n')

s = pd.read_fwf(
    sio(h.replace(':', '')),
    header=None,
    names=['key', 'value']
).ffill().groupby('key').value.apply('\n'.join)

df = pd.read_csv(sio(b), delim_whitespace=True)

然后看看我们的便利工作

print(s.head(), df.head(), sep='\n\n')

key
Date Range                               1960-01-01 to 2018-03-01
Frequency                                                 Monthly
Last Updated                                2018-04-24 701 AM CDT
Notes           Starting with the 2005-02-16 release, the seri...
Release                              New Residential Construction
Name: value, dtype: object

         DATE  VALUE
0  1960-01-01   1092
1  1960-02-01   1088
2  1960-03-01    955
3  1960-04-01   1016
4  1960-05-01   1052

如何在将文本文件中的数据提取到pandas时忽略垃圾数据？

2 个答案: