Question

我正在尝试使用pandas read_csv函数结合skiprows和nrows逐块读取巨大的csv文件。我使用while循环来做到这一点。

当我到达文件末尾（跳过的行数超过文件中的行数）时，我收到EmptyDataError。

这本身没有问题，我尝试尝试一下-除外。但是，我的方法不起作用...有人知道为什么吗？关于错误处理，我是一个初学者。这是我的代码：

chunksize = 1000
inputfile = "testfile_with_five_rows.txt"
chunknumber = 0
while True:
    try: 
        data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
        print(data)
    except EmptyDataError:
        break

    chunknumber+=1

我得到以下NameError，而不是处理FileNotFound异常：

    ---------------------------------------------------------------------------
EmptyDataError                            Traceback (most recent call last)
<ipython-input-22-e3736135fc2f> in <module>()
      5     try:
----> 6         data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
      7         print(data)

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
    677 
--> 678         return _read(filepath_or_buffer, kwds)
    679 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    439     # Create the parser.
--> 440     parser = TextFileReader(filepath_or_buffer, **kwds)
    441 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in __init__(self, f, engine, **kwds)
    786 
--> 787         self._make_engine(self.engine)
    788 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in _make_engine(self, engine)
   1013         if engine == 'c':
-> 1014             self._engine = CParserWrapper(self.f, **self.options)
   1015         else:

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in __init__(self, src, **kwds)
   1707 
-> 1708         self._reader = parsers.TextReader(src, **kwds)
   1709 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__()

EmptyDataError: No columns to parse from file

During handling of the above exception, another exception occurred:

NameError                                 Traceback (most recent call last)
<ipython-input-22-e3736135fc2f> in <module>()
      6         data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
      7         print(data)
----> 8     except EmptyDataError:
      9         break
     10 

NameError: name 'EmptyDataError' is not defined

无法在python中处理EmptyDataError

0 个答案: