Question

我正在尝试将CSV文件读入pandas数据帧。它包含数字，字符串和日期的混合。我试过了：

trades_df = pd.DataFrame.from_csv('trades.csv')

并收到错误：

---------------------------------------------------------------------------
ParserError                               Traceback (most recent call last)
<ipython-input-10-af86bdcdc851> in <module>()
      1 # read a database from CSV and load it into a pandas dataframe
----> 2 trades_df = pd.DataFrame.from_csv('trades.csv', infer_datetime_format=True)
      3 trades_df.head()

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/core/frame.py in from_csv(cls, path, header, sep, index_col, parse_dates, encoding, tupleize_cols, infer_datetime_format)
   1249                           parse_dates=parse_dates, index_col=index_col,
   1250                           encoding=encoding, tupleize_cols=tupleize_cols,
-> 1251                           infer_datetime_format=infer_datetime_format)
   1252 
   1253     def to_sparse(self, fill_value=None, kind='block'):

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision)
    653                     skip_blank_lines=skip_blank_lines)
    654 
--> 655         return _read(filepath_or_buffer, kwds)
    656 
    657     parser_f.__name__ = name

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    409 
    410     try:
--> 411         data = parser.read(nrows)
    412     finally:
    413         parser.close()

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
   1003                 raise ValueError('skipfooter not supported for iteration')
   1004 
-> 1005         ret = self._engine.read(nrows)
   1006 
   1007         if self.options.get('as_recarray'):

~/anaconda/envs/env3_insight/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
   1746     def read(self, nrows=None):
   1747         try:
-> 1748             data = self._reader.read(nrows)
   1749         except StopIteration:
   1750             if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)()

ParserError: Error tokenizing data. C error: Expected 49 fields in line 1110195, saw 65

我认为由于CSV文件中的日期格式，pandas并不满意。我尝试了不同的编码，infer_datetime_format = True，dtype = str，dtype = object。没有运气:(任何想法？

Answer 1

好的，我最终搞清楚了。这些是我的两种方法：

将所有内容创建为字符串，然后稍后更改类型。

mystuff = pandas.read_csv(trades.csv, dtype=str)

然后我改变了类型：

mystuff['col_a'] = mystuff[['col_b']].apply(pd.to_numeric)
mystuff['col_c'] = mystuff['col_c'].apply(pd.to_datetime, format='%Y%m%d', errors='coerce')

从一开始就指定每列的数据类型。

pandas.read_csv(path,dtype={'col_a':str,'col_b':int, 'col_c':datetime...})

pd.DataFrame.from_csv与infer_datetime_format相关的错误

1 个答案: