我想使用逗号作为小数点分隔符来打开一个csv文件。我通过了decimal=","
。但是,我总是收到错误的csv文件的最后一行的float值未成功解析(我尝试删除csv文件的某些行,并且该错误总是在最后一行...)。
浮点值在csv中也被引用,因此我通过了quoting=csv.QUOTE_ALL
。
有人知道这里会发生什么吗?如果我删除了dtype
选项,它会起作用,但是列amount
是一个字符串...
这是我使用的代码(为了便于阅读而重新格式化):
import pandas as pd
import numpy as np
dateparse = lambda x: pd.datetime.strptime(x, '%d/%m/%Y')
expenses = pd.read_csv("/home/jovyan/work/expenses.csv",
quoting=csv.QUOTE_ALL, decimal=",", header=None, na_values=[""],
names=['date','category','amount','currency','note','tags'],
parse_dates=["date"], date_parser=dateparse,
dtype={"date": str,"category": str, "amount":float, "currency": str,
"note": str, "tags": str})
答案 0 :(得分:0)
我认为这是我不了解的熊猫行为问题,但看来问题出在读取的数据中。浮柱也使用了千位分隔符,例如'1.345,50'。这些线路造成了麻烦。解决方案是使用read_csv选项thousands='.'
。
错误消息令人困惑,因为报告为不可转换为浮点的值取自csv文件的最后一行,而不是引起麻烦的行。...
作为参考,以下是错误消息:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()
TypeError: Cannot cast array from dtype('O') to dtype('float64') according to the rule 'safe'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-109-e77d6530ab30> in <module>()
3 import csv
4 dateparse = lambda x: pd.datetime.strptime(x, '%d/%m/%Y')
----> 5 expenses = pd.read_csv("/home/jovyan/work/test.csv", quoting=csv.QUOTE_ALL, decimal=",", header=None, na_values=[""],names=['date','category','amount','currency','note','tags'],parse_dates=["date"], date_parser=dateparse, dtype={"date": str,"category": str, "amount":float, "currency": str, "note": str, "tags": str})
6 expenses['tags']=expenses['tags'].str.split(",")
7 #, dtype={"date": str,"category": str, "amount":float, "currency": str, "note": str, "tags": str}
/opt/conda/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
676 skip_blank_lines=skip_blank_lines)
677
--> 678 return _read(filepath_or_buffer, kwds)
679
680 parser_f.__name__ = name
/opt/conda/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
444
445 try:
--> 446 data = parser.read(nrows)
447 finally:
448 parser.close()
/opt/conda/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
1034 raise ValueError('skipfooter not supported for iteration')
1035
-> 1036 ret = self._engine.read(nrows)
1037
1038 # May alter columns / col_dict
/opt/conda/lib/python3.6/site-packages/pandas/io/parsers.py in read(self, nrows)
1846 def read(self, nrows=None):
1847 try:
-> 1848 data = self._reader.read(nrows)
1849 except StopIteration:
1850 if self._first_chunk:
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_column_data()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()
ValueError: could not convert string to float: '0,99'