Question

我遇到了pandas.to_csv在datetime64类型的列上删除值的问题。

In [24]: df
Out[24]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 28982 entries, 0 to 28981
Data columns (total 4 columns):
value    28982  non-null values
date1    28982  non-null values
date2    22772  non-null values
date3    28982  non-null values
dtypes: datetime64[ns](3), float64(1)

In [25]: df.tail()
Out[25]: 
       value               date1               date2               date3
28977  25.44 2002-08-21 00:00:00 2013-05-03 00:00:00 2007-09-01 00:00:00
28978  25.86 2002-08-21 00:00:00 2013-05-03 00:00:00 2007-09-01 00:00:00
28979  26.08 2002-08-21 00:00:00 2013-05-03 00:00:00 2007-09-01 00:00:00
28980  25.84 2002-08-21 00:00:00 2013-05-03 00:00:00 2007-09-01 00:00:00
28981  25.35 2002-08-21 00:00:00 2013-05-03 00:00:00 2007-09-01 00:00:00

In [26]: df.to_csv('test.csv', index = False)

In [27]: df2 = pd.read_csv('test.csv', header = 0)

In [28]: df2
Out[28]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 28982 entries, 0 to 28981
Data columns (total 4 columns):
value    28982  non-null values
date1    28982  non-null values
date2    21070  non-null values
date3    17036  non-null values
dtypes: float64(1), object(3)

In [29]: df2.tail()
Out[29]: 
       value                date1 date2 date3
28977  25.44  2002-08-21 00:00:00   NaN   NaN
28978  25.86  2002-08-21 00:00:00   NaN   NaN
28979  26.08  2002-08-21 00:00:00   NaN   NaN
28980  25.84  2002-08-21 00:00:00   NaN   NaN
28981  25.35  2002-08-21 00:00:00   NaN   NaN

如图所示，我将df写入文件并立即将其读回df2，csv文件中的date2和date3列在底部有很多缺失值。这是一个错误吗？顺便说一句，我使用的是Pandas 0.11。

Answer 1

这是一个已知问题：https://github.com/pydata/pandas/issues/3062

解决方法基本上是这样的：

for c in datetime_columns_that_have_NaT:

     df[c] = df[c].astype('object')

df.to_csv()

如果你指定了parse_dates = [that_column_num]

，当你读回来的时候

它会起作用

或者，你可以像你一样写，然后像这样阅读：

dfc = pd.read_csv('test.csv',index_col=0).convert_objects(convert_dates='coerce')

将强制进行日期转换

Pandas to_csv会丢弃值

1 个答案: