Question

我遇到Python + Numpy + Pandas的问题。

我有一个时间戳列表，精度为毫秒，编码为字符串。然后我把它们分成10ms的分辨率，这很顺利。当我将圆形时间戳添加到DataFrame作为新列时，会出现错误 - datetime64对象的值被完全破坏。

我做错了吗？或者是Pandas / NumPy错误？

顺便说一句，我怀疑，这个错误只出现在Windows上 - 我昨天在Mac上尝试相同的代码时没有注意到它（尚未验证）。

import numpy
import pandas as pd

# We create a list of strings. 
time_str_arr = ['2017-06-30T13:51:15.854', '2017-06-30T13:51:16.250',
                '2017-06-30T13:51:16.452', '2017-06-30T13:51:16.659']
# Then we create a time array, rounded to 10ms (actually floored, 
# not rounded), everything seems to be fine here.
rounded_time = numpy.array(time_str_arr, dtype="datetime64[10ms]")
rounded_time 

# Then we create a Pandas DataFrame and assign the time array as a 
# column to it. The datetime64 is destroyed.
d = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']),
  'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}
df = pd.DataFrame(d)
df = df.assign(wrong_time=rounded_time)
df

我得到的输出：

    one two wrong_time
a   1.0 1.0 1974-10-01 18:11:07.585
b   2.0 2.0 1974-10-01 18:11:07.625
c   3.0 3.0 1974-10-01 18:11:07.645
d   NaN 4.0 1974-10-01 18:11:07.665

pd.show_versions（）的输出：

INSTALLED VERSIONS
commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.20.1
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
xarray: None
IPython: 5.3.0
sphinx: 1.5.6
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.2.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.7
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.6.0
html5lib: 0.999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

Answer 1

在我看来这是错误的，因为很明显numpy.datetime64会在内部投放到Timestamp。

对我而言，使用to_datetime：

df = df.assign(wrong_time=pd.to_datetime(rounded_time))
print (df)
   one  two              wrong_time
a  1.0  1.0 2017-06-30 13:51:15.850
b  2.0  2.0 2017-06-30 13:51:16.250
c  3.0  3.0 2017-06-30 13:51:16.450
d  NaN  4.0 2017-06-30 13:51:16.650

另一种解决方案是ns：

df = df.assign(wrong_time=rounded_time.astype('datetime64[ns]'))
print (df)
   one  two              wrong_time
a  1.0  1.0 2017-06-30 13:51:15.850
b  2.0  2.0 2017-06-30 13:51:16.250
c  3.0  3.0 2017-06-30 13:51:16.450
d  NaN  4.0 2017-06-30 13:51:16.650

Answer 2

我在Pandas Git存储库中打开了一个问题。并得到了Jeff Reback建议的解决方案：我们只使用floor（）函数对时间戳进行舍入，而不是创建奇怪的10ms datetime64对象：

In [16]: # We create a list of strings. 
...: time_str_arr = ['2017-06-30T13:51:15.854', '2017-06-30T13:51:16.250',
...:                 '2017-06-30T13:51:16.452', '2017-06-30T13:51:16.659']

In [17]: pd.to_datetime(time_str_arr).floor('10ms')
Out[17]: DatetimeIndex(['2017-06-30 13:51:15.850000', '2017-06-30 13:51:16.250000', '2017-06-30 13:51:16.450000', '2017-06-30 13:51:16.650000'], dtype='datetime64[ns]', freq=None)

来自https://github.com/pandas-dev/pandas/issues/17183

的解决方案

添加到Pandas DataFrame时出现datetime64错误

2 个答案: