从Datetime操作创建TimeDeltas的pandas错误

时间:2015-08-21 09:47:03

标签: python datetime numpy pandas timedelta

我查看了其他几个相关问题hereherehere,但没有一个问题与我发生过完全相同的问题。

我正在使用Pandas版本0.16.2。我在Pandas数据框中有几列,dtype为datetime64 [ns]:

In [6]: date_list = ["SubmittedDate","PolicyStartDate", "PaidUpDate", "MaturityDate", "DraftDate", "CurrentValuationDate", "DOB", "InForceDate"]

In [11]: data[date_list].head()

Out[11]:
      SubmittedDate PolicyStartDate PaidUpDate MaturityDate DraftDate  \
    0           NaT      2002-11-18        NaT   2041-03-04       NaT
    1           NaT      2015-01-13        NaT          NaT       NaT
    2           NaT      2014-10-15        NaT          NaT       NaT
    3           NaT      2009-08-27        NaT          NaT       NaT
    4           NaT      2007-04-19        NaT   2013-10-01       NaT

      CurrentValuationDate        DOB InForceDate
    0           2015-04-30 1976-03-04  2002-11-18
    1                  NaT 1949-09-27  2015-01-13
    2                  NaT 1947-06-15  2014-10-15
    3           2015-07-30 1960-06-07  2009-08-27
    4           2010-04-21 1950-10-01  2007-04-19

这些最初是字符串格式(例如'1976-03-04'),我使用以下方式转换为datetime对象:

In [7]: for datecol in date_list:
   ...:         data[datecol] = pd.to_datetime(data[datecol], coerce=True, errors = 'raise')

以下是每个列的dtypes:

In [8]: for datecol in date_list:
              print data[datecol].dtypes

返回:

datetime64[ns]
datetime64[ns]
datetime64[ns]
datetime64[ns]
datetime64[ns]
datetime64[ns]
datetime64[ns]
datetime64[ns]

到目前为止,这么好。但我想要做的是为每个列创建一个新列,从特定日期开始以天为单位(作为整数)。

In [13]: current_date = pd.to_datetime("2015-07-31")

我第一次跑这个:

In [14]: for i in date_list:
   ....:         data[i+"InDays"] = data[i].apply(lambda x: current_date - x)

但是,当我检查返回列的dtype时:

In [15]: for datecol in date_list:
   ....:         print data[datecol + "InDays"].dtypes

我得到了这些:

object
timedelta64[ns]
object
timedelta64[ns]
object
timedelta64[ns]
timedelta64[ns]
timedelta64[ns]

我不知道为什么他们三个是对象,什么时候应该是timedeltas。我接下来要做的是:

In [16]: for i in date_list:
   ....:         data[i+"InDays"] = data[i+"InDays"].dt.days

这种方法适用于timedelta列。但是,由于其中三列不是timedeltas,我收到此错误:

AttributeError: Can only use .dt accessor with datetimelike values

我怀疑这三列中有一些值阻止​​Pandas将它们转换为timedeltas。我无法弄清楚如何弄清楚这些价值观可能是什么。

1 个答案:

答案 0 :(得分:1)

出现此问题是因为您有三列仅包含NaT值,这会导致在您对其应用条件时将这些列视为对象。

你应该在apply部分放置某种条件,默认为NaT时的某个时间点。示例 -

for i in date_list:
    data[i+"InDays"] = data[i].apply(lambda x: current_date - x if x is not pd.NaT else pd.Timedelta(0))

或者如果你不能做到这一点,你应该把你想做的条件 - data[i+"InDays"] = data[i+"InDays"].dt.days,只有在系列的dtype允许的情况下才能接受。

或者更简单的方法来更改apply部分以直接获得您想要的内容 -

for i in date_list:
    data[i+"InDays"] = data[i].apply(lambda x: (current_date - x).days if x is not pd.NaT else x)

这会输出 -

In [110]: data
Out[110]:
  SubmittedDate PolicyStartDate PaidUpDate MaturityDate DraftDate  \
0           NaT      2002-11-18        NaT   2041-03-04       NaT
1           NaT      2015-01-13        NaT          NaT       NaT
2           NaT      2014-10-15        NaT          NaT       NaT
3           NaT      2009-08-27        NaT          NaT       NaT
4           NaT      2007-04-19        NaT   2013-10-01       NaT

  CurrentValuationDate        DOB InForceDate SubmittedDateInDays  \
0           2015-04-30 1976-03-04  2002-11-18                 NaT
1                  NaT 1949-09-27  2015-01-13                 NaT
2                  NaT 1947-06-15  2014-10-15                 NaT
3           2015-07-30 1960-06-07  2009-08-27                 NaT
4           2010-04-21 1950-10-01  2007-04-19                 NaT

   PolicyStartDateInDays PaidUpDateInDays MaturityDateInDays DraftDateInDays  \
0                   4638              NaT              -9348             NaT
1                    199              NaT                NaN             NaT
2                    289              NaT                NaN             NaT
3                   2164              NaT                NaN             NaT
4                   3025              NaT                668             NaT

  CurrentValuationDateInDays  DOBInDays  InForceDateInDays
0                         92      14393               4638
1                        NaN      24048                199
2                        NaN      24883                289
3                          1      20142               2164
4                       1927      23679               3025

如果您希望将NaT更改为NaN,则可以使用 -

for i in date_list:
    data[i+"InDays"] = data[i].apply(lambda x: (current_date - x).days if x is not pd.NaT else np.NaN)

示例/演示 -

In [114]: for i in date_list:
   .....:     data[i+"InDays"] = data[i].apply(lambda x: (current_date - x).days if x is not pd.NaT else np.NaN)
   .....:

In [115]: data
Out[115]:
  SubmittedDate PolicyStartDate PaidUpDate MaturityDate DraftDate  \
0           NaT      2002-11-18        NaT   2041-03-04       NaT
1           NaT      2015-01-13        NaT          NaT       NaT
2           NaT      2014-10-15        NaT          NaT       NaT
3           NaT      2009-08-27        NaT          NaT       NaT
4           NaT      2007-04-19        NaT   2013-10-01       NaT

  CurrentValuationDate        DOB InForceDate  SubmittedDateInDays  \
0           2015-04-30 1976-03-04  2002-11-18                  NaN
1                  NaT 1949-09-27  2015-01-13                  NaN
2                  NaT 1947-06-15  2014-10-15                  NaN
3           2015-07-30 1960-06-07  2009-08-27                  NaN
4           2010-04-21 1950-10-01  2007-04-19                  NaN

   PolicyStartDateInDays  PaidUpDateInDays  MaturityDateInDays  \
0                   4638               NaN               -9348
1                    199               NaN                 NaN
2                    289               NaN                 NaN
3                   2164               NaN                 NaN
4                   3025               NaN                 668

   DraftDateInDays  CurrentValuationDateInDays  DOBInDays  InForceDateInDays
0              NaN                          92      14393               4638
1              NaN                         NaN      24048                199
2              NaN                         NaN      24883                289
3              NaN                           1      20142               2164
4              NaN                        1927      23679               3025