Question

将日期转换为时间戳，将Spark日期从时间戳转换为unix_timestamp的问题返回null。

scala>  import org.apache.spark.sql.functions.unix_timestamp
scala> spark.sql("select from_unixtime(unix_timestamp(('2017-08-13 00:06:05'),'yyyy-MM-dd HH:mm:ss')) AS date").show(false)

+----+
|date|
+----+
|null|
+----+

Answer 1

问题是智利的时间变化，非常感谢。

+-------------------+---------+
|     DateIntermedia|TimeStamp|
+-------------------+---------+
|13-08-2017 00:01:07|     null|
|13-08-2017 00:10:33|     null|
|14-08-2016 00:28:42|     null|
|13-08-2017 00:04:43|     null|
|13-08-2017 00:33:51|     null|
|14-08-2016 00:28:08|     null|
|14-08-2016 00:15:34|     null|
|14-08-2016 00:21:04|     null|
|13-08-2017 00:34:13|     null|
+-------------------+---------+

解决方案，设置timeZone：

  spark.conf.set("spark.sql.session.timeZone", "UTC-6")

将日期转换为时间戳，将unix_timestamp的Spark日期转换为时间戳的问题返回null

1 个答案: