如何使用sparklyr [可重现示例]从时间戳中提取小时?

时间:2019-05-10 10:18:54

标签: r apache-spark-sql lubridate sparklyr

请为可复制的目的找到以下示例。我的数据帧是df,我正在尝试从新创建的列“ local_time”中获取本地时间。我尝试从local_date获取local_hour时遇到问题。不确定whar是错的

df<-data.frame(utc_time_stamp=c("2019-04-28 18:35:26","2019-04-28 21:28:58","2019-04-28 18:59:01"),time_zone=c("Asia/Tokyo","Australia/Brisbane","Etc/GMT-12"))


#loading data into spark

df1<-copy_to(sc,df,"df1",overwrite = T)


# creating local time using time zone from UTC time stamps

df2 <- df1 %>%
group_by(time_zone) %>%
mutate(local_time = from_utc_timestamp(timestamp(utc_time_stamp) ,time_zone))


# Extracting hour from local time column

df3 <- df2 %>%
group_by(time_zone) %>%
mutate(local_hour = hour(timestamp(local_time)))

这是我执行与小时提取相关的代码时遇到的错误

  Error: org.apache.spark.sql.catalyst.parser.ParseException: 
missing ')' at 'FROM'(line 1, pos 65)

== SQL ==
SELECT `utc_time_stamp`, `time_zone`, `local_time`, EXTRACT(hour FROM timestamp(`local_time`)) AS `local_hour`
-----------------------------------------------------------------^^^
FROM (SELECT `utc_time_stamp`, `time_zone`, 
from_utc_timestamp(timestamp(`utc_time_stamp`), `time_zone`) AS `local_time`
FROM `df1`) `dbplyr_190`

0 个答案:

没有答案