Scala:从大纪元开始的星期几

时间:2019-12-30 14:18:07

标签: scala apache-spark apache-spark-sql user-defined-functions

我正在尝试从新纪元开始获取星期几。

我可以使用时间戳记来获取日期,但不能使用epochtime代替时间戳记来获取日期。

下面是我尝试过的事情:

import java.time.LocalDate
import java.time.format.DateTimeFormatter
val df = DateTimeFormatter.ofPattern("dd/MM/yyyy")
val dayOfWeek = LocalDate.parse("21/05/2017",df).getDayOfWeek

dayOfWeek: java.time.DayOfWeek = SUNDAY

我正在寻找以下类似的内容,以便能够以INT表示工作日。例如星期一为1,依此类推,但使用纪元时间而不是时间戳。样本 epochtime:1576683390

 import spark.sqlContext.implicits._

  val df = Seq(("2019-07-01 12:01:19.000"),
("2019-06-24 12:01:19.000"),
("2019-11-16 16:44:55.406"),
("2019-11-16 16:50:59.406")).toDF("input_timestamp")

df.withColumn("input_timestamp",
to_timestamp(col("input_timestamp"))).withColumn("week_day_number", date_format(col("input_timestamp"), "u")).withColumn("week_day_abb", date_format(col("input_timestamp"), "E")).show(false)

+-----------------------+---------------+------------+
|input_timestamp        |week_day_number|week_day_abb|
+-----------------------+---------------+------------+
|2019-07-01 12:01:19    |1              |Mon         |
|2019-06-24 12:01:19    |1              |Mon         |
|2019-11-16 16:44:55.406|6              |Sat         |
|2019-11-16 16:50:59.406|6              |Sat         |
+-----------------------+---------------+------------+

先谢谢了!

1 个答案:

答案 0 :(得分:3)

Sekar,如果我正确理解了您的问题,则可以将纪元转换为时间戳记:

import spark.implicits._
import org.apache.spark.sql.functions._

val df = Seq(1576683390).toDF("input_timestamp")

df.withColumn("input_timestamp", $"input_timestamp".cast(TimestampType))
  .withColumn("week_day_number", date_format(col("input_timestamp"), "u"))
  .withColumn("week_day_abb", date_format(col("input_timestamp"), "E"))
  .show(false)

+---------------------+---------------+------------+
|input_timestamp      |week_day_number|week_day_abb|
+---------------------+---------------+------------+
|2019-12-18 16:36:30.0|3              |Wed         |
+---------------------+---------------+------------+

编辑

使用简单的Sequence和LocalDateTime:

import java.time.{Instant, LocalDateTime, ZoneOffset}

val seq = Seq(1576683390)

seq.map{epoch =>
  val dayOfWeek = LocalDateTime.ofInstant(Instant.ofEpochSecond(epoch), ZoneOffset.UTC).getDayOfWeek
  (dayOfWeek.getValue, dayOfWeek.toString)
}.foreach{println}

输出:(3,WEDNESDAY)

相关问题