无法使用整数值

时间:2019-10-12 16:32:15

标签: scala apache dataframe apache-spark apache-spark-sql

Employee Dataframe

在我的Employees DataFrame中,我想用特定值替换佣金列中的空值(让它为500)。我正在使用以下命令替换所有null值,但我不知道Map中null的键值应该是什么:

    `employeeDF.na.replace("commission", Map("" -> 1000)).show()`

I am getting this error :

    <console>:32: error: type mismatch;
         found   : scala.collection.immutable.Map[String,Int]
         required: Map[Any,Any]
        Note: String <: Any, but trait Map is invariant in type A.
        You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
               employeeDF.na.replace("commission", Map("" -> 1000)).show()

What should I put in key part  of map ??

1 个答案:

答案 0 :(得分:1)

您可以使用以下两者之一:

employeeDF.na.fill(1000, Seq("commission")).show()


import org.apache.spark.sql.functions._
employeeDF.withColumn("commission", when(col("commission").isNull, 1000) otherwise col("commission")).show()