在我的Employees DataFrame中,我想用特定值替换佣金列中的空值(让它为500)。我正在使用以下命令替换所有null值,但我不知道Map中null的键值应该是什么:
`employeeDF.na.replace("commission", Map("" -> 1000)).show()`
I am getting this error :
<console>:32: error: type mismatch;
found : scala.collection.immutable.Map[String,Int]
required: Map[Any,Any]
Note: String <: Any, but trait Map is invariant in type A.
You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10)
employeeDF.na.replace("commission", Map("" -> 1000)).show()
What should I put in key part of map ??
答案 0 :(得分:1)
您可以使用以下两者之一:
employeeDF.na.fill(1000, Seq("commission")).show()
import org.apache.spark.sql.functions._
employeeDF.withColumn("commission", when(col("commission").isNull, 1000) otherwise col("commission")).show()