计算Dataframes中列的平均字符串长度

Question

我迷失了如何使用scala计算数据框中任何列的平均字符串长度。我已经能够轻松地为数字列执行以下操作

val avgDF = df.dtypes.filter(x => x._2 == "DoubleType").map(ct =>avg(col(ct._1))).toList

Answer 1

import org.apache.spark.sql.functions._

val avgDF = df.agg(mean(length(col("yourColumn"))))

Answer 2

val findLength = udf {（ColValue：String）=＆gt; ColValue.size}

  myData.dtypes.filter(x=>x._2=="StringType").foreach(f=>
  myData.select(avg(findLength(col(f._1)))).show()      
  )

样本数据

Name|Age|email
Hari|12|hary@h0otmail.ocm
Hari|12|hary@h0otmail.ocm
Hari|12|hary@h0otmail.ocm
Hari|12|hary@h0otmail.ocm
Hasasasi|12|hary@h0otmail.in

输出

+-------------------+
|AVG(scalaUDF(Name))|
+-------------------+
|                4.8|
+-------------------+


+--------------------+
|AVG(scalaUDF(email))|
+--------------------+
|                16.8|
+--------------------+