我已将CSV加载到数据框中并使用SPARK SQL执行操作。
我的列名决定有＆＃34;对＆＃34;或＆＃34;错误＆＃34;值。
我想找出正确决定/总决定的比例。
答案 0 :(得分：0)
在snippet下面使用spark sql来获得正确决策的比例..
scala> val df = sc.parallelize(Seq((1,"right"),(2,"right"),(3,"right"),(4,"wrong"))).toDF("col1","col2")
df: org.apache.spark.sql.DataFrame = [col1: int, col2: string]
scala> df.registerTempTable("test_table")
warning: there was one deprecation warning; re-run with -deprecation for details
scala> spark.sql("""SELECT sum(CASE WHEN col2 = "right" THEN 1 ELSE 0 END)/count(*) as percentage FROM test_table""").show()
+----------+
|percentage|
+----------+
| 0.75|
+----------+