AWS EC2 Spark / Hadoop集群。
以下baisc K-Means sparklyr代码与Spark 2.0.1一起使用
kmeans_model <- iris_tbl %>%
select(Petal_Width, Petal_Length) %>%
ml_kmeans(centers = 3)
我已升级到 Spark 2.1.1 ,我收到此错误
Error: java.lang.IllegalArgumentException: Field "features" does not exist.
at org.apache.spark.sql.types.StructType$$anonfun$apply$1.apply(StructType.scala:264)
at org.apache.spark.sql.types.StructType$$anonfun$apply$1.apply(StructType.scala:264)
...
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
我已经使用代码
进行了一些测试kmeans_model <- iris_tbl %>%
select(Petal_Width, Petal_Length) %>%
ml_kmeans(k = 3, features = c("Petal_Length", "Petal_Width"))
或
kmeans_model <- iris_tbl %>%
dplyr::select(Petal_Width, Petal_Length) %>%
ml_kmeans(centers = 3, features = c("Petal_Length", "Petal_Width"))
但仍然会得到同样的错误。