如何将DataFrame写入MySQL表

时间:2017-11-09 23:11:43

标签: scala apache-spark dataframe apache-spark-sql

很抱歉,如果听起来模糊不清,可以解释编写现有DataFrame的步骤" df"进入MySQL表说" product_mysql"而另一种方式。

1 个答案:

答案 0 :(得分:6)

请参阅this databricks article : Connecting to SQL Databases using JDBC.

import org.apache.spark.sql.SaveMode

val df = spark.table("...")
println(df.rdd.partitions.length)
// given the number of partitions above, users can reduce the partition value by calling coalesce() or increase it by calling repartition() to manage the number of connections.
df.repartition(10).write.mode(SaveMode.Append).jdbc(jdbcUrl, "product_mysql", connectionProperties)