Question

spark_write_jdbc(members_df,
    name = "Mbrs",
    options = list(
      url = paste0("jdbc:mysql://",mysql_host,":",mysql_port,"/",dbname),
      user = mysql_user,
      password = mysql_password),
    mode = "append")

导致以下异常：

Error: java.sql.SQLException: No suitable driver
  at java.sql.DriverManager.getDriver(DriverManager.java:315)

.jar文件位于运行RStudio的服务器上的文件夹中，配置详情如下。我们可以通过RMySql包访问MySql，这样MySql就可以正常使用了。

config$`spark.sparklyr.shell.driver-class-path` <- "/dev/shm/temp/mysql-connector-java-5.1.44-bin.jar"

Answer 1

即使问题不同，我认为答案仍然适用于此：

How to use a predicate while reading from JDBC connection?

通过sparklyr连接到JDBC MySQL的代码（我做了一些小的更改以简化代码，由Jake Russ编写）

library(sparklyr)
library(dplyr)

config <- spark_config()
#config$`sparklyr.shell.driver-class-path` <- "E:\\spark232_hadoop27\\jars\\mysql-connector-java-5.1.47-bin.jar"
#in my case, using RStudio and Sparkly this seemed to be optional

sc <- spark_connect(master = "local")

db_tbl <- spark_read_jdbc(sc,
                  name    = "table_name",  
                  options = list(url      = "jdbc:mysql://localhost:3306/schema_name",
                                 user     = "root",
                                 password = "password",
                                 dbtable  = "table_name"))

如何使用sparklyr spark_write_jdbc连接到MySql

1 个答案: