Question

我尝试将spark数据帧的内容写入mysql表。但是我总是得到以下错误：

var squadPlayers = {
    'qb': null,
    'rb1': null,
    'rb2': null,
    'wr1': null,
    'wr2': null,
    'te': null,
    'k': null,
    'flex': null,
    'def': null
};

var getPosKeys = function(){
    var keys = [];
    for (var key in squadPlayers) {
        if (squadPlayers.hasOwnProperty(key)) {
            keys.push(key);
        }
    }
    return keys;
}

var initSquad = function(){
    var keys = getPosKeys();
    console.log(keys);    
    keys.forEach(function(element) {
        var formRef = '#contestEntry_' + element;
        console.log(formRef);
        var playerID = $(formRef).val();
        console.log({'playerID': playerID}); // 633397
        var btnRef = '#draftBtn-'+playerID;
        console.log({'btnRef' : btnRef }); // '#draftBtn-633397'
        var playerAttr = $(btnRef).attr('data-player');
        console.log(playerAttr); // undefined
        var playerData = $(btnRef).data('player');
        console.log(playerData); // undefined

       //do something with the data.....        
       //addPlayerToSquad($)
    }, this);
}

我正在使用spark-1.6.0。对数据库的查询工作没有任何问题：获得正确的设置后，我可以运行查询并获得结果没有任何问题。

java.sql.SQLException: No suitable driver found for 'URL'

但是，将数据帧写入表格不起作用：

val mysqlDriver = "Driver"
val mysqlUsername = "name"
val mysqlPwd = "pwd"
val mysqlUrl = "jdbc:mysql://myUrl" 
 val sqlContext = new org.apache.spark.sql.SQLContext(sc)
Class.forName(mysqlDriver).newInstance

val dataRDD = new JdbcRDD(sc, () =>
  DriverManager.getConnection(mysqlUrl, mysqlUsername, mysqlPwd) ,
  "a query",
  0,
  1000000,
  2,
  r => r.getString("something") + ", " + r.getString("somethingmore") + ", " + r.getString("somethingelse"))

由于此方法似乎已被弃用，我也尝试过：

todb.insertIntoJDBC(mysqlUrl + "/db" + "?user=name&password=pwd", "table", false)

尝试了这两个版本：

val prop = new java.util.Properties
prop.setProperty("user", mysqlUsername)
prop.setProperty("password",mysqlPwd)

以及这两个版本：

prop.setProperty("driver", "/somedir/mysql-connector-java-5.1.37-bin.jar")
prop.setProperty("driver", "mysqlDriver")

在启动spark-shell时，我还将驱动程序的jar作为todb.write.jdbc(mysqlUrl + "/boosting" + "?user=mysqlUsername&password=mysqlPwd", "boosting.user_cluster", prop) todb.write.jdbc(mysqlUrl, "table", prop)参数之一。我还试着设置这个：

–jars

对此问题的任何帮助都将非常感谢！

Answer 1

MySQL的正确driver类是com.mysql.jdbc.Driver。假设其他人设置正确：

prop.setProperty("driver", "com.mysql.jdbc.Driver")

将spark数据帧写入mysql表时出错

1 个答案: