无法从Azure Databricks连接到SQL Server受管实例

时间:2019-01-09 10:07:21

标签: scala azure jdbc databricks azure-databricks

我正在尝试从数据块连接到Azure Sql托管实例。我正在使用Scala连接到它。我从Microsoft网站(https://docs.azuredatabricks.net/spark/latest/data-sources/sql-databases.html)复制的代码

我实际的scala代码:(我更改了凭据和IP。但是我确保它们是正确的,因为我已从sql server托管实例选项中的连接字符串中复制了它们)

Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver")
val jdbcHostname = "dev-migdb.nf53e3653n43.database.windows.net"
val jdbcPort = 1433
val jdbcDatabase = "MYDB"

// Create the JDBC URL without passing in the user and password parameters.
val jdbcUrl = s"jdbc:sqlserver://${jdbcHostname}:${jdbcPort};database=${jdbcDatabase};loginTimeout=90"

// Create a Properties() object to hold the parameters.
import java.util.Properties
val connectionProperties = new Properties()

connectionProperties.put("user", "db-devmigmgd")
connectionProperties.put("password", "pwd##321232123")

val driverClass = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
connectionProperties.setProperty("Driver", driverClass)

val employees_table = spark.read.jdbc(jdbcUrl, "employees", connectionProperties)
0

错误:

com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host dev-migdb.nf53e3653n43.database.windows.net, port 1433 has failed. Error: "Connection timed out: no further information. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
    at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:227)
    at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:284)
    at com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2435)
    at com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:635)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2010)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1687)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1528)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:866)
    at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:569)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:336)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:286)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:274)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:198)
    at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:301)
    at linee84eb162c20345fc84ad591cfefe930f29.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command-999597493877319:49)
    at linee84eb162c20345fc84ad591cfefe930f29.$read$$iw$$iw$$iw$$iw$$iw.<init>(command-999597493877319:104)

另一方面:

  1. 我能够从同一Azure订阅上存在的VM(使用SSMS)连接到同一托管实例
  2. 我用.Net编写并托管在该VM上的自定义应用程序也能够连接到同一实例

此外,我无法从我在上述VM上使用spark shell执行的scala代码连接到同一实例。但是我得到的错误是不同的。请在下面找到错误。

com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'db-devmigmgd@dev-migdb'.
  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:197)
  at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:246)
  at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:83)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:2529)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1905)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:1893)
  at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4874)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1045)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700)
  at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
  at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
  at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:238)
  ... 76 elided

1 个答案:

答案 0 :(得分:0)

感谢@simon_dmorias

根据您与Microsoft团队的评论和讨论,不同VNET上的资源才是真正的问题。

因此,作为最佳实践,请始终在相同的VNET中创建资源。