我正尝试使用Spark连接器和AAD包连接到Azure SQL,如下面的链接所述...
但是无论使用身份验证类型为“ ActiveDirectoryPassword”还是“ accessToken”,都会出现错误。
使用“ accessToken”,我得到“用户“ NT Authority \ Anonymous Logon”的登录失败。.并且使用“ ActiveDirectoryPassword”,我得到了类似的错误:(我能够成功运行import com.microsoft.aad .adal4j.AuthenticationException ...仍然没有运气。)
java.lang.NoClassDefFoundError: com/microsoft/aad/adal4j/AuthenticationException
at com.microsoft.sqlserver.jdbc.SQLServerConnection.getFedAuthToken(SQLServerConnection.java:4035)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.onFedAuthInfo(SQLServerConnection.java:4011)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.processFedAuthInfo(SQLServerConnection.java:3980)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onFedAuthInfo(tdsparser.java:264)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:100)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:4772)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3581)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:81)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3541)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7240)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2869)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2395)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2042)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1889)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1120)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:700)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.<init>(JDBCRelation.scala:115)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:52)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:341)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:164)
at org.apache.spark.sql.DataFrameReader.jdbc(DataFrameReader.scala:254)
at com.microsoft.azure.sqldb.spark.connect.DataFrameReaderFunctions.sqlDB(DataFrameReaderFunctions.scala:44)
... 55 elided
按照以下链接上的说明进行操作: https://docs.microsoft.com/en-us/azure/sql-database/sql-database-spark-connector#connect-spark-to-azure-sql-database-using-aad-authentication
遵循以下步骤来解决此问题:
使用Spark内核为Scala命令创建新笔记本并在以下命令下运行:
%% configure -f {“ conf”: { “ spark.jars.packages”:“ com.microsoft.azure:azure-sqldb-spark:1.0.2,com.microsoft.azure:adal4j:1.6.3” } }
然后运行:
导入com.microsoft.azure.sqldb.spark.config.Config 导入com.microsoft.azure.sqldb.spark.connect._ 导入com.microsoft.aad.adal4j._ 导入com.microsoft.aad.adal4j.AuthenticationException
其次:
val config = Config(Map( “ url”->“ xxxxx.database.windows.net”, “数据库名称”->“ db1”, “ dbtable”->“ dbo.BuildVersion”, “用户”->“ xyz@domain.com”, “密码”->“ xxxxxxxx”, “身份验证”->“ ActiveDirectoryPassword”, “ hostNameInCertificate”->“ * .database.windows.net”, “加密”->“真” ))
要阅读上表,请运行:
val collection = spark.read.sqlDB(config)
出现错误。
预期会获得表结果,但会出现异常。