Question

我无法使用 databricks spark cluster 将数据写入**** cosmos db ****。但是，我尝试了Stackoverflow和Github中的所有链接和解决方案，并尝试了每种版本的所有可能的jar。

错误堆栈为：

java.lang.NoSuchMethodError：

com.microsoft.azure.documentdb.Offer.getContent（）Lorg / json / JSONObject; 在 com.microsoft.azure.cosmosdb.spark.CosmosDBConnection.getCollectionThroughput（CosmosDBConnection.scala：163）在 com.microsoft.azure.cosmosdb.spark.CosmosDBSpark $ .save（CosmosDBSpark.scala：173）在 com.microsoft.azure.cosmosdb.spark.CosmosDBSpark $ .save（CosmosDBSpark.scala：501）在 com.microsoft.azure.cosmosdb.spark.DefaultSource.createRelation（DefaultSource.scala：74）在 org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run（SaveIntoDataSourceCommand.scala：45）在 org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult $ lzycompute（commands.scala：72）在 org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult（commands.scala：70）在 org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute（commands.scala：88）在 org.apache.spark.sql.execution.SparkPlan $$ anonfun $ execute $ 1.apply（SparkPlan.scala：143）在 org.apache.spark.sql.execution.SparkPlan $$ anonfun $ execute $ 1.apply（SparkPlan.scala：131）在 org.apache.spark.sql.execution.SparkPlan $$ anonfun $ executeQuery $ 5.apply（SparkPlan.scala：183）在 org.apache.spark.rdd.RDDOperationScope $ .withScope（RDDOperationScope.scala：151）在 org.apache.spark.sql.execution.SparkPlan.executeQuery（SparkPlan.scala：180）在 org.apache.spark.sql.execution.SparkPlan.execute（SparkPlan.scala：131）在 org.apache.spark.sql.execution.QueryExecution.toRdd $ lzycompute（QueryExecution.scala：114）在 org.apache.spark.sql.execution.QueryExecution.toRdd（QueryExecution.scala：114）在 org.apache.spark.sql.DataFrameWriter $$ anonfun $ runCommand $ 1.apply（DataFrameWriter.scala：690）在 org.apache.spark.sql.DataFrameWriter $$ anonfun $ runCommand $ 1.apply（DataFrameWriter.scala：690）在 org.apache.spark.sql.execution.SQLExecution $$ anonfun $ withCustomExecutionEnv $ 1.apply（SQLExecution.scala：99）在 org.apache.spark.sql.execution.SQLExecution $ .withSQLConfPropagated（SQLExecution.scala：228）在 org.apache.spark.sql.execution.SQLExecution $ .withCustomExecutionEnv（SQLExecution.scala：85）k

我的代码是：

import org.joda.time._
import org.joda.time.format._

import com.microsoft.azure.cosmosdb.spark.schema._
import com.microsoft.azure.cosmosdb.spark.CosmosDBSpark
import com.microsoft.azure.cosmosdb.spark.config.Config

import org.apache.spark.sql.functions._

val configMap = Map(
  "Endpoint" -> "MY-CONNECTION-ENDPOINT",
  "Masterkey" -> "MY-KEY",
  "Database" -> "Families",
  "Collection" -> "Families","Upsert"->"true")

val config = Config(configMap)
val df = spark.range(5).select(col("id").cast("string").as("value"))
df.write.mode(SaveMode.Overwrite).cosmosDB(config)

无法将数据帧写入Azure Cosmo DB

0 个答案: