在火花结构化流中访问数据帧的单元格值

Question

我想访问结构化流数据帧的单元格值以将其传递给 Impala

代码：

  val CANDIDATESWABTEST_df: DataFrame = spark.readStream
  .format("kafka")
  .option("kafka.bootstrap.servers", "xxxx:9092")
  .option("subscribe", "xxxx")
  .option("group.id", "tests")
  .option("startingOffsets", "latest")
  .option("failOnDataLoss", false)
  .load()
val parsed_CANDIDATESWABTEST_df: DataFrame = CANDIDATESWABTEST_df
  .select(from_json(expr("cast(value as string) as actualValue"), schemaCANDIDATESWABTEST).as("s")).select("s.*").as("dftests")

尝试 #1

  var querySQL = "select * from v_candidate_info where candidate_id=" +parsed_CANDIDATESWABTEST_df.select("CANDIDATEID").take(0) +""

尝试#2

  var querySQL = "select * from v_candidate_info where candidate_id=" +parsed_CANDIDATESWABTEST_df.select("CANDIDATEID")+""

根据 link 尝试 #3

.as[String].collect()

所有这些尝试都没有奏效