Question

在以下代码中的w.toSeq.toDS()上获取nullpointer异常。

var w = new ListBuffer[String]()
jList match {
  case Some(x) => {
    for (record <- x) {
      w += mapper.writeValueAsString(record)
    }
  }
  case None => println(s"No data read from file : ${f}")
}

import spark.implicits._
val jsonDataSet = w.toSeq.toDS()

有什么想法吗？

Answer 1

toSeq.toDS()似乎没有问题。您共享的邮政编码应无任何错误/异常。我已经准备了以下测试数据，并且能够成功运行。请找到下面的示例代码。它应该与其他一些代码有关。我会要求您共享整个代码。

import org.apache.spark.sql.SparkSession
import scala.collection.mutable.ListBuffer

object TestToSeqToDs extends App {

  val spark = SparkSession
    .builder()
    .master("local")
    .appName("Test toSeq.toDS() function")
    .getOrCreate()

  def jList(): Option[Seq[String]] = {
    try {
      Some(Seq(("john"), ("Edward")))
    } catch {
      case e: Exception => None
    }
  }

  var w = new ListBuffer[String]()

  jList match {
    case Some(x) => {
      for (record <- x) {
        w += record
      }
    }
    case None => println("That didn't work.")
  }

  import spark.implicits._

  val jsonDataSet = w.toSeq.toDS()
   //OR
  val jsonDataSet = w.toDS()

  println(jsonDataSet.show())

+------+
| value|
+------+
|  john|
|Edward|
+------+

}

谢谢！

toSeq.toDS（）提供java.lang.NullPointerException

1 个答案: