我已经编写了通用的Scala方法,该方法在读取配置单元列定义之后创建了配置单元表。方法的签名,同样是:
def createHiveTable(tableName: String, metaDataList: List[SchemaColumnDefinition]) = { ....}
case class SchemaColumnDefinition(name: String,
dataType: String,
precision: Option[Integer],
scale: Option[Integer]) {
我已经为同一单元编写了一个单元测试用例
"createHiveTable" should " create hive table with provided column definition" in {
// GIVEN
val columns = SchemaColumnDefinition("testIntColumn", "INT", None, None) ::
SchemaColumnDefinition("testStringColumn", "STRING", None, None) ::
SchemaColumnDefinition("testDecimalColumn", "DECIMAL", None, None) ::
Nil
val testTableName = "testTable"
// WHEN
HiveTableManager.createHiveTable(testTableName, columns)
// THEN
sparkSession.catalog.listColumns(testTableName).foreach { columnDefinition =>
columnDefinition.name match {
case "testIntColumn" => assert(columnDefinition.dataType.toLowerCase == "int")
case "testStringColumn" => assert(columnDefinition.dataType.toLowerCase == "string")
case "testDecimalColumn" => assert(columnDefinition.dataType.toLowerCase == "decimal(10,0)")
}
}
}
但是在同样运行时,出现以下异常:
[scalatest] Cause: java.io.NotSerializableException: org.scalatest.Assertions$AssertionsHelper
[scalatest] Serialization stack:
[scalatest] - object not serializable (class: org.scalatest.Assertions$AssertionsHelper, value: org.scalatest.Assertions$AssertionsHelper@420068a)
[scalatest] - field (class: org.scalatest.FlatSpec, name: assertionsHelper, type: class org.scalatest.Assertions$AssertionsHelper)
[scalatest] - object (class com.demo.spark.executor.hive.HiveTableManagerTest, HiveTableManagerTest)
如果我将“ assert”替换为“ println”语句,则测试用例通过。
我尝试用“ Serializable”扩展测试类,但仍然存在相同的异常。