Question

我有一个简单的参数化spark应用程序，其中包含用于传递参数的一行：

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
    at example.SparkExample$.main(SparkExample.scala:37)
    at example.SparkExample.main(SparkExample.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:846)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:194)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:921)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:932)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

所以我为此编写了集成测试。在此测试中，我创建了Kubernetes CRD，它又运行了spark应用程序-定义了主应用程序类文件和jar位置。问题是，当我运行这些测试时，我得到了这样的错误：

  override def beforeAll(): Unit = {
    val customObjectBody = testsUtils.convertYamlToJson(sparkAppYamlFileName)

    val sparkAppCreation = apiInstance.createNamespacedCustomObject(crdGroup, crdVersion, crdNamespace, crdPlural, customObjectBody,"false")
  }


  test("Read file from resources does not throw Exception"){
    noException should be thrownBy testsUtils.getResourceLines(testFileName)
  }


  test("Read file from resources throws InvalidInputException wrong filename"){
    an[NullPointerException] should be thrownBy testsUtils.getResourceLines(testWrongFileName)
  }

错误指向带有参数（我在上面定义）的行。因此，问题在于我没有将参数正确传递给类。那么，如果我想将各种测试文件名传递给应用程序，该如何运行这些测试？

这里是所有测试方法：

<form action="" method="post">

将参数传递给Spark集成测试中的类

0 个答案: