Question

我创建了类似于此的spark工作：How to retrieve coefficient names after label encoding and one hot encoding on scikit-learn?

将导入声明更改为：

import io.hydrosphere.mist.lib.spark2.{ContextSupport, MistJob}

因为现有导入import io.hydrosphere.mist.api.{ContextSupport, MistJob}无法解决。

使用sbt package创建了jar并将其添加到router.conf，如下所示：

first-job = {
  path = "path to jar"
  className = "SimpleContext$"
  namespace = "foo"
}

运行上述作业的curl命令无法返回

找不到请求的资源。

但是当指向现有curl时，mist-examples-spark2.jar命令运行正常。这是否意味着jar创建时存在问题？有没有人能够运行Mist Jobs？

我运行mist-master的控制台中的记录器显示：

17-07-11 21:56:52 [mist-akka.actor.default-dispatcher-18] ERROR phere.mist.master.JobEndpoints:44 Job's loading failed for first-job
java.lang.NoClassDefFoundError: io/hydrosphere/mist/lib/spark2/MistJob
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
.
.

导入声明是否有问题？

Answer 1

Currently, we are working to make our library better and it leads to that it can have binary incompatibility between versions. It means that you should ensure that your local mist installation( or version of docker image) is same as the version of mist-lib that you use to build job.

在Hydrosphere Mist中执行简单的Spark作业

1 个答案: