我们如何尝试使用它?
在代码中写道:
import com.databricks.dbutils_v1.DBUtilsHolder.dbutils
class Job {
// business logic ..
val jobResult: String = ???
dbutils.notebook.exit(jobResult)
}
当我们将代码打包到jar中时,我们得到:
[error] missing or invalid dependency detected while loading class file 'NotebookUtils.class'.
[error] Could not access term common in package com.databricks.backend,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
[error] A full rebuild may help if 'NotebookUtils.class' was compiled against an incompatible version of com.databricks.backend.
[error] one error found
[error] (compile:compileIncremental) Compilation failed
答案 0 :(得分:0)
原因甚至是dbutils-api
的最新版本(即0.0.3)现在依赖于com.databricks.backend.common
(既不是dbutils-api的一部分,也不是其他任何公共databricks库的一部分)。 / p>
trait NotebookUtils ... {
...
def getContext() : com.databricks.backend.common.rpc.CommandContext
def setContext(ctx : com.databricks.backend.common.rpc.CommandContext) : scala.Unit
}
而:
jar tvf dbutils-api_2.11-0.0.3.jar | grep backend
439 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/Default.class
1337 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo$$anonfun$hashCode$1.class
1265 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo$$anonfun$hashCode$2.class
4109 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/FileInfo.class
1256 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/GrantInfo.class
1480 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/MountInfo.class
452 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/PackageGroup.class
0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/
0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/
0 Tue Jan 01 11:00:00 MSK 1980 com/databricks/backend/daemon/dbutils/
最可能的原因是com.databricks.backend.common
在dbutils定义文件中标记为提供的作用域(但是我不能100%确定,因为此lib在开源中不可用)。
我宁愿将其视为Databricks库中的错误。