火花点燃罐子的问题

时间:2018-04-11 19:08:58

标签: apache-spark spark-dataframe ignite lib

我试图在Spark 2.1中使用ignite 2.4。 我在spark-shell

的点火二进制文件中添加了以下库的路径
--conf spark.diver.extraClassPath=/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/optional/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/ignite-indexing/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/ignite-spring/* --conf spark.executor.extraClassPath=/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/optional/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/ignite-indexing/*:/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/ignite-spring/*

但是,我无法导入任何lib,例如

import org.apache.ignite.configuration._ 
error: object ignite is not a member of package org.apache

我该如何解决这个问题?

1 个答案:

答案 0 :(得分:0)

  1. /home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/optional/*: - Ignite在optional中包含许多子文件夹,但java不包含子路径到classpath中。要包含火花罐,您应该将/home/sshuser/apache-ignite-fabric-2.4.0-bin/libs/optional/ignite-spark/*添加到类路径中。

  2. 请阅读官方文档 - https://apacheignite-fs.readme.io/v2.4/docs/installation-deployment