从文档看来,设置/ etc / defaults / dse以启用所有上述功能就足够了。 意味着,还添加了SPARK支持。 现在,节点工具还指示一切正常并且正在运行,并且(单节点)集群的类型为Graph,Search,Analytics 因此,启用工作正常,但是尝试使用分析会导致供应级别出现异常,这可能与Spark配置有关。但是那里没有任何文件。 似乎其他人也被这个问题困扰。
这是一个完整的例外: 构造Graph / TraversalSource时遇到错误-com.google.inject.ProvisionException:无法配置,请参见以下错误:1)注入构造函数com.datastax.bdp.gcore.datastore.DataStoreException的错误:无法执行语句b5807f0d-b1b0- com.datastax.bdp.graph.impl.DseGraphImpl。(DseGraphImpl.java:189)处的4bc3-bd77-794d21725fbb(com.datastax.bdp.graph.impl.GraphModule.configure(未知源)(通过模块:com.datastax) .bdp.graph.impl.DseGraphFactoryImpl $$ Lambda $ 1430/208354649-> com.google.inject.util.Modules $ OverrideModule-> com.datastax.bdp.graph.impl.GraphModule),同时查找com.datastax.bdp.graph .impl.DseGraphImpl 1错误 构造Graph / TraversalSource时遇到错误-无法运行Spark或无法检测到Spark时,无法创建OLAP遍历源。
请注意,尝试从Studio或gremlin控制台使用图分析查询时会收到此异常。 cassandra日志文件等中没有特定的错误。 此外,从dse服务的角度来看,一切似乎都是正确的,但实际上它实际上并不起作用。
这些是在使用某些常量后出现的错误:
WARN [SPARK-MASTER] 2018-07-08 18:25:36,640 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:25:36,643 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:25:36,643 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
WARN [SPARK-WORKER] 2018-07-08 18:25:36,671 SPARK-WORKER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-WORKER] 2018-07-08 18:25:36,672 SPARK-WORKER InternalServiceRunner.java:45 - SPARK-WORKER caused an exception in state STARTING:
ERROR [SPARK-MASTER] 2018-07-08 18:25:41,644 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
WARN [SPARK-MASTER] 2018-07-08 18:25:41,645 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:25:41,647 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:25:41,647 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
ERROR [SPARK-MASTER] 2018-07-08 18:25:46,648 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
WARN [SPARK-MASTER] 2018-07-08 18:25:46,649 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:25:46,652 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:25:46,652 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
WARN [SPARK-WORKER] 2018-07-08 18:25:46,673 SPARK-WORKER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-WORKER] 2018-07-08 18:25:46,673 SPARK-WORKER InternalServiceRunner.java:45 - SPARK-WORKER caused an exception in state STARTING:
ERROR [SPARK-MASTER] 2018-07-08 18:25:51,653 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
WARN [SPARK-MASTER] 2018-07-08 18:25:51,653 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:25:51,656 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:25:51,656 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
ERROR [SPARK-MASTER] 2018-07-08 18:25:56,657 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
WARN [SPARK-MASTER] 2018-07-08 18:25:56,658 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:25:56,660 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:25:56,660 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
WARN [SPARK-WORKER] 2018-07-08 18:25:56,673 SPARK-WORKER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-WORKER] 2018-07-08 18:25:56,674 SPARK-WORKER InternalServiceRunner.java:45 - SPARK-WORKER caused an exception in state STARTING:
ERROR [SPARK-MASTER] 2018-07-08 18:26:01,661 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
WARN [SPARK-MASTER] 2018-07-08 18:26:01,662 SPARK-MASTER Logging.scala:47 - Not overriding system property java.library.path
ERROR [SPARK-MASTER] 2018-07-08 18:26:01,665 SPARK-MASTER Logging.scala:91 - Failed to bind MasterRedirectingUI
ERROR [SPARK-MASTER] 2018-07-08 18:26:01,665 SPARK-MASTER Logging.scala:72 - Failed to stop redirecting UI
ERROR [SPARK-MASTER] 2018-07-08 18:26:06,665 SPARK-MASTER InternalServiceRunner.java:45 - SPARK-MASTER caused an exception in state NOT_STARTED:
答案 0 :(得分:0)
好的,有了这个日志,它变得越来越清晰。真正的罪魁祸首是以下消息:Failed to bind MasterRedirectingUI
-这意味着Spark Master无法启动,因为有人已经在监听它试图使用的端口-默认情况下它是7080
,但是它可能会被覆盖在文件SPARK_MASTER_WEBUI_PORT
中设置spark-env.sh
变量(在tarball安装的resources/spark/conf
中或/etc/dse/spark/
中)。
检查是否没有其他进程在这些端口上侦听-您可以使用netstat -plnt
来执行此操作...