Question

我已经按照办公文档安装了cdh6.2，并且它运行正常。但是我仍然想要代码pySpark并在色相中运行它，而不是在pySpark-shell中编码pySpark。我在CHD的“色调配置”页面中尝试了很多配置。我的所作所为：

[notebook]

  [[interpreters]]

    [[[spark]]]
        name=Scala
        interface=livy
    [[[pyspark]]]
        name=PySpark
        interface=livy
    [[[r]]]
        name=R
        interface=livy

安装了版本为0.6.0的livy，livy服务器日志：


19/05/09 18:01:25 INFO server.AccessManager: AccessControlManager acls disabled;users with view permission: ;users with modify permission: ;users with super permission: ;other allowed users: *
19/05/09 18:01:25 INFO utils.LineBufferedStream: WARNING: User-defined SPARK_HOME (/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark) overrides detected (/opt/cloudera/parcels/CDH/lib/spark).
19/05/09 18:01:25 INFO utils.LineBufferedStream: WARNING: Running spark-class from user-defined location.
19/05/09 18:01:26 INFO utils.LineBufferedStream: Welcome to
19/05/09 18:01:26 INFO utils.LineBufferedStream:       ____              __
19/05/09 18:01:26 INFO utils.LineBufferedStream:      / __/__  ___ _____/ /__
19/05/09 18:01:26 INFO utils.LineBufferedStream:     _\ \/ _ \/ _ `/ __/  '_/
19/05/09 18:01:26 INFO utils.LineBufferedStream:    /___/ .__/\_,_/_/ /_/\_\   version 2.4.0-cdh6.2.0
19/05/09 18:01:26 INFO utils.LineBufferedStream:       /_/
19/05/09 18:01:26 INFO utils.LineBufferedStream:                         
19/05/09 18:01:26 INFO utils.LineBufferedStream: Using Scala version 2.11.12, OpenJDK 64-Bit Server VM, 1.8.0_212
19/05/09 18:01:26 INFO utils.LineBufferedStream: Branch HEAD
19/05/09 18:01:26 INFO utils.LineBufferedStream: Compiled by user jenkins on 2019-03-14T07:01:24Z
19/05/09 18:01:26 INFO utils.LineBufferedStream: Revision HEAD
19/05/09 18:01:26 INFO utils.LineBufferedStream: Url 
19/05/09 18:01:26 INFO utils.LineBufferedStream: Type --help for more information.
19/05/09 18:01:26 INFO recovery.StateStore$: Using BlackholeStateStore for recovery.
19/05/09 18:01:26 INFO sessions.BatchSessionManager: Recovered 0 batch sessions. Next session id: 0
19/05/09 18:01:26 INFO sessions.InteractiveSessionManager: Recovered 0 interactive sessions. Next session id: 0
19/05/09 18:01:26 INFO sessions.InteractiveSessionManager: Heartbeat watchdog thread started.
19/05/09 18:01:26 INFO util.log: Logging initialized @1490ms
19/05/09 18:01:26 INFO server.Server: jetty-9.3.24.v20180605, build timestamp: 2018-06-06T01:11:56+08:00, git hash: 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
19/05/09 18:01:26 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@1522d8a0{/,file:///opt/apache-livy-0.6.0-incubating-bin/src/main/org/apache/livy/server,AVAILABLE}
19/05/09 18:01:26 INFO server.AbstractNCSARequestLog: Opened /opt/apache-livy-0.6.0-incubating-bin/logs/2019_05_09.request.log
19/05/09 18:01:26 INFO server.AbstractConnector: Started ServerConnector@77a98a6a{HTTP/1.1,[http/1.1]}{0.0.0.0:8998}
19/05/09 18:01:26 INFO server.Server: Started @1674ms
19/05/09 18:01:26 INFO server.WebServer: Starting server on http://cdh.n1.com:8998

Answer 1

添加后可以看到：

[desktop]
app_blacklist=

[notebook]
  show_notebooks=true

在Hue的笔记本中设置Cloudera CDH6.2编码pySpark并运行

1 个答案: