zeppelin with spark 1.5.2独立集群错误

时间:2015-12-11 10:31:19

标签: apache-spark apache-zeppelin apache-spark-1.5

我已将Zeppelin 0.5.5二进制文件下载到redhat 6.3实例中。我也成功地建立了火花1.5.2。现在,我想让zeppelin使用spark版本而不是zeppelin附带的预编译版本。

为此,我编辑了conf / so,它只包含这条未注释的行:

export SPARK_HOME=/home/eron/spark-1.5.2/

我还编辑了zeppelin GUI,现在我有了Master set:

master = spark://master:7077

然后我点击重启并尝试再次阅读教程:

val bankText = sc.textFile("/mnt/zeppelin-0.5.5-incubating-bin-all/bank.csv")
  

org.apache.thrift.transport.TTransportException at   org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)   在org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)   在   org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)   在   org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)   在   org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)   在   org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)   在   org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService $ Client.recv_interpret(RemoteInterpreterService.java:220)   在   org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService $ Client.interpret(RemoteInterpreterService.java:205)   在   org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:211)   在   org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)   在org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:207)   在org.apache.zeppelin.scheduler.Job.run(Job.java:170)at   org.apache.zeppelin.scheduler.RemoteScheduler $ JobRunner.run(RemoteScheduler.java:304)   在   java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:471)   在java.util.concurrent.FutureTask.run(FutureTask.java:262)at   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:178)   在   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)   在   java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)   在   java.util.concurrent.ThreadPoolExecutor中的$ Worker.run(ThreadPoolExecutor.java:615)   在java.lang.Thread.run(Thread.java:745)

另外,只是做

sc

的产率:

  

java.net.ConnectException:拒绝连接   java.net.PlainSocketImpl.socketConnect(Native Method)at   java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)   在   java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)   在   java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)   在java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)at   java.net.Socket.connect(Socket.java:579)at   org.apache.thrift.transport.TSocket.open(TSocket.java:182)at at   org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)   在   org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)   在   org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)   在   org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)   在   org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)   在   org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)   在   org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)   在   org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:192)   在   org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)   在org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:207)   在org.apache.zeppelin.scheduler.Job.run(Job.java:170)at   org.apache.zeppelin.scheduler.RemoteScheduler $ JobRunner.run(RemoteScheduler.java:304)   在   java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:471)   在java.util.concurrent.FutureTask.run(FutureTask.java:262)at   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:178)   在   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)   在   java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)   在   java.util.concurrent.ThreadPoolExecutor中的$ Worker.run(ThreadPoolExecutor.java:615)   在java.lang.Thread.run(Thread.java:745)花了0秒。

我做错了什么想法?

1 个答案:

答案 0 :(得分:5)

Zeppelin 0.5.5的版本不适用于高于1.5.0的任何Apache spark版本,因此您需要从github下载Zeppelin 0.6.0并使用命令“mvn clean package -Pspark-1.5 -DskipTests”重新编译Zeppelin 0.5.5默认构建Spark 1.4。