Question

这是我到目前为止所做的步骤：

下载spark-1.4.1-bin-hadoop2.6.tgz
解压
.spark-1.4.1彬hadoop2.6 / sbin目录/ start-all.sh

Master正在工作，但是奴隶没有开始

这是输出：

[ec2-user@ip-172-31-24-107 ~]$ sudo ./spark-1.4.1-bin-hadoop2.6/sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to /home/ec2-user/spark-1.4.1-bin-hadoop2.6/sbin/../logs/spark-root-org.apache.spark.deploy.master.Master-1-ip-172-31-24-107.out
localhost: Permission denied (publickey).
[ec2-user@ip-172-31-24-107 ~]$

这是安全日志

Aug  9 00:09:30 ip-172-31-24-107 sudo: ec2-user : TTY=pts/0 ; PWD=/home/ec2-user ; USER=root ; COMMAND=./spark-1.4.1-bin-hadoop2.6/sbin/start-all.sh
Aug  9 00:09:32 ip-172-31-24-107 sshd[4828]: Connection closed by 127.0.0.1 [preauth]

我认为问题出在SSH上，但我无法在谷歌上找到解决方案......

知道如何解决我的SSH问题吗？

Answer 1

您需要设置无密码ssh。尝试：

 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

然后重新启动群集。如果这不起作用，请发布新的错误消息。

Answer 2

它实际上是一个两步过程;

生成公共/私有rsa密钥对。

ubuntu@master:~$ ssh-keygen -t rsa -P ""

Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): 
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
80:4d:40:f6:3a:09:32:07:74:25:cc:cd:f2:b3:75:10 ubuntu@master.flexilogix
The key's randomart image is:
+--[ RSA 2048]----+
|o.o+Bo.E.        |
| ..=.B .         |
|o o + + .        |
| + . = o .       |
|    + + S        |
|     o           |
|                 |
|                 |
|                 |
+-----------------+

设置无密码ssh;

ubuntu@master:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

完成这两个步骤后，您应该能够验证它;

    ubuntu@master:~$ ssh localhost

如何在独立环境中启动apache-spark slave实例？

2 个答案: