Question

我正试图在datastax（4.6）上启动火花。当我执行./dse spark时，我得到了：

java.lang.SecurityException: Authentication failed! Credentials required
    at com.sun.jmx.remote.security.JMXPluggableAuthenticator.authenticationFailure(JMXPluggableAuthenticator.java:211)
    at com.sun.jmx.remote.security.JMXPluggableAuthenticator.authenticate(JMXPluggableAuthenticator.java:163)
    at sun.management.jmxremote.ConnectorBootstrap$AccessFileCheckerAuthenticator.authenticate(ConnectorBootstrap.java:219)
    at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:232)
    at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199)
    at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
    at sun.rmi.transport.Transport$1.run(Transport.java:177)
    at sun.rmi.transport.Transport$1.run(Transport.java:174)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
    at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
    at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
    at sun.rmi.server.UnicastRef.invoke(Unknown Source)
    at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
    at javax.management.remote.rmi.RMIConnector.getConnection(Unknown Source)
    at javax.management.remote.rmi.RMIConnector.connect(Unknown Source)
    at javax.management.remote.JMXConnectorFactory.connect(Unknown Source)
    at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:151)
    at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:107)
    at com.datastax.bdp.tools.DseTool.<init>(DseTool.java:92)
    at com.datastax.bdp.tools.DseTool.main(DseTool.java:1048)
Welcome to
    ____              __
   / __/__  ___ _____/ /__
  _\ \/ _ \/ _ `/ __/  '_/
 /___/ .__/\_,_/_/ /_/\_\   version 1.1.0
    /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_21)
Type in expressions to have them evaluated.
Type :help for more information.
Creating SparkContext...
Initializing SparkContext with MASTER: spark://master:7077
Created spark context..
Spark context available as sc.
15/05/18 09:27:03 WARN Remoting: Tried to associate with unreachable      remote address [akka.tcp://sparkMaster@master:7077]. Address is now gated for 60000 ms, all messages to this address will be delivered to dead letters.
15/05/18 09:27:03 WARN AppClient$ClientActor: Could not connect to     akka.tcp://sparkMaster@master:7077: akka.remote.InvalidAssociation: Invalid    address: akka.tcp://sparkMaster@master:7077
HiveSQLContext available as hc.
CassandraSQLContext available as csc.
Type in expressions to have them evaluated.
Type :help for more information.

scala> 15/05/18 09:28:03 ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up.
15/05/18 09:28:03 ERROR TaskSchedulerImpl: Exiting due to error from cluster scheduler: All masters are unresponsive! Giving up.

奇怪的是

1，我已禁用密码授权程序，但仍然有 java.lang.SecurityException：身份验证失败！需要凭证

2，我设置了SPARK_MASTER = my_hostname，这是与cassandra节点相同的主机，但仍然所有主人都没有响应

有人可以帮我这个吗？感谢

Answer 1

确保正确设置dse_auth键空间的复制因子：

http://docs.datastax.com/en/datastax_enterprise/4.0/datastax_enterprise/sec/secConfSysAuthKeyspRepl.html

程序

根据您的环境，根据以下示例之一设置复制因子： SimpleStrategy示例：

ALTER KEYSPACE "system_auth"
   WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };

NetworkTopologyStrategy示例：

ALTER KEYSPACE "system_auth"
   WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'dc1' : 3, 'dc2' : 2};

Answer 2

要通过身份验证启动dse上的spark，您还可以以这种方式传递凭据

dse spark -Dcassandra.username=user -Dcassandra.password=pass

无法启动datastax上的火花

2 个答案:

程序