从本地强调EC2上的Cassandra实例

时间：2015-02-05 14:06:23

标签： amazon-ec2 cassandra datastax-enterprise

对于如何从我的本地计算机（使用cassandra-stress util）强调在EC2上运行的Cassandra实例，我将不胜感激。

  • EC2上的群集：运行DSE 4.6的五个节点。
  • 本地机器：cassandra-stress，包含在Cassandra 2.1.2中。

更改Security Group后，从本地计算机调用的stress util能够连接到EC2上的给定实例。

我允许来自本地计算机IP的端口9160和9042上的入站TCP连接。

sh cassandra-stress write -node 54.xxx.197.xxx

输出是：

ec0007:bin planger$ sh cassandra-stress write -node 54.xxx.197.xxx
Unable to create stress keyspace: Keyspace names must be case-insensitively unique ("Keyspace1" conflicts with "Keyspace1")
Warming up WRITE with 50000 iterations...
WARN  13:45:52 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.33) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:52 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.34) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:52 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.35) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:52 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.32) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:53 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.33) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:53 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.34) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:53 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.35) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
WARN  13:45:53 Found host with 0.0.0.0 as rpc_address, using listen_address (/172.31.33.32) to contact it instead. If this is incorrect you should avoid the use of 0.0.0.0 server side.
INFO  13:45:53 Using data-center name 'Cassandra' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
INFO  13:45:53 New Cassandra host /172.31.33.33:9042 added
INFO  13:45:53 New Cassandra host /172.31.33.32:9042 added
Connected to cluster: esentriDSEcluster
Datatacenter: Cassandra; Host: /54.xxx.197.xxx; Rack: rack1
INFO  13:45:53 New Cassandra host /172.31.33.35:9042 added
Datatacenter: Analytics; Host: /172.31.33.35; Rack: rack1
Datatacenter: Solr; Host: /172.31.33.34; Rack: rack1
Datatacenter: Cassandra; Host: /172.31.33.33; Rack: rack1
Datatacenter: Cassandra; Host: /172.31.33.32; Rack: rack1
INFO  13:45:53 New Cassandra host /172.31.33.34:9042 added
INFO  13:45:53 New Cassandra host /54.171.197.133:9042 added
ERROR 13:45:58 Error creating pool to /172.31.33.33:9042
com.datastax.driver.core.TransportException: [/172.31.33.33:9042] Cannot connect
    at com.datastax.driver.core.Connection.<init>(Connection.java:104) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:28) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.Connection$Factory.open(Connection.java:458) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:85) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:241) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:42) ~[cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:273) [cassandra-driver-core-2.0.5.jar:na]
    at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:265) [cassandra-driver-core-2.0.5.jar:na]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_55]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: /172.31.33.33:9042
    at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137) ~[netty-3.9.0.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) ~[netty-3.9.0.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ~[netty-3.9.0.Final.jar:na]
    at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.9.0.Final.jar:na]
    ... 3 common frames omitted

问题似乎是联系的EC2实例返回其他四个实例的私有IP。怎么改变？我认为它应该返回公共IP。

EC2节点上的cassandra.yaml配置如下所示：

seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "172.31.33.33,172.31.33.35,172.31.33.34"

listen_address: 172.31.33.32  # (different on every node)

# broadcast_address: 

start_rpc: true
rpc_address: 0.0.0.0
rpc_port: 9160

更改种子或指定broadcast_address会破坏群集。

提前感谢您的帮助。

更新1

我通过将其值设置为节点的公共IP地址来更改每个节点上的rpc_address。然后重新启动每个节点。 

sudo service dse stop
sudo service dse start

但这打破了群集。 DSE启动后的短时间内，节点将再次关闭。

来自/var/log/cassandra/system.log 

 INFO [main] 2015-02-06 09:54:06,128 CassandraDaemon.java (line 135) Logging initialized
 INFO [main] 2015-02-06 09:54:06,156 DseDaemon.java (line 382) DSE version: 4.6.0
 INFO [main] 2015-02-06 09:54:06,157 DseDaemon.java (line 383) Hadoop version: 1.0.4.13
 INFO [main] 2015-02-06 09:54:06,157 DseDaemon.java (line 384) Hive version: 0.12.0.5
 INFO [main] 2015-02-06 09:54:06,158 DseDaemon.java (line 385) Pig version: 0.10.1
 INFO [main] 2015-02-06 09:54:06,159 DseDaemon.java (line 386) Solr version: 4.6.0.3.3
 INFO [main] 2015-02-06 09:54:06,159 DseDaemon.java (line 387) Sqoop version: 1.4.4.14.2
 INFO [main] 2015-02-06 09:54:06,160 DseDaemon.java (line 388) Mahout version: 0.8
 INFO [main] 2015-02-06 09:54:06,160 DseDaemon.java (line 389) Appender version: 3.1.0
 INFO [main] 2015-02-06 09:54:06,161 DseDaemon.java (line 390) Spark version: 1.1.0.2
 INFO [main] 2015-02-06 09:54:06,161 DseDaemon.java (line 391) Shark version: 1.1.0
 INFO [main] 2015-02-06 09:54:06,416 DseConfig.java (line 345) Loading settings from file:/etc/dse/dse.yaml
 INFO [main] 2015-02-06 09:54:06,605 DseConfig.java (line 385) Load of settings is done.
 INFO [main] 2015-02-06 09:54:06,611 DseConfig.java (line 409) CQL slow log is enabled
 INFO [main] 2015-02-06 09:54:06,611 DseConfig.java (line 410) CQL system info tables are not enabled
 INFO [main] 2015-02-06 09:54:06,611 DseConfig.java (line 411) Resource level latency tracking is not enabled
 INFO [main] 2015-02-06 09:54:06,613 DseConfig.java (line 412) Database summary stats are not enabled
 INFO [main] 2015-02-06 09:54:06,614 DseConfig.java (line 413) Cluster summary stats are not enabled
 INFO [main] 2015-02-06 09:54:06,614 DseConfig.java (line 414) Histogram data tables are not enabled
 INFO [main] 2015-02-06 09:54:06,614 DseConfig.java (line 415) User level latency tracking is not enabled
 INFO [main] 2015-02-06 09:54:06,614 DseConfig.java (line 416) Solr latency snapshots are not enabled
 INFO [main] 2015-02-06 09:54:06,615 DseConfig.java (line 417) Solr slow sub-query log is not enabled
 INFO [main] 2015-02-06 09:54:06,615 DseConfig.java (line 418) Solr indexing error log is not enabled
 INFO [main] 2015-02-06 09:54:06,615 DseConfig.java (line 419) Solr update handler metrics are not enabled
 INFO [main] 2015-02-06 09:54:06,615 DseConfig.java (line 420) Solr request handler metrics are not enabled
 INFO [main] 2015-02-06 09:54:06,615 DseConfig.java (line 421) Solr index statistics reporting is not enabled
 INFO [main] 2015-02-06 09:54:06,616 DseConfig.java (line 422) Solr cache statistics reporting is not enabled
 INFO [main] 2015-02-06 09:54:06,629 YamlConfigurationLoader.java (line 80) Loading settings from file:/etc/dse/cassandra/cassandra.yaml
 INFO [main] 2015-02-06 09:54:06,683 DatabaseDescriptor.java (line 143) Data files directories: [/raid0/cassandra/data]
 INFO [main] 2015-02-06 09:54:06,683 DatabaseDescriptor.java (line 144) Commit log directory: /raid0/cassandra/commitlog
 INFO [main] 2015-02-06 09:54:06,683 DatabaseDescriptor.java (line 184) DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
 INFO [main] 2015-02-06 09:54:06,683 DatabaseDescriptor.java (line 198) disk_failure_policy is stop
 INFO [main] 2015-02-06 09:54:06,684 DatabaseDescriptor.java (line 199) commit_failure_policy is stop
 INFO [main] 2015-02-06 09:54:06,690 DatabaseDescriptor.java (line 269) Global memtable threshold is enabled at 930MB
 INFO [main] 2015-02-06 09:54:06,692 Workload.java (line 80) Setting my workload to Cassandra
 INFO [main] 2015-02-06 09:54:06,693 DseDelegateSnitch.java (line 43) Initialized DseDelegateSnitch with workload Cassandra, delegating to com.datastax.bdp.snitch.DseSimpleSnitch
 INFO [main] 2015-02-06 09:54:06,830 DatabaseDescriptor.java (line 408) Not using multi-threaded compaction
 INFO [main] 2015-02-06 09:54:07,075 YamlConfigurationLoader.java (line 80) Loading settings from file:/etc/dse/cassandra/cassandra.yaml
 INFO [main] 2015-02-06 09:54:07,090 YamlConfigurationLoader.java (line 80) Loading settings from file:/etc/dse/cassandra/cassandra.yaml
 INFO [main] 2015-02-06 09:54:07,150 PerformanceObjectsController.java (line 321) ClusterSummaryStats plugin using 4 async writers
 INFO [main] 2015-02-06 09:54:07,150 PerformanceObjectsController.java (line 355) ClusterSummaryStats refresh rate set to 10000 (was 0)
 INFO [main] 2015-02-06 09:54:07,151 PerformanceObjectsController.java (line 321) CqlSystemInfo plugin using 1 async writers
 INFO [main] 2015-02-06 09:54:07,151 PerformanceObjectsController.java (line 355) CqlSystemInfo refresh rate set to 10000 (was 0)
 INFO [main] 2015-02-06 09:54:07,152 PerformanceObjectsController.java (line 321) DbSummaryStats plugin using 4 async writers
 INFO [main] 2015-02-06 09:54:07,152 PerformanceObjectsController.java (line 355) DbSummaryStats refresh rate set to 10000 (was 0)
 INFO [main] 2015-02-06 09:54:07,152 PerformanceObjectsController.java (line 321) HistogramDataTables plugin using 4 async writers
 INFO [main] 2015-02-06 09:54:07,153 PerformanceObjectsController.java (line 355) HistogramDataTables refresh rate set to 10000 (was 0)
 INFO [main] 2015-02-06 09:54:07,153 PerformanceObjectsController.java (line 321) ResourceLatencyTracking plugin using 4 async writers
 INFO [main] 2015-02-06 09:54:07,153 PerformanceObjectsController.java (line 355) ResourceLatencyTracking refresh rate set to 10000 (was 0)
 INFO [main] 2015-02-06 09:54:07,154 PerformanceObjectsController.java (line 321) UserLatencyTracking plugin using 1 async writers
 INFO [main] 2015-02-06 09:54:07,154 PerformanceObjectsController.java (line 355) UserLatencyTracking refresh rate set to 10000 (was 0)
(…) 
 INFO [main] 2015-02-06 09:54:14,717 StorageService.java (line 514) Cassandra version: 2.0.11.83
 INFO [main] 2015-02-06 09:54:14,718 StorageService.java (line 515) Thrift API version: 19.39.0
 INFO [main] 2015-02-06 09:54:14,722 StorageService.java (line 516) CQL supported versions: 2.0.0,3.1.7 (default: 3.1.7)
 INFO [main] 2015-02-06 09:54:14,745 StorageService.java (line 539) Loading persisted ring state
 INFO [main] 2015-02-06 09:54:14,758 StorageService.java (line 677) Starting up server gossip
(…)
 INFO [main] 2015-02-06 09:54:14,885 MessagingService.java (line 473) Starting Messaging Service on port 7000
 INFO [main] 2015-02-06 09:54:14,917 YamlConfigurationLoader.java (line 80) Loading settings from file:/etc/dse/cassandra/cassandra.yaml
(…)
 INFO [main] 2015-02-06 09:54:14,977 StorageService.java (line 1521) Node /172.31.33.33 state jump to normal
 INFO [main] 2015-02-06 09:54:14,987 CassandraDaemon.java (line 543) Waiting for gossip to settle before accepting client requests...
 INFO [main] 2015-02-06 09:54:22,988 CassandraDaemon.java (line 575) No gossip backlog; proceeding
 INFO [main] 2015-02-06 09:54:23,011 AuditLogger.java (line 32) Audit logging is disabled
 INFO [main] 2015-02-06 09:54:23,031 EndpointStatePersister.java (line 56) EndpointStatePersister started
 WARN [main] 2015-02-06 09:54:23,031 Workload.java (line 100) Couldn't determine workload for /172.31.33.35 from value NULL
 WARN [main] 2015-02-06 09:54:23,036 Workload.java (line 100) Couldn't determine workload for /172.31.33.32 from value NULL
 WARN [main] 2015-02-06 09:54:23,036 Workload.java (line 100) Couldn't determine workload for /172.31.33.36 from value NULL
 WARN [main] 2015-02-06 09:54:23,037 Workload.java (line 100) Couldn't determine workload for /172.31.33.34 from value NULL
 INFO [main] 2015-02-06 09:54:23,038 EndpointStateTracker.java (line 80) EndpointStateTracker started
 INFO [main] 2015-02-06 09:54:23,041 DseDaemon.java (line 441) Waiting for other nodes to become alive...
 WARN [main] 2015-02-06 09:54:33,827 DseDaemon.java (line 444) The following nodes seems to be down: [/172.31.33.35, /172.31.33.32, /172.31.33.36, /172.31.33.34]. Some Cassandra operations may fail with UnavailableException.
 INFO [main] 2015-02-06 09:54:33,827 DseDaemon.java (line 454) Wait for nodes completed
 INFO [main] 2015-02-06 09:54:33,836 PluginManager.java (line 262) Activating plugin: com.datastax.bdp.plugin.DseSystemPlugin
 INFO [main] 2015-02-06 09:54:33,837 PluginManager.java (line 344) Plugin activated: com.datastax.bdp.plugin.DseSystemPlugin
 INFO [main] 2015-02-06 09:54:33,837 PluginManager.java (line 262) Activating plugin: com.datastax.bdp.leases.PeriodicTaskOwnershipPlugin
 INFO [main] 2015-02-06 09:54:33,842 PluginManager.java (line 344) Plugin activated: com.datastax.bdp.leases.PeriodicTaskOwnershipPlugin
 INFO [main] 2015-02-06 09:54:33,919 Server.java (line 156) Starting listening for CQL clients on /54.xxx.180.xxx:9042...
ERROR [main] 2015-02-06 09:54:33,934 DseDaemon.java (line 492) Unable to start DSE server.
org.jboss.netty.channel.ChannelException: **Failed to bind to**: /**54.xxx.180.xxx:9042**
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
    at org.apache.cassandra.transport.Server.run(Server.java:157)
    at org.apache.cassandra.transport.Server.start(Server.java:108)
    at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:443)
    at com.datastax.bdp.server.DseDaemon.start(DseDaemon.java:486)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:509)
    at com.datastax.bdp.server.DseDaemon.main(DseDaemon.java:659)
Caused by: java.net.BindException: Cannot assign requested address
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:444)
    at sun.nio.ch.Net.bind(Net.java:436)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
    at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
 INFO [Thread-2] 2015-02-06 09:54:33,937 DseDaemon.java (line 505) DSE shutting down...
 INFO [StorageServiceShutdownHook] 2015-02-06 09:54:33,941 Gossiper.java (line 1307) Announcing shutdown
 INFO [Thread-2] 2015-02-06 09:54:33,954 PluginManager.java (line 304) Deactivating plugin: com.datastax.bdp.leases.PeriodicTaskOwnershipPlugin
 INFO [Thread-2] 2015-02-06 09:54:33,954 PluginManager.java (line 304) Deactivating plugin: com.datastax.bdp.plugin.DseSystemPlugin
 INFO [Thread-2] 2015-02-06 09:54:33,954 PluginManager.java (line 356) All plugins are stopped.
 INFO [Thread-2] 2015-02-06 09:54:33,954 CassandraDaemon.java (line 463) Cassandra shutting down...
 INFO [StorageServiceShutdownHook] 2015-02-06 09:54:35,945 MessagingService.java (line 701) Waiting for messaging service to quiesce
 INFO [ACCEPT-/172.31.33.33] 2015-02-06 09:54:35,945 MessagingService.java (line 941) MessagingService has terminated the accept() thread

1 个答案:

答案 0 :(得分：0)

我认为压力就像任何DataStax驱动程序一样，并使用每个节点的RPC地址进行通信。目前你已经配置了0.0.0.0，这可能是它选择你的内部aws地址的原因。

Cassandra.yaml

中有三个地址可供使用

  1. 监听地址 - 这是其他Cassandra节点将用于与此节点通信的IP地址。您希望这是性能的内部AWS IP地址。

  2. RPC地址 - 这是您的客户端连接到的地址，如果您的客户端不在AWS或同一AWS区域中，则可能是您要配置以匹配外部AWS地址的地址。也适用于压力。

  3. 广播地址 - 如果您使用多个数据中心或AWS区域，则并非所有节点都可以通过内部IP相互访问。您可以为不同数据中心中的节点指定外部IP地址，但仍可以相互通信。在许多情况下，您根本不需要此设置，它将默认为您的监听地址。

    4. 如果有帮助，请告诉我。

相关问题
最新问题