Question

我正在尝试使用hadoop map-reduce运行分析，而不是存储在cassandra中的数据。为此，我使用maven依赖项CqlInputFormat中提供的类cassandra-all。目前，我们一直在生产环境中使用此依赖项的2.0.10版本。此外，我们正在使用版本为2.1.1的caassandra-driver-core。

现在，当我向jobtracker提交一个简单的map-reduce作业时，我的所有mapper任务都会因以下异常而失败。

另一个需要注意的重要事项是，如果我使用2.0 CqlPagingInputFormat中提供的cassandra-all，一切正常。但是此输入格式已在更高版本中删除。

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)))
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:258)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:267)
    at org.apache.cassandra.hadoop.cql3.CqlRecordReader.initialize(CqlRecordReader.java:137)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)))
    at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103)
    at com.datastax.driver.core.SessionManager.execute(SessionManager.java:446)
    at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:482)
    at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:88)
    at com.datastax.driver.core.AbstractSession.executeAsync(AbstractSession.java:60)
    at com.datastax.driver.core.Cluster.connect(Cluster.java:260)
    ... 9 more

欢迎任何意见。

Answer 1

使用cassandra-all-2.0.10时遇到了完全相同的问题，但使用版本cassandra-all-2.0.12解决了问题。

我认为更正是在此提交中： Potentially use non-local hosts in CqlConfigHelper

Hadoop映射作业因com.datastax.driver.core.exceptions.NoHostAvailableException而失败

1 个答案: