Question

我使用tar球安装了Cassandra-2.2.7。 Cassandra工作正常，我在一个密钥空间创建了表。现在我想将此表数据传输到HDFS。我正在使用Sqoop-1.4.6。我将以下库文件保存在$ SQOOP_HOME / lib /，

下

apache-cassandra-2.2.7.jar
apache-cassandra-thrift-2.2.7.jar
cassandra-jdbc-1.2.5.jar
cassandra-all-1.2.0.jar
libthrift-0.8.0.jar
thrift-server-0.3.7.jar

然后我运行了命令，

bin/sqoop list-tables --driver org.apache.cassandra.cql.jdbc.CassandraDriver --connect jdbc:cassandra://127.0.0.1:7199/sample --username cassandra --password cassandra

当我运行此命令时，我收到以下错误，

Warning: /usr/lib/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/08/15 16:52:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
16/08/15 16:52:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/08/15 16:52:41 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
16/08/15 16:52:41 INFO manager.SqlManager: Using default fetchSize of 1000
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/hadoop-1.2.1/lib/sf2hadoop.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/hadoop-1.2.1/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
16/08/15 16:52:41 ERROR manager.SqlManager: Error reading database metadata: java.sql.SQLNonTransientConnectionException: org.apache.thrift.transport.TTransportException
java.sql.SQLNonTransientConnectionException: org.apache.thrift.transport.TTransportException
	at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:159)
	at org.apache.cassandra.cql.jdbc.CassandraDriver.connect(CassandraDriver.java:92)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:885)
	at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
	at org.apache.sqoop.manager.SqlManager.listTables(SqlManager.java:520)
	at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: org.apache.thrift.transport.TTransportException
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
	at org.apache.cassandra.thrift.Cassandra$Client.recv_describe_cluster_name(Cassandra.java:1247)
	at org.apache.cassandra.thrift.Cassandra$Client.describe_cluster_name(Cassandra.java:1235)
	at org.apache.cassandra.cql.jdbc.CassandraConnection.<init>(CassandraConnection.java:130)
	... 13 more
Could not retrieve tables list from server
16/08/15 16:52:41 ERROR tool.ListTablesTool: manager.listTables() returned null

我认为这是某种Thrift错误，但即使我用其他几个版本替换lib-thrift jar，也无法纠正它。

Answer 1

Cassandra JDBC驱动程序很老了。上次我尝试使用它时，我需要针对最新的cassandra thrift库手动编译它并修复一些编译错误。

我建议使用Spark作业来传输数据，或者如果你必须使用sqoop，这取决于你正在做什么DataStax Enterprise包含一个用于Cassandra的自定义sqoop驱动程序。

尝试使用Sqoop将数据从Cassandra传输到Hadoop时出错

1 个答案: