使用Pelops在Cassandra上传批量数据

时间:2012-04-03 12:20:56

标签: cassandra socketexception pelops

我正在使用Pelops与Cassandra合作。我想使用mutator插入批量数据。当我插入大约20000条记录时,它工作正常,但是当我超过记录的数量时,它会显示“通过对等方重置连接:套接字写入错误”。我正在编写以下代码。

Pelops.addPool(CASSANDRA_POOL, new Cluster(CASSANDRA_HOST,
            CASSANDRA_PORT), CASSANDRA_KEYSPACE);
Mutator mutator = Pelops.createMutator(CASSANDRA_POOL);

//writing cols in mutator
mutator.execute(CL_ONE)

但它适用于一些有限的记录。

我收到以下错误:

2757 17:42:01,211 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Attempting to borrow free connection for node 'localhost'

2757 17:42:01,211 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Borrowing connection 'Connection[MyKeyspace][localhost:9160][44937684]'

2977 17:42:01,431 WARN  org.scale7.cassandra.pelops.Operand - Operation failed as result of network exception. Connection to node localhost is being marked as corrupt (and will probably be be destroyed). Cause of failure is org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset by peer: socket write error

2977 17:42:01,431 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Returned connection 'Connection[MyKeyspace][localhost:9160][44937684]' has been closed or is marked as corrupt

2977 17:42:01,431 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Destroying connection 'Connection[MyKeyspace][localhost:9160][44937684]'

Exception in thread "main" org.scale7.cassandra.pelops.exceptions.TransportException: java.net.SocketException: Connection reset by peer: socket write error
    at org.scale7.cassandra.pelops.exceptions.IExceptionTranslator$ExceptionTranslator.translate(IExceptionTranslator.java:57)
    at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:115)
    at org.scale7.cassandra.pelops.Mutator.execute(Mutator.java:102)
    at org.scale7.cassandra.pelops.Mutator.execute(Mutator.java:71)
    at org.brickred.Cassandra.insertData(Cassandra.java:182)
    at org.brickred.Test.main(Test.java:7)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset by peer: socket write error
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
    at org.apache.thrift.transport.TFramedTransport.flush(TFramedTransport.java:157)
    at org.apache.cassandra.thrift.Cassandra$Client.send_batch_mutate(Cassandra.java:1020)
    at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:1008)
    at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:97)
    at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:93)
    at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:81)
    ... 4 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(Unknown Source)
    at java.net.SocketOutputStream.write(Unknown Source)
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
    ... 10 more

0 个答案:

没有答案