我们有两个DSE节点(3.2.4)用于开发目的。 Cassandra密钥空间的复制因子设置为2.当通过查询执行Solr删除而不指定一致性级别(即:仲裁)时,在删除大量行(大约500K左右)时偶尔会出现以下异常。将一致性级别设置为1时,也会观察到相同的异常(删除大约100万行时)。我想知道是否有解决方法。例外情况如下:
WARN [http-8983-1] 2014-03-27 09:21:59,008 CassandraUtil.java (line 44) Error Operation timed out - received only 0 responses. on attempt 1 out of 4 with CL ONE...
WARN [http-8983-1] 2014-03-27 09:22:09,109 CassandraUtil.java (line 44) Error Operation timed out - received only 0 responses. on attempt 2 out of 4 with CL ONE...
WARN [http-8983-1] 2014-03-27 09:22:19,211 CassandraUtil.java (line 44) Error Operation timed out - received only 0 responses. on attempt 3 out of 4 with CL ONE...
WARN [http-8983-1] 2014-03-27 09:22:29,312 CassandraUtil.java (line 44) Error Operation timed out - received only 0 responses. on attempt 4 out of 4 with CL ONE...
ERROR [http-8983-1] 2014-03-27 09:22:29,415 SolrException.java (line 124) org.apache.solr.common.SolrException: java.lang.RuntimeException: Insert command failed after 4 attempts, source exception follows.
at com.datastax.bdp.search.solr.handler.update.CassandraDirectUpdateHandler.deleteByQuery(CassandraDirectUpdateHandler.java:236)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:80)
at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:899)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:757)
at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55)
at org.apache.solr.handler.loader.XMLLoader.processDelete(XMLLoader.java:346)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:277)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1841)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:669)
at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.execute(CassandraDispatchFilter.java:165)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:137)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:94)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Insert command failed after 4 attempts, source exception follows.
at com.datastax.bdp.util.CassandraUtil.robustInsert(CassandraUtil.java:56)
at com.datastax.bdp.search.solr.ThriftCassandraRowWriter.doDeletes(ThriftCassandraRowWriter.java:189)
at com.datastax.bdp.search.solr.ThriftCassandraRowWriter.access$100(ThriftCassandraRowWriter.java:39)
at com.datastax.bdp.search.solr.ThriftCassandraRowWriter$DeletingHitCollector.collect(ThriftCassandraRowWriter.java:241)
at org.apache.lucene.search.Scorer.score(Scorer.java:65)
at org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.score(ConstantScoreQuery.java:245)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
at com.datastax.bdp.search.solr.ThriftCassandraRowWriter.deleteByQuery(ThriftCassandraRowWriter.java:168)
at com.datastax.bdp.search.solr.handler.update.CassandraDirectUpdateHandler.deleteByQuery(CassandraDirectUpdateHandler.java:217)
... 38 more
Caused by: org.apache.cassandra.exceptions.WriteTimeoutException: Operation timed out - received only 0 responses.
at org.apache.cassandra.service.AbstractWriteResponseHandler.get(AbstractWriteResponseHandler.java:91)
at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:198)
at com.datastax.bdp.util.CassandraUtil.robustInsert(CassandraUtil.java:39)
... 47 more
答案 0 :(得分:0)
感谢您报告此问题。我们正在考虑并考虑改进。
目前,我所能推荐的只是在失败的情况下重试该操作,稍有延迟,因为问题可能只是群集正忙。
您可能还想考虑较小的删除,使用选择较少文档/行的较窄查询。
我将检查您是否可以合理地调整超时以处理群集延迟问题。
编辑:您可以使用write_request_timeout_in_ms在cassandra.yaml中提高超时