我神秘的时间在哪里?

时间:2011-10-09 20:26:51

标签: ruby solr jetty rsolr

我有一个ruby脚本,使用rsolr ruby​​gem生成XML并将它们发布到Jetty Server浮出的Apache Solrjavadoc Update Command)。我的脚本使用以下代码记录特定时间

405       unless docs.empty?
406         begin
407           log.info("Adding to solr")
408           response = solr.add(docs)
409           log.info("#{(id_2*100.0)/last_id}% Done")
410           if response['responseHeader']['status'] != 0
411             log.fatal("Document ids  not sent")
412             #log.fatal(Solr::Request::AddDocument.new(docs_single).to_s)
413             log.close
414             exit
415           end
416           log.info("#{Time.now.to_f - starttime}s to feed Solr. #{id_1} to #{id_2}")
417         rescue Exception => e
418           log.fatal("Document ids not sent => ")
419           #log.fatal(Solr::Request::AddDocument.new(docs_single).to_s)
420           #log.fatal(docs)
421           log.close   
422           exit
423         end

生成的日志与

类似
I, [2011-10-09T15:03:42.617048 #30092]  INFO -- : Executing - SELECT * FROM solr_feeddata_2 WHERE id >= 5879999 AND id < 5881999
I, [2011-10-09T15:03:44.086661 #30092]  INFO -- : External Data1 fetch time: 1.45462989807129
I, [2011-10-09T15:03:44.109514 #30092]  INFO -- : External Data2 fetch time: 0.0226790904998779
I, [2011-10-09T15:03:44.109611 #30092]  INFO -- : 1.49255704879761s to fetch details from database. 5879999 to 5881999
I, [2011-10-09T15:03:44.109702 #30092]  INFO -- : Adding data1, data2, building docs
I, [2011-10-09T15:03:45.912603 #30092]  INFO -- : 3.29554414749146s to build documents. 5879999 to 5881999
I, [2011-10-09T15:03:45.912730 #30092]  INFO -- : Adding to solr
I, [2011-10-09T15:04:24.797620 #30092]  INFO -- : 61.180855194502% Done
I, [2011-10-09T15:04:24.797744 #30092]  INFO -- : 42.180694103241s to feed Solr. 5879999 to 5881999

根据这个日志,Solr采取(42.18 - 3.29 - 1.49 - 2)35.4s作出回应。 (见下面的评论)

同时,此特定更新的Solr日志与

类似
INFO: {add=[5879999, 5880000, 5880001, 5880002, 5880003, 5880004, 5880005, 5880007, ... (1468 adds)]} 0 5780
Oct 9, 2011 3:04:24 PM org.apache.solr.core.SolrCore execute
INFO: [core0] webapp=/solr path=/update params={wt=ruby} status=0 QTime=5780 
Oct 9, 2011 3:04:42 PM org.apache.solr.update.processor.LogUpdateProcessor finish

这清楚地表明,Solr花了5.78秒来添加文档,启动响应发送并关闭日志更新程序。

这两项服务都在同一台机器上运行,在网络内运行,并且它们的ping摘要是

rtt min/avg/max/mdev = 0.008/0.010/0.022/0.006 ms

对于处理的每个批处理数据,此模式清晰可见。尽管我真诚地努力解开这个谜团,但我无法理解这种行为。

我的Solr mergeFactor为10,autoCommit已关闭。

0 个答案:

没有答案