我们的堆栈使用SOLR（通过Ruby太阳黑子宝石）。我们还有一个脚本，可以在SOLR上激发成千上万的搜索条件，看看哪些搜索条件得分很高并且消除了其他搜索条件。该脚本设置为使用24个线程运行，但许多请求因url failed. HTTP Error 502: Bad Gateway而失败。
显然它曾经从头到尾经历过没有问题 - 但这是在我开展项目之前。现在，它唯一的工作方式是使用一个线程，但这需要太长时间。因为我从来没有见过它，所以我不知道造成这个问题的原因。
我的问题是;我可以采取哪些步骤来调试和修复？感谢
[编辑]
完整错误：
Url failed: https://example.com/solr/example-sunspot/select?q=%2B%22Boat%22%20%2B%22Best%22%20%2B%22Plush%22%20%2B%22Melissa%20%26%20Doug%22%20%2Bname_texts%3A%22Toy%22%7E4%5E4%20-name_texts%3A%22wardrobe%22%20-name_texts%3A%22cupboard%22%20-name_texts%3A%22drawers%22%20-name_texts%3A%22box%22%20-name_texts%3A%22footmuff%22%20-name_texts%3A%22buggy%22%20-name_texts%3A%22step%20stool%22%20-name_texts%3A%22sleeping%20bag%22%20-name_texts%3A%22bouncer%22%20-name_texts%3A%22bedding%22%20-name_texts%3A%22cot%22%20-name_texts%3A%22changing%20bag%22%20-name_texts%3A%22room%20set%22%20-name_texts%3A%22book%22%20-name_texts%3A%22feeding%22%20-name_texts%3A%22storage%22&fq=type%3AExternalProduct&fq=available_b%3Atrue&fl=*+score&wt=json&defType=edismax&qf=name_texts+description_text+non_stem_name_text&pf=name_texts%5E2.0&stopwords=true&lowercaseOperators=true&rows=1&scoreDist=true (HTTP Error 502: Bad Gateway)
/var/log/solr/stderr.log：
log4j:ERROR Failed to flush writer,
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.log(Log4jLoggerAdapter.java:601)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:620)
at org.eclipse.jetty.util.log.JettyAwareLogger.info(JettyAwareLogger.java:314)
at org.eclipse.jetty.util.log.Slf4jLog.info(Slf4jLog.java:74)
at org.eclipse.jetty.server.Server.doStart(Server.java:270)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1259)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:473)
at org.eclipse.jetty.start.Main.start(Main.java:615)
at org.eclipse.jetty.start.Main.main(Main.java:96)
/var/log/nginx/error.log：
2017/03/15 11:44:10 [error] 1219#0: *10797360 connect() failed (111: Connection refused) while connecting to upstream, client: 95.146.105.87, server: solr-production.cahootsy.com, request: "GET /solr/caho
otsy-sunspot/select?q=%2B%22Toddler%22%20%22Cute%22%5E4%20%2B%22Comfort%22%20%2B%22Ragtales%22%20%2Bname_texts%3A%22Teddy%20Bear%22%7E4%5E4%20-name_texts%3A%22lingerie%22%20-name_texts%3A%22wallpaper%22%2
0-name_texts%3A%22box%22%20-name_texts%3A%22dress%22%20-name_texts%3A%22quilt%22%20-name_texts%3A%22acrylic%22%20-name_texts%3A%22ribbon%22%20-name_texts%3A%22footmuff%22%20-name_texts%3A%22buggy%22%20-na
me_texts%3A%22step%20stool%22%20-name_texts%3A%22sleeping%20bag%22%20-name_texts%3A%22bouncer%22%20-name_texts%3A%22changing%20bag%22%20-name_texts%3A%22room%20set%22%20-name_texts%3A%22feeding%22%20-name_texts%3A%22storage%22&fq=type%3AExternalProduct&fq=available_b%3Atrue&fl=*+score&wt=json&defType=edismax&qf=name_texts+description_text+non_stem_name_text&pf=name_texts%5E2.0&stopwords=true&lowercaseOperators=true&rows=1&scoreDist=true HTTP/1.1", upstream: "http://127.0.0.1:8983/solr/cahootsy-sunspot/select?q=%2B%22Toddler%22%20%22Cute%22%5E4%20%2B%22Comfort%22%20%2B%22Ragtales%22%20%2Bname_texts%3A%22Teddy%20Bear%22%7E4%5E4%20-name_texts%3A%22lingerie%22%20-name_texts%3A%22wallpaper%22%20-name_texts%3A%22box%22%20-name_texts%3A%22dress%22%20-name_texts%3A%22quilt%22%20-name_texts%3A%22acrylic%22%20-name_texts%3A%22ribbon%22%20-name_texts%3A%22footmuff%22%20-name_texts%3A%22buggy%22%20-name_texts%3A%22step%20stool%22%20-name_texts%3A%22sleeping%20bag%22%20-name_texts%3A%22bouncer%22%20-name_texts%3A%22changing%20bag%22%20-name_texts%3A%22room%20set%22%20-name_texts%3A%22feeding%22%20-name_texts%3A%22storage%22&fq=type%3AExternalProduct&fq=available_b%3Atrue&fl=*+score&wt=json&defType=edismax&qf=name_texts+description_text+non_stem_name_text&pf=name_texts%5E2.0&stopwords=true&lowercaseOperators=true&rows=1&scoreDist=true", host: "solr-production.cahootsy.com"
/var/log/solr/stdout.log：
908831 [qtp24433162-47] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 4803295 (1562039497978281984)]} 0 1 908899 [qtp24433162-55] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626369 (1562039498050633728)]} 0 0
908947 [qtp24433162-13] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3634992 (1562039498100965376)]} 0 0
908971 [qtp24433162-34] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626370 (1562039498126131200)]} 0 0
909008 [qtp24433162-49] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3631728 (1562039498164928512)]} 0 1
909047 [qtp24433162-75] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626371 (1562039498205822976)]} 0 1
909102 [qtp24433162-39] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3645291 (1562039498263494656)]} 0 0
909152 [qtp24433162-18] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626372 (1562039498315923456)]} 0 0
909202 [qtp24433162-41] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3632336 (1562039498368352256)]} 0 0
909252 [qtp24433162-36] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626373 (1562039498420781056)]} 0 0
909308 [qtp24433162-35] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3643432 (1562039498479501312)]} 0 0
909363 [qtp24433162-20] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3631978 (1562039498537172992)]} 0 0
909415 [qtp24433162-51] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626374 (1562039498591698944)]} 0 0
909467 [qtp24433162-66] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626375 (1562039498646224896)]} 0 0
909527 [qtp24433162-54] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3646560 (1562039498709139456)]} 0 0
909552 [qtp24433162-30] INFO org.apache.solr.core.SolrCore ? UnInverted multi-valued field {field=category_ids_im,memSize=21348590,tindexSize=178,time=148,phase1=141,nTerms=1293,bigTerms=1,termInstances=4492323,uses=0}
909566 [qtp24433162-30] INFO org.apache.solr.core.SolrCore ? UnInverted multi-valued field {field=collection_ids_im,memSize=20225677,tindexSize=470,time=13,phase1=9,nTerms=4121,bigTerms=0,termInstances=17715,uses=0}
909598 [qtp24433162-71] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626376 (1562039498783588352)]} 0 0
909623 [qtp24433162-65] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3634957 (1562039498809802752)]} 0 0
909657 [qtp24433162-60] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626377 (1562039498845454336)]} 0 0
909710 [qtp24433162-38] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3631099 (1562039498899980288)]} 0 1
909760 [qtp24433162-44] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626378 (1562039498953457664)]} 0 0
909803 [qtp24433162-42] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3635001 (1562039498998546432)]} 0 0
909869 [qtp24433162-46] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3635754 (1562039499067752448)]} 0 0
909923 [qtp24433162-64] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626379 (1562039499123326976)]} 0 1
909944 [qtp24433162-30] INFO org.apache.solr.core.SolrCore ? UnInverted multi-valued field {field=community_ids_ims,memSize=20188632,tindexSize=44,time=10,phase1=7,nTerms=6,bigTerms=0,termInstances=2522,uses=0}
909952 [qtp24433162-67] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626380 (1562039499154784256)]} 0 0
909974 [qtp24433162-30] INFO org.apache.solr.core.SolrCore ? UnInverted multi-valued field {field=inferred_community_ids_ims,memSize=20188920,tindexSize=44,time=29,phase1=26,nTerms=6,bigTerms=3,termInstances=122033,uses=0}
909975 [qtp24433162-30] INFO org.apache.solr.core.SolrCore ? [cahootsy-sunspot] webapp=/solr path=/select params={f.inferred_community_ids_ims.facet.mincount=1&facet.field=category_ids_im&facet.field=collection_ids_im&facet.field=colour_s&facet.field=brand_ss&facet.field=retailer_ss&facet.field=community_ids_ims&facet.field=inferred_community_ids_ims&f.retailer_ss.facet.mincount=0&start=0&f.category_ids_im.facet.mincount=0&fq=type:ExternalProduct&fq=is_retailer_b:true&fq=-retailer_normalized_s:(asos)&fq=available_b:true&fq=category_ids_im:(2304)&sort=cahootsy_score_i+desc,+random_1632017+asc&rows=16&f.brand_ss.facet.mincount=0&f.community_ids_ims.facet.mincount=1&q=*:*&f.collection_ids_im.facet.mincount=1&f.category_ids_im.facet.limit=-1&wt=ruby&facet=true&f.colour_s.facet.mincount=0} hits=18140 status=0 QTime=636
910006 [qtp24433162-29] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3630950 (1562039499211407360)]} 0 0
910048 [qtp24433162-28] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626381 (1562039499255447552)]} 0 0
910098 [qtp24433162-58] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 3634841 (1562039499307876352)]} 0 0
910167 [qtp24433162-63] INFO org.apache.solr.update.processor.LogUpdateProcessor ? [cahootsy-sunspot] webapp=/solr path=/update params={wt=ruby} {add=[ExternalProduct 2626382 (1562039499380228096)]} 0 0
910211 [qtp24433162-59] INFO org.apache.solr.core.SolrCore ? [cahootsy-sunspot] webapp=/solr path=/select params={f.inferred_community_ids_ims.facet.mincount=1&facet.field=category_ids_im&facet.field=collection_ids_im&facet.field=colour_s&facet.field=brand_ss&facet.field=retailer_ss&facet.field=community_ids_ims&facet.field=inferred_community_ids_ims&f.retailer_ss.facet.mincount=0&start=0&f.category_ids_im.facet.mincount=0&fq=type:ExternalProduct&fq=is_retailer_b:true&fq=retailer_normalized_s:(asos)&fq=available_b:true&fq=category_ids_im:(2304)&sort=cahootsy_score_i+desc,+random_1632017+asc&rows=6&f.brand_ss.facet.mincount=0&f.community_ids_ims.facet.mincount=1&q=*:*&f.collection_ids_im.facet.mincount=1&f.category_ids_im.facet.limit=-1&wt=ruby&facet=true&f.colour_s.facet.mincount=0} hits=15791 status=0 QTime=5
[编辑2]
我尝试拖尾/var/log/solr/stdout.log，同时重现问题并在日志中找到：
8846 [commitScheduler-9-thread-1] WARN org.apache.solr.core.SolrCore ? [cahootsy-sunspot] Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later.
8848 [commitScheduler-9-thread-1] ERROR org.apache.solr.update.CommitTracker ? auto commit error...:org.apache.solr.common.SolrException: Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later.
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1672)
at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:607)
at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)