Solr 6.4.1更新很长时间

时间:2017-03-13 10:50:11

标签: performance insert-update solr6

Solr 6.4.1需要很长时间才能更新。我有Solr 6.4.1。已编入约600 000份文件。

当我进行更新时,大约需要20到60秒。阻止我的应用(网页)太长时间。

  • Solr Logs没有显示内存或其他内容不足的内容。
  • 搜索非常快。 (我在同一台机器上搜索和索引)
  • 没有很多搜索查询(可能是20分钟)
  • 这台机器上运行的postgresql运行solr。

我的机器:

1x Intel Quad-Core Xeon E3-1230V2 [ 4 Core(s) ]
RAM 16 GB
120 GB SSD

我的solrconfig.xml:

<?xml version="1.0" encoding="UTF-8" ?>
<config>
  <luceneMatchVersion>6.4.1</luceneMatchVersion>
  <lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-cell-\d.*\.jar" />

  <lib dir="${solr.install.dir:../../../..}/contrib/clustering/lib/" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-clustering-\d.*\.jar" />

  <lib dir="${solr.install.dir:../../../..}/contrib/langid/lib/" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-langid-\d.*\.jar" />

  <lib dir="${solr.install.dir:../../../..}/contrib/velocity/lib" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-velocity-\d.*\.jar" />
  <dataDir>${solr.data.dir:}</dataDir>
  <directoryFactory name="DirectoryFactory"
                    class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
  <codecFactory class="solr.SchemaCodecFactory"/>
  <indexConfig>
    <lockType>${solr.lock.type:native}</lockType>
  </indexConfig>
  <jmx />
  <updateHandler class="solr.DirectUpdateHandler2">
    <updateLog>
      <str name="dir">${solr.ulog.dir:}</str>
      <int name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int>
    </updateLog>
    <autoCommit>
      <maxDocs>10000</maxDocs> <!-- maximum uncommited docs before autocommit triggered -->
      <maxTime>600000</maxTime>
      <openSearcher>false</openSearcher>
    </autoCommit>

    <autoSoftCommit>
      <maxTime>15000</maxTime>
    </autoSoftCommit>
  </updateHandler>
  <query>
    <maxBooleanClauses>1024</maxBooleanClauses>
    <filterCache class="solr.FastLRUCache"
                 size="4096"
                 initialSize="1024"
                 autowarmCount="1024"/>
    <queryResultCache class="solr.LRUCache"
                      size="4096"
                      initialSize="1024"
                      autowarmCount="512"/>
    <documentCache class="solr.LRUCache"
                   size="4096"
                   initialSize="512"
                   autowarmCount="0"/>

    <!-- custom cache currently used by block join -->
    <cache name="perSegFilter"
           class="solr.search.LRUCache"
           size="10"
           initialSize="0"
           autowarmCount="10"
           regenerator="solr.NoOpRegenerator" />

    <enableLazyFieldLoading>true</enableLazyFieldLoading>
    <queryResultWindowSize>40</queryResultWindowSize>

    <!-- Maximum number of documents to cache for any entry in the
         queryResultCache. 
      -->
    <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
    <listener event="newSearcher" class="solr.QuerySenderListener">
      <arr name="queries">
        <!--
           <lst><str name="q">solr</str><str name="sort">price asc</str></lst>
           <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>
          -->
      </arr>
    </listener>
    <listener event="firstSearcher" class="solr.QuerySenderListener">
      <arr name="queries">
        <!--
        <lst>
          <str name="q">static firstSearcher warming in solrconfig.xml</str>
        </lst>
        -->
      </arr>
    </listener>

    <!-- Use Cold Searcher

         If a search request comes in and there is no current
         registered searcher, then immediately register the still
         warming searcher and use it.  If "false" then all requests
         will block until the first searcher is done warming.
      -->
    <useColdSearcher>true</useColdSearcher>

  </query>

  <requestDispatcher handleSelect="false" >
    <requestParsers enableRemoteStreaming="true"
                    multipartUploadLimitInKB="2048000"
                    formdataUploadLimitInKB="2048"
                    addHttpRequestToContext="false"/>

    <!-- HTTP Caching

         Set HTTP caching related parameters (for proxy caches and clients).

         The options below instruct Solr not to output any HTTP Caching
         related headers
      -->
    <httpCaching never304="true" />
  </requestDispatcher>

  <requestHandler name="/mlt" class="solr.MoreLikeThisHandler" >
    <lst name="defaults"><!-- 
      <str name="mlt.fl">title</str> -->
      <str name="mlt">true</str>
      <str name="mlt.mintf">1</str>
     <!--  <str name="mlt.mindf">2</str>  -->
     <!--  <str name="mlt.minwl">4</str>  -->
      <!-- <str name="mlt.boost">true</str> -->
      <str name="mlt.count">100</str>
      <str name="mlt.match.include">false</str> 
      <str name="echoParams">all</str>
    </lst>
  </requestHandler>


<!-- Suggestions in search input -->
<!-- need call curl http://127.0.0.1:8983/solr/collection1/suggest?suggest.build=true (put in cron)-->
<searchComponent name="suggest" class="solr.SuggestComponent">
  <lst name="suggester">
    <str name="name">suggest</str>
    <str name="lookupImpl">FreeTextLookupFactory</str> 
    <str name="dictionaryImpl">DocumentDictionaryFactory</str>
    <str name="field">term_suggest</str>
    <str name="ngrams">3</str>
    <float name="threshold">0.004</float>
    <str name="highlight">false</str>
    <str name="buildOnCommit">false</str>
    <str name="buildOnStartup">false</str>
    <str name="separator"> </str>
    <str name="suggestFreeTextAnalyzerFieldType">text_suggest</str>
    <str name="queryAnalyzerFieldType">phrase_suggest</str>
  </lst>
</searchComponent>

<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
  <lst name="defaults">
    <str name="suggest.dictionary">suggest</str>
    <str name="suggest">true</str>
    <str name="suggest.count">10</str>
  </lst>
  <arr name="components">
    <str>suggest</str>
  </arr>
</requestHandler>


  <requestHandler name="/select" class="solr.SearchHandler">
    <!-- default values for query parameters can be specified, these
         will be overridden by parameters in the request
      -->
    <lst name="defaults">
      <str name="echoParams">explicit</str>
      <int name="rows">10</int>
      <str name="df">text</str>
    </lst>
  </requestHandler>

  <!-- A request handler that returns indented JSON by default -->
  <requestHandler name="/query" class="solr.SearchHandler">
    <lst name="defaults">
      <str name="echoParams">explicit</str>
      <str name="wt">json</str>
      <str name="indent">true</str>
    </lst>
  </requestHandler>


  <!-- A Robust Example

       This example SearchHandler declaration shows off usage of the
       SearchHandler with many defaults declared

       Note that multiple instances of the same Request Handler
       (SearchHandler) can be registered multiple times with different
       names (and different init parameters)
    -->
  <requestHandler name="/browse" class="solr.SearchHandler" useParams="query,facets,velocity,browse">
    <lst name="defaults">
      <str name="echoParams">explicit</str>
    </lst>
  </requestHandler>

  <initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell,/browse">
    <lst name="defaults">
      <str name="df">_text_</str>
    </lst>
  </initParams>

  <initParams path="/update/**">
    <lst name="defaults">
      <str name="update.chain">add-unknown-fields-to-the-schema</str>
    </lst>
  </initParams>

  <!-- Solr Cell Update Request Handler

       http://wiki.apache.org/solr/ExtractingRequestHandler 

    -->
  <requestHandler name="/update/extract"
                  startup="lazy"
                  class="solr.extraction.ExtractingRequestHandler" >
    <lst name="defaults">
      <str name="lowernames">true</str>
      <str name="fmap.meta">ignored_</str>
      <str name="fmap.content">_text_</str>
    </lst>
  </requestHandler>

  <searchComponent name="spellcheck" class="solr.SpellCheckComponent">

    <str name="queryAnalyzerFieldType">text_general</str>

    <!-- Multiple "Spell Checkers" can be declared and used by this
         component
      -->

    <!-- a spellchecker built from a field of the main index -->
    <lst name="spellchecker">
      <str name="name">default</str>
      <str name="field">_text_</str>
      <str name="classname">solr.DirectSolrSpellChecker</str>
      <!-- the spellcheck distance measure used, the default is the internal levenshtein -->
      <str name="distanceMeasure">internal</str>
      <!-- minimum accuracy needed to be considered a valid spellcheck suggestion -->
      <float name="accuracy">0.5</float>
      <!-- the maximum #edits we consider when enumerating terms: can be 1 or 2 -->
      <int name="maxEdits">2</int>
      <!-- the minimum shared prefix when enumerating terms -->
      <int name="minPrefix">1</int>
      <!-- maximum number of inspections per result. -->
      <int name="maxInspections">5</int>
      <!-- minimum length of a query term to be considered for correction -->
      <int name="minQueryLength">4</int>
      <!-- maximum threshold of documents a query term can appear to be considered for correction -->
      <float name="maxQueryFrequency">0.01</float>
      <!-- uncomment this to require suggestions to occur in 1% of the documents
        <float name="thresholdTokenFrequency">.01</float>
      -->
    </lst>

  </searchComponent>

  <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <!-- Solr will use suggestions from both the 'default' spellchecker
           and from the 'wordbreak' spellchecker and combine them.
           collations (re-written queries) can include a combination of
           corrections from both spellcheckers -->
      <str name="spellcheck.dictionary">default</str>
      <str name="spellcheck">on</str>
      <str name="spellcheck.extendedResults">true</str>
      <str name="spellcheck.count">10</str>
      <str name="spellcheck.alternativeTermCount">5</str>
      <str name="spellcheck.maxResultsForSuggest">5</str>
      <str name="spellcheck.collate">true</str>
      <str name="spellcheck.collateExtendedResults">true</str>
      <str name="spellcheck.maxCollationTries">10</str>
      <str name="spellcheck.maxCollations">5</str>
    </lst>
    <arr name="last-components">
      <str>spellcheck</str>
    </arr>
  </requestHandler>

  <!-- Term Vector Component

       http://wiki.apache.org/solr/TermVectorComponent
    -->
  <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>

  <!-- A request handler for demonstrating the term vector component

       This is purely as an example.

       In reality you will likely want to add the component to your 
       already specified request handlers. 
    -->
  <requestHandler name="/tvrh" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <bool name="tv">true</bool>
    </lst>
    <arr name="last-components">
      <str>tvComponent</str>
    </arr>
  </requestHandler>

  <!-- Clustering Component. (Omitted here. See the default Solr example for a typical configuration.) -->

  <!-- Terms Component

       http://wiki.apache.org/solr/TermsComponent

       A component to return terms and document frequency of those
       terms
    -->
  <searchComponent name="terms" class="solr.TermsComponent"/>

  <!-- A request handler for demonstrating the terms component -->
  <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <bool name="terms">true</bool>
      <bool name="distrib">false</bool>
    </lst>
    <arr name="components">
      <str>terms</str>
    </arr>
  </requestHandler>


  <!-- Query Elevation Component

       http://wiki.apache.org/solr/QueryElevationComponent

       a search component that enables you to configure the top
       results for a given query regardless of the normal lucene
       scoring.
    -->
  <searchComponent name="elevator" class="solr.QueryElevationComponent" >
    <!-- pick a fieldType to analyze queries -->
    <str name="queryFieldType">string</str>
    <str name="config-file">elevate.xml</str>
  </searchComponent>

  <!-- A request handler for demonstrating the elevator component -->
  <requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
    <lst name="defaults">
      <str name="echoParams">explicit</str>
    </lst>
    <arr name="last-components">
      <str>elevator</str>
    </arr>
  </requestHandler>

  <!-- Highlighting Component

       http://wiki.apache.org/solr/HighlightingParameters
    -->
  <searchComponent class="solr.HighlightComponent" name="highlight">
    <highlighting>
      <!-- Configure the standard fragmenter -->
      <!-- This could most likely be commented out in the "default" case -->
      <fragmenter name="gap"
                  default="true"
                  class="solr.highlight.GapFragmenter">
        <lst name="defaults">
          <int name="hl.fragsize">100</int>
        </lst>
      </fragmenter>

      <!-- A regular-expression-based fragmenter 
           (for sentence extraction) 
        -->
      <fragmenter name="regex"
                  class="solr.highlight.RegexFragmenter">
        <lst name="defaults">
          <!-- slightly smaller fragsizes work better because of slop -->
          <int name="hl.fragsize">70</int>
          <!-- allow 50% slop on fragment sizes -->
          <float name="hl.regex.slop">0.5</float>
          <!-- a basic sentence pattern -->
          <str name="hl.regex.pattern">[-\w ,/\n\&quot;&apos;]{20,200}</str>
        </lst>
      </fragmenter>

      <!-- Configure the standard formatter -->
      <formatter name="html"
                 default="true"
                 class="solr.highlight.HtmlFormatter">
        <lst name="defaults">
          <str name="hl.simple.pre"><![CDATA[<em>]]></str>
          <str name="hl.simple.post"><![CDATA[</em>]]></str>
        </lst>
      </formatter>

      <!-- Configure the standard encoder -->
      <encoder name="html"
               class="solr.highlight.HtmlEncoder" />

      <!-- Configure the standard fragListBuilder -->
      <fragListBuilder name="simple"
                       class="solr.highlight.SimpleFragListBuilder"/>

      <!-- Configure the single fragListBuilder -->
      <fragListBuilder name="single"
                       class="solr.highlight.SingleFragListBuilder"/>

      <!-- Configure the weighted fragListBuilder -->
      <fragListBuilder name="weighted"
                       default="true"
                       class="solr.highlight.WeightedFragListBuilder"/>

      <!-- default tag FragmentsBuilder -->
      <fragmentsBuilder name="default"
                        default="true"
                        class="solr.highlight.ScoreOrderFragmentsBuilder">
        <!-- 
        <lst name="defaults">
          <str name="hl.multiValuedSeparatorChar">/</str>
        </lst>
        -->
      </fragmentsBuilder>

      <!-- multi-colored tag FragmentsBuilder -->
      <fragmentsBuilder name="colored"
                        class="solr.highlight.ScoreOrderFragmentsBuilder">
        <lst name="defaults">
          <str name="hl.tag.pre"><![CDATA[
               <b style="background:yellow">,<b style="background:lawgreen">,
               <b style="background:aquamarine">,<b style="background:magenta">,
               <b style="background:palegreen">,<b style="background:coral">,
               <b style="background:wheat">,<b style="background:khaki">,
               <b style="background:lime">,<b style="background:deepskyblue">]]></str>
          <str name="hl.tag.post"><![CDATA[</b>]]></str>
        </lst>
      </fragmentsBuilder>

      <boundaryScanner name="default"
                       default="true"
                       class="solr.highlight.SimpleBoundaryScanner">
        <lst name="defaults">
          <str name="hl.bs.maxScan">10</str>
          <str name="hl.bs.chars">.,!? &#9;&#10;&#13;</str>
        </lst>
      </boundaryScanner>

      <boundaryScanner name="breakIterator"
                       class="solr.highlight.BreakIteratorBoundaryScanner">
        <lst name="defaults">
          <!-- type should be one of CHARACTER, WORD(default), LINE and SENTENCE -->
          <str name="hl.bs.type">WORD</str>
          <!-- language and country are used when constructing Locale object.  -->
          <!-- And the Locale object will be used when getting instance of BreakIterator -->
          <str name="hl.bs.language">en</str>
          <str name="hl.bs.country">US</str>
        </lst>
      </boundaryScanner>
    </highlighting>
  </searchComponent>

 <!-- https://cwiki.apache.org/confluence/display/solr/Schema+Factory+Definition+in+SolrConfig#SchemaFactoryDefinitioninSolrConfig-Switchingfromschema.xmltoManagedSchema -->
  <schemaFactory class="ClassicIndexSchemaFactory"/>


  <updateRequestProcessorChain name="add-unknown-fields-to-the-schema">
    <!-- UUIDUpdateProcessorFactory will generate an id if none is present in the incoming document -->
    <processor class="solr.UUIDUpdateProcessorFactory" />
    <processor class="solr.RemoveBlankFieldUpdateProcessorFactory"/>
    <processor class="solr.FieldNameMutatingUpdateProcessorFactory">
      <str name="pattern">[^\w-\.]</str>
      <str name="replacement">_</str>
    </processor>
    <processor class="solr.ParseBooleanFieldUpdateProcessorFactory"/>
    <processor class="solr.ParseLongFieldUpdateProcessorFactory"/>
    <processor class="solr.ParseDoubleFieldUpdateProcessorFactory"/>
    <processor class="solr.ParseDateFieldUpdateProcessorFactory">
      <arr name="format">
        <str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str>
        <str>yyyy-MM-dd'T'HH:mm:ss,SSSZ</str>
        <str>yyyy-MM-dd'T'HH:mm:ss.SSS</str>
        <str>yyyy-MM-dd'T'HH:mm:ss,SSS</str>
        <str>yyyy-MM-dd'T'HH:mm:ssZ</str>
        <str>yyyy-MM-dd'T'HH:mm:ss</str>
        <str>yyyy-MM-dd'T'HH:mmZ</str>
        <str>yyyy-MM-dd'T'HH:mm</str>
        <str>yyyy-MM-dd HH:mm:ss.SSSZ</str>
        <str>yyyy-MM-dd HH:mm:ss,SSSZ</str>
        <str>yyyy-MM-dd HH:mm:ss.SSS</str>
        <str>yyyy-MM-dd HH:mm:ss,SSS</str>
        <str>yyyy-MM-dd HH:mm:ssZ</str>
        <str>yyyy-MM-dd HH:mm:ss</str>
        <str>yyyy-MM-dd HH:mmZ</str>
        <str>yyyy-MM-dd HH:mm</str>
        <str>yyyy-MM-dd</str>
      </arr>
    </processor>
<!--     <processor class="solr.AddSchemaFieldsUpdateProcessorFactory">
      <str name="defaultFieldType">strings</str>
      <lst name="typeMapping">
        <str name="valueClass">java.lang.Boolean</str>
        <str name="fieldType">booleans</str>
      </lst>
      <lst name="typeMapping">
        <str name="valueClass">java.util.Date</str>
        <str name="fieldType">tdates</str>
      </lst>
      <lst name="typeMapping">
        <str name="valueClass">java.lang.Long</str>
        <str name="valueClass">java.lang.Integer</str>
        <str name="fieldType">tlongs</str>
      </lst>
      <lst name="typeMapping">
        <str name="valueClass">java.lang.Number</str>
        <str name="fieldType">tdoubles</str>
      </lst>
    </processor> -->
    <processor class="solr.LogUpdateProcessorFactory"/>
    <processor class="solr.DistributedUpdateProcessorFactory"/>
    <processor class="solr.RunUpdateProcessorFactory"/>
  </updateRequestProcessorChain>
  <queryResponseWriter name="json" class="solr.JSONResponseWriter">
    <!-- For the purposes of the tutorial, JSON responses are written as
     plain text so that they are easy to read in *any* browser.
     If you expect a MIME type of "application/json" just remove this override.
    -->
    <str name="content-type">text/plain; charset=UTF-8</str>
  </queryResponseWriter>

  <!--
     Custom response writers can be declared as needed...
    -->
  <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter" startup="lazy">
    <str name="template.base.dir">${velocity.template.base.dir:}</str>
    <str name="solr.resource.loader.enabled">${velocity.solr.resource.loader.enabled:true}</str>
    <str name="params.resource.loader.enabled">${velocity.params.resource.loader.enabled:false}</str>
  </queryResponseWriter>

  <!-- XSLT response writer transforms the XML output by any xslt file found
       in Solr's conf/xslt directory.  Changes to xslt files are checked for
       every xsltCacheLifetimeSeconds.  
    -->
  <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
    <int name="xsltCacheLifetimeSeconds">5</int>
  </queryResponseWriter>

</config>

Solr config:

java -server -Xms2g -Xmx2g -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/ssd/solr-6.4.1/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/ssd/solr-6.4.1/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/ssd/solr-6.4.1/server -Dsolr.solr.home=/ssd/solr-6.4.1/server/solr -Dsolr.install.dir=/ssd/solr-6.4.1 -Xss256k -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/ssd/solr-6.4.1/bin/oom_solr.sh 8983 /ssd/solr-6.4.1/server/logs -jar start.jar --module=http

我的上衣:

top - 10:27:49 up 21 days,  1:26, 11 users,  load average: 1.96, 1.87, 1.62
Tasks: 255 total,   2 running, 253 sleeping,   0 stopped,   0 zombie
Cpu(s): 16.5%us,  1.4%sy,  0.0%ni, 80.6%id,  1.4%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:  16302964k total, 13768272k used,  2534692k free,   109104k buffers
Swap:  4194300k total,    54360k used,  4139940k free,  7578944k cached

  PID  PPID    TIME+  %CPU %MEM  PR  NI S  VIRT SWAP  RES  UID COMMAND
 6848     1 585:52.68  0.0 28.5  20   0 S 4619m  36m 4.4g  497 redis-server
28423     1   3:30.74 98.3  4.4  20   0 S 6655m    0 701m  501 java
28589 28569   0:11.55 98.3  0.4  20   0 R  272m    0  69m   26 postmaster
28603 28569   0:02.17 17.7  0.2  20   0 S  225m    0  30m   26 postmaster
28590 28569   0:01.30  0.0  0.2  20   0 S  218m    0  23m   26 postmaster
28605 28569   0:05.64  0.0  0.1  20   0 S  219m    0  23m   26 postmaster
28598 28569   0:01.88  0.0  0.1  20   0 S  217m    0  22m   26 postmaster
28601 28569   0:00.85  0.0  0.1  20   0 S  216m    0  22m   26 postmaster
28600 28569   0:03.71  0.0  0.1  20   0 S  216m    0  22m   26 postmaster
28599 28569   0:02.55  0.0  0.1  20   0 S  217m    0  21m   26 postmaster
28594 28569   0:02.18  3.9  0.1  20   0 S  217m    0  21m   26 postmaster
28569     1   0:00.01  0.0  0.1  20   0 S  206m    0  21m   26 postmaster
28604 28569   0:01.42  0.0  0.1  20   0 S  217m    0  21m   26 postmaster
28607 28569   0:00.43  0.0  0.1  20   0 S  212m    0  17m   26 postmaster
28608 28569   0:00.78  0.0  0.1  20   0 S  211m    0  16m   26 postmaster
28592 28569   0:11.66  0.0  0.1  20   0 S  210m    0  16m   26 postmaster
28610 28569   0:01.61  0.0  0.1  20   0 S  209m    0  15m   26 postmaster
28593 28569   0:00.15 15.7  0.1  20   0 S  209m    0  14m   26 postmaster
28595 28569   0:01.58  0.0  0.1  20   0 S  209m    0  14m   26 postmaster
28591 28569   0:12.55  0.0  0.1  20   0 S  208m    0  14m   26 postmaster
28602 28569   0:24.07  0.0  0.1  20   0 S  208m    0  14m   26 postmaster
28597 28569   0:11.68  0.0  0.1  20   0 S  208m    0  14m   26 postmaster
28596 28569   0:00.65  3.9  0.1  20   0 S  208m    0  14m   26 postmaster
28606 28569   0:00.23  0.0  0.1  20   0 S  208m    0  13m   26 postmaster
28586 28569   0:00.07  0.0  0.1  20   0 S  208m    0  13m   26 postmaster
28587 28569   0:00.09  0.0  0.1  20   0 S  208m    0  13m   26 postmaster
28611 28569   0:00.18  0.0  0.1  20   0 S  208m    0  12m   26 postmaster
28588 28569   0:00.06  0.0  0.1  20   0 S  208m    0  12m   26 postmaster
21377     1   0:00.64  0.0  0.0  20   0 S  172m    0 5924    0 tuned
31278     1   1:19.31  0.0  0.0  20   0 S  154m    0 5180    0 mosh-server
26481     1   0:01.48  0.0  0.0  20   0 S  155m    0 4132    0 mosh-server
25672  2148   0:00.00  0.0  0.0  20   0 S 80984    0 3440   89 pickup
26639     1   0:00.68  0.0  0.0  20   0 S  155m    0 3264    0 mosh-server
 1605     1   0:04.71  0.0  0.0  20   0 S  243m  300 3200    0 rsyslogd
 7324     1   0:05.83  0.0  0.0  20   0 S  154m    0 3128    0 mosh-server
26552     1   0:01.55  0.0  0.0  20   0 S  154m    0 2988    0 mosh-server
21349     1   0:00.00  0.0  0.0  20   0 S  118m    0 2820    0 pmqos-static.py

日志中提交的信息:

2017-03-13 09:32:36.787 INFO  (qtp1348949648-71) [   x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={commit=true}{add=[videos_manager.video.23228 (1561746500237656064)],commit=} 0 30773
2017-03-13 09:32:36.818 INFO  (qtp1348949648-68) [   x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2017-03-13 09:32:36.818 INFO  (qtp1348949648-68) [   x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@49e5d032
2017-03-13 09:32:36.847 INFO  (qtp1348949648-85) [   x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/mlt params={q=id:videos_manager.video.472654&fl=*,score&start=0&mlt.fl=text&fq=(is_offensive:(false)+AND+status:(online))&fq=django_ct:(videos_manager.video)&rows=60&wt=json} status=0 QTime=0
2017-03-13 09:32:36.963 INFO  (qtp1348949648-68) [   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2b0f0079[collection1] main]
2017-03-13 09:32:36.966 INFO  (qtp1348949648-68) [   x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush

尝试更新:

2017-03-13 08:02:41.640 INFO  (qtp1348949648-16) [   x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={commit=true}{add=[videos_manager.video.850182 (1561740853406334976)],commit=} 0 20864

1 个答案:

答案 0 :(得分:0)

幸运的是,我很快就找到了答案。我无法告诉其中一个参数是否使它快速(我认为它是autoCommit)但它实际上非常快(我跟着一些关于solr优化的文章)。

这是新的solrconfig.xml:

2017-03-13 12:40:31.307 INFO  (qtp1348949648-13) [   x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={commit=true}{add=[videos_manager.video.34987 (1561758354571788288)],commit=} 0 123

立即更新:

<autoCommit>
  <maxDocs>15000</maxDocs>

  <!-- I suspect This make slow updates !!!!!!!
  <maxTime>60000</maxTime>
   -->

  <openSearcher>false</openSearcher>
</autoCommit>

如果有人可以解释这个伎俩有什么变化,以及为什么它可能有用,因为Solr是一个非常重的工厂要理解。

希望得到这个帮助。

编辑:autoCommit中确定的maxTime会降低更新速度。如果我评论这一行,一切都很快,把它放回去,而且升起非常慢。

{{1}}