Spring数据solr - 通过DIH导入全数据

时间:2015-03-10 20:16:38

标签: solr dataimporthandler spring-data-solr

我使用solr 4.10.2和Spring数据solr进行集成。 为了保护solr,我在web.xml中添加了一个安全条目。 因此,在Solrcontext中进行了以下更改。 注意: - 我们的应用程序中只有一个核心。

@Configuration
@EnableSolrRepositories(basePackages = { "com.wellmanage.mnpi.repository.solr" }, multicoreSupport = true)
public class SolrContext {

    @Value("${solr.url}")
    private String solrHost;

    @Value("${solr.user}")
    private String solrUser;

    @Value("${solr.password}")
    private String solrPassword;

    @Bean
    public SolrServer solrServer() {
        return new HttpSolrServer(solrHost);
    }

    @Bean
    public SolrTemplate solrTemplate() {
      return new SolrTemplate(solrServerFactory());
    }

    @Bean
    public SolrServerFactory solrServerFactory() {

      Credentials credentials = new UsernamePasswordCredentials(solrUser, solrPassword);
      return new HttpSolrServerFactory(solrServer(), "", credentials , "DIGEST");
    }

现在,我想每5分钟安排一次全数据导入。

@EnableAutoConfiguration
@EnableScheduling
@Service
public class DataImportScheduler {

    private static final Log logger = LogFactory
            .getLog(DataImportScheduler.class);

    @Autowired
    private SolrTemplate solrTemplate;

    @Scheduled(fixedDelayString = "${solr.dataimport.interval.inmilliseconds}" )
    public void importData() {
        logger.info("Solr Full Data Import Started");

        SolrResponse response = solrTemplate.execute(new SolrCallback<SolrResponse>() {

            @Override
            public SolrResponse doInSolr(SolrServer solrServer) throws SolrServerException, IOException {

              return new SolrRequest(METHOD.GET, "/security/dataimport?command=full-import&clean=false&commit=true") {

                @Override
                public SolrResponse process(SolrServer server) throws SolrServerException, IOException {
                  SolrResponseBase response = new SolrResponseBase();
                  response.setResponse(server.request(this));
                  return response;
                }

                @Override
                public Collection<ContentStream> getContentStreams()
                        throws IOException {
                    // TODO Auto-generated method stub
                    return null;
                }

                @Override
                public SolrParams getParams() {
                    // TODO Auto-generated method stub
                    return null;
                }

              }.process(solrServer);

            }
          });

        logger.info("Solr Full Data Import Completed");
    }
}

安全是核心名称。

通过此更改获得以下错误....

2015-03-10 16:00:13.734 ERROR 11180 --- [ask-scheduler-3] o.s.integration.handler.LoggingHandler   : org.springframework.data.solr.UncategorizedSolrException: Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/7.0.55 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 400 - {msg=XMLWriter does not support version: 2,code=400}</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>{msg=XMLWriter does not support version: 2,code=400}</u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/7.0.55</h3></body></html>; nested exception is org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/7.0.55 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 400 - {msg=XMLWriter does not support version: 2,code=400}</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>{msg=XMLWriter does not support version: 2,code=400}</u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/7.0.55</h3></body></html>
    at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:136)
    at com.wellmanage.mnpi.solr.scheduler.DataImportScheduler.importData(DataImportScheduler.java:41)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html><head><title>Apache Tomcat/7.0.55 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 400 - {msg=XMLWriter does not support version: 2,code=400}</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>{msg=XMLWriter does not support version: 2,code=400}</u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/7.0.55</h3></body></html>
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:459)
    at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
    at com.wellmanage.mnpi.solr.scheduler.DataImportScheduler$1$1.process(DataImportScheduler.java:51)
    at com.wellmanage.mnpi.solr.scheduler.DataImportScheduler$1.doInSolr(DataImportScheduler.java:68)
    at com.wellmanage.mnpi.solr.scheduler.DataImportScheduler$1.doInSolr(DataImportScheduler.java:1)
    at org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:132)
    ... 14 more

不确定我在这里缺少什么...

由于

1 个答案:

答案 0 :(得分:1)

缺少的部分是设置ReponseParser,了解Solr DIH回答请求的方式。添加XMLResponseParser应该可以解决问题。

template.execute(new SolrCallback<SolrResponse>() {

  @Override
  public SolrResponse doInSolr(SolrServer solrServer) throws SolrServerException, IOException {

    SolrRequest request = new SolrRequest(METHOD.GET, "/dataimport?command=status") {

      @Override
      public SolrResponse process(SolrServer server) throws SolrServerException, IOException {
        SolrResponseBase response = new SolrResponseBase();
        response.setResponse(server.request(this));
        return response;
      }

      @Override
      public Collection<ContentStream> getContentStreams() throws IOException {
        // TODO Auto-generated method stub
        return null;
      }

      @Override
      public SolrParams getParams() {
        // TODO Auto-generated method stub
        return null;
      }

    };
    request.setResponseParser(new XMLResponseParser());
    return request.process(solrServer);
  }
});