使用elasticsearch scroll api

时间:2017-10-13 07:06:09

标签: java elasticsearch elasticsearch-5

我正在使用elasticsearch(ES版本5)滚动API来检索所有文档,然后写入csv文件。我的代码如下。这是有效的,但有一点问题。下载文件需要5分钟以上。

try
    {
        TransportClient client = getTransportClient( NoSQLConnectionPool.ELASTIC_STAT );

        if( client != null )
        {
            SearchResponse scrollResp = client.prepareSearch( Constants.INDEX )
                    .addSort( fieldSort( Constants.PRICE ).order( ASC ) )
                    .setScroll( new TimeValue( 60000 ) )
                    .setQuery( buildBoolQuery( request ) )
                    .setSize( 100 ).get(); //max of 100 hits will be returned for each scroll

            //Scroll until no hits are returned
            do
            {
                List<JsonElement> list = getAllElement( scrollResp.getHits().getHits() );
                //                  resultsList.addAll( results );

                buildReportContent( sb, list ); //iterate list and append data to string builder(sb)

                scrollResp = client.prepareSearchScroll( scrollResp.getScrollId() ).setScroll( new TimeValue( 60000 ) ).execute().actionGet();
            }
            while( scrollResp.getHits().getHits().length != 0 ); // Zero hits mark the end of the scroll and the while loop.
        }

        return CsvFileWriter.csvFileWrite( sb );

    }
    catch ( Exception e )
    {
        e.printStackTrace();

    }

有任何建议可以更有效地完成这项工作吗?

谢谢!

0 个答案:

没有答案