Question

我正在使用RestHighLevelClient版本7.2连接到ElasticSearch集群版本7.2。我的集群有3个主节点和2个数据节点。数据节点内存配置：2核心和8 GB。我已经习惯了在我的Spring Boot项目中下面的代码来创建RestHighLevelClient实例。

Private Sub Worksheet_Change(ByVal Target As Range)

    'Only run if something changes in column D or E
    If Target.Column = 4 Or Target.Column = 5 Then

        'Turn off any events so that we don't encounter recursion
        Application.EnableEvents = False

        'This will help readability a bit
        Dim sht As Worksheet
        Set sht = ThisWorkbook.Worksheets("Sheet1")

        Dim A As Integer
        Dim i As Long

        'This needs to be removed - it's irrelevant as i is used as an iterable on the next line
        'i = 5

        For i = 5 To 12

            If sht.Range("D" & i).Value = "" Or sht.Range("D" & i).Value = 0 Then
                'What's the point of using a variable here?
                A = sht.Range("E" & i).Value - sht.Range("C" & i).Value
                sht.Range("F" & i).Value = A
            Else
                'Order of operations - is that important here?
                'Are we certain these fields are numeric?
                sht.Range("F" & i).Value = sht.Range("D" & i).Value * sht.Range("B" & i).Value _
                + sht.Range("E" & i).Value - sht.Range("C" & i).Value
            End If

        Next i

        'Turn it back on once we're done
        Application.EnableEvents = True

    End If

End Sub

RestHighLevelClient是单例bean。间歇性地，我同时收到GET和PUT请求的SocketTimeoutException。索引大小约为50 MB。我尝试增加套接字超时值，但是仍然收到相同的错误。我是否缺少某些配置？任何帮助将不胜感激。

Answer 1

我得到的问题只是想分享，以便可以帮助他人。我正在使用负载均衡器连接到ElasticSerach群集。正如您从RestClientBuilder代码中看到的那样，我仅使用了loadbalancer主机和端口。尽管我有多个主节点，但在连接超时的情况下，RestClient仍不会重试我的请求。

RestClientBuilder builder = RestClient.builder(new HttpHost(elasticHost, elasticPort))
                .setHttpClientConfigCallback(httpClientBuilder ->httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider).setDefaultIOReactorConfig(IOReactorConfig.custom().setIoThreadCount(5).build()));

根据RestClient代码，如果我们使用单个主机，则在出现任何连接问题时不会重试。因此，我如下更改了代码，它开始起作用。

RestClientBuilder builder = RestClient.builder(new HttpHost(elasticHost, 9200),new HttpHost(elasticHost, 9201))).setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));

有关完整的RestClient代码，请参阅https://github.com/elastic/elasticsearch/blob/master/client/rest/src/main/java/org/elasticsearch/client/RestClient.java

RestClient中的重试代码块

private Response performRequest(final NodeTuple<Iterator<Node>> nodeTuple,
                                    final InternalRequest request,
                                    Exception previousException) throws IOException {
        RequestContext context = request.createContextForNextAttempt(nodeTuple.nodes.next(), nodeTuple.authCache);
        HttpResponse httpResponse;
        try {
            httpResponse = client.execute(context.requestProducer, context.asyncResponseConsumer, context.context, null).get();
        } catch(Exception e) {
            RequestLogger.logFailedRequest(logger, request.httpRequest, context.node, e);
            onFailure(context.node);
            Exception cause = extractAndWrapCause(e);
            addSuppressedException(previousException, cause);
            if (nodeTuple.nodes.hasNext()) {
                return performRequest(nodeTuple, request, cause);
            }
            if (cause instanceof IOException) {
                throw (IOException) cause;
            }
            if (cause instanceof RuntimeException) {
                throw (RuntimeException) cause;
            }
            throw new IllegalStateException("unexpected exception type: must be either RuntimeException or IOException", cause);
        }
        ResponseOrResponseException responseOrResponseException = convertResponse(request, context.node, httpResponse);
        if (responseOrResponseException.responseException == null) {
            return responseOrResponseException.response;
        }
        addSuppressedException(previousException, responseOrResponseException.responseException);
        if (nodeTuple.nodes.hasNext()) {
            return performRequest(nodeTuple, request, responseOrResponseException.responseException);
        }
        throw responseOrResponseException.responseException;
    }

Answer 2

我正面临着同样的问题，看到这一点，我意识到重试也在每台主机上发生在我这一边（我有3台主机，例外发生在3个线程中）。我想发布它，因为您可能会遇到相同的问题，或者由于相同的SocketConnection异常而可能有其他人来发布此帖子。

在搜索官方文档时，HighLevelRestClient使用RestClient进行后台处理，而RestClient使用CloseableHttpAsyncClient，后者具有连接池。 ElasticSearch指定您应该在完成后关闭连接（这在应用程序中“完成”的定义听起来是模棱两可的），但是通常在互联网上，我发现您应该在应用程序关闭或结束时关闭连接，而不是当您完成查询时。

现在在apache的官方文档中，他们有一个示例来处理连接池，我正在尝试遵循该示例，我将尝试复制该方案并发布，如果此程序可以解决我的问题，则可以找到代码在这里：

https://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientEvictExpiredConnections.java

这是我到目前为止所拥有的：

@Bean(name = "RestHighLevelClientWithCredentials", destroyMethod = "close")
public RestHighLevelClient elasticsearchClient(ElasticSearchClientConfiguration elasticSearchClientConfiguration,
                                               RestClientBuilder.HttpClientConfigCallback httpClientConfigCallback) {
    return new RestHighLevelClient(
            RestClient
                    .builder(getElasticSearchHosts(elasticSearchClientConfiguration))
                    .setHttpClientConfigCallback(httpClientConfigCallback)
    );
}

@Bean
@RefreshScope
public RestClientBuilder.HttpClientConfigCallback getHttpClientConfigCallback(
        PoolingNHttpClientConnectionManager poolingNHttpClientConnectionManager,
        CredentialsProvider credentialsProvider
) {
    return httpAsyncClientBuilder -> {
        httpAsyncClientBuilder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
        httpAsyncClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
        httpAsyncClientBuilder.setConnectionManager(poolingNHttpClientConnectionManager);
        return httpAsyncClientBuilder;
    };
}

公共类ElasticSearchClientManager {

private ElasticSearchClientManager.IdleConnectionEvictor idleConnectionEvictor;

/**
 * Custom client connection manager to create a connection watcher
 *
 * @param elasticSearchClientConfiguration elasticSearchClientConfiguration
 * @return PoolingNHttpClientConnectionManager
 */
@Bean
@RefreshScope
public PoolingNHttpClientConnectionManager getPoolingNHttpClientConnectionManager(
        ElasticSearchClientConfiguration elasticSearchClientConfiguration
) {
    try {
        SSLIOSessionStrategy sslSessionStrategy = new SSLIOSessionStrategy(getTrustAllSSLContext());
        Registry<SchemeIOSessionStrategy> sessionStrategyRegistry = RegistryBuilder.<SchemeIOSessionStrategy>create()
                .register("http", NoopIOSessionStrategy.INSTANCE)
                .register("https", sslSessionStrategy)
                .build();
        ConnectingIOReactor ioReactor = new DefaultConnectingIOReactor();
        PoolingNHttpClientConnectionManager poolingNHttpClientConnectionManager =
                new PoolingNHttpClientConnectionManager(ioReactor, sessionStrategyRegistry);
        idleConnectionEvictor = new ElasticSearchClientManager.IdleConnectionEvictor(poolingNHttpClientConnectionManager,
                elasticSearchClientConfiguration);
        idleConnectionEvictor.start();
        return poolingNHttpClientConnectionManager;
    } catch (IOReactorException e) {
        throw new RuntimeException("Failed to create a watcher for the connection pool");
    }
}

private SSLContext getTrustAllSSLContext() {
    try {
        return new SSLContextBuilder()
                .loadTrustMaterial(null, (x509Certificates, string) -> true)
                .build();
    } catch (Exception e) {
        throw new RuntimeException("Failed to create SSL Context with open certificate", e);
    }
}

public IdleConnectionEvictor.State state() {
    return idleConnectionEvictor.evictorState;
}

@PreDestroy
private void finishManager() {
    idleConnectionEvictor.shutdown();
}


public static class IdleConnectionEvictor extends Thread {

    private final NHttpClientConnectionManager nhttpClientConnectionManager;
    private final ElasticSearchClientConfiguration elasticSearchClientConfiguration;

    @Getter
    private State evictorState;
    private volatile boolean shutdown;

    public IdleConnectionEvictor(NHttpClientConnectionManager nhttpClientConnectionManager,
                                 ElasticSearchClientConfiguration elasticSearchClientConfiguration) {
        super();
        this.nhttpClientConnectionManager = nhttpClientConnectionManager;
        this.elasticSearchClientConfiguration = elasticSearchClientConfiguration;
    }

    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(elasticSearchClientConfiguration.getExpiredConnectionsCheckTime());
                    // Close expired connections
                    nhttpClientConnectionManager.closeExpiredConnections();
                    // Optionally, close connections
                    // that have been idle longer than 5 sec
                    nhttpClientConnectionManager.closeIdleConnections(elasticSearchClientConfiguration.getMaxTimeIdleConnections(),
                            TimeUnit.SECONDS);
                    this.evictorState = State.RUNNING;
                }
            }
        } catch (InterruptedException ex) {
            this.evictorState = State.NOT_RUNNING;
        }
    }

    private void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }

    public enum State {
        RUNNING,
        NOT_RUNNING
    }
}

}

间歇性SocketTimeoutException与elasticsearch-rest-client-7.2.0

2 个答案: