使用BulkAll方法

时间:2017-08-28 16:15:02

标签: c# .net elasticsearch nest

BulkAll方法正在索引所有正确的记录并返回health = green;但是BulkAllObservable通过onError轨道返回,并带有以下消息:

POST不成功的低级别调用:/ [索引名称] / [索引类型] / _ bulk?pretty = true **#此API调用的审计跟踪:   - [1] BadResponse:Node:http://localhost:9200/ Took:00:00:00.0028955

ServerError:ServerError:400Type:parse_exception原因:“请求正文是必需的”

OriginalException:System.Net.WebException:远程服务器返回错误:(400)Bad Request。**

at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)    在System.Threading.Tasks.TaskFactory 1.FromAsyncCoreLogic(IAsyncResult iar, Func 2 endFunction,Action 1 endAction, Task 1 promise,Boolean requiresSynchronization) ---从抛出异常的先前位置开始的堆栈跟踪结束---    在System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(任务任务)    在System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(任务任务)    在System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(任务任务)    在Elasticsearch.Net.HttpConnection.d__14`1.MoveNext()

请求:

# Response:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "request body is required"
      }
    ],
    "type" : "parse_exception",
    "reason" : "request body is required"
  },

我们尝试改变退避时间和批量大小无济于事。

方法:

var cancellationTokenSource = new CancellationTokenSource();
    return _elasticClient.BulkAll(capacities.Value, ba => ba
        .Index(indexName)
        .Type(IndexType)
        .MaxDegreeOfParallelism(_settingsHelper.GetBulkAllDegreeOfParallelism())
        // in case of 429 response (too many requests), how long we should wait before retrying
        .BackOffTime(TimeSpan.FromSeconds(_settingsHelper.GetBulkAllBackOffTimeoutInSeconds()))
        // in case of 429 response, how many times to retry before failing
        .BackOffRetries(_settingsHelper.GetBulkAllBackOffRetries())
        // number of documents to send in each request
        .Size(_settingsHelper.GetBulkAllBatchSize())
        .RefreshOnCompleted(),
        cancellationTokenSource.Token
    );

致电代码:

public async Task BuildIndex(string indexName)
{
    _logger.Debug($"MatchingIndexEngine - BuildIndex(indexName) - Index: {indexName}");

    Lazy<IEnumerable<Elastic.CarrierCapacity>> _lazyElasticCapacities;

    var indexExistsResponse = await IndexExistsAsync(indexName).ConfigureAwait(false);
    if (!indexExistsResponse)
    {
        await CreateIndexAsync(indexName).ConfigureAwait(false);
        _lazyElasticCapacities = new Lazy<IEnumerable<Elastic.CarrierCapacity>>(GetPagedElasticCapacities);

        var handle = new ManualResetEvent(false);
        var bulkAllObservable = _matchingIndexRepository.BulkAll(_lazyElasticCapacities, indexName);

        var observer = new BulkAllObserver(
            onNext: r =>
            {
                if (r.Page % (_settingsHelper.PagesPerLog()) == 0)
                {
                    _logger.Debug($"BulkIndexCapacitiesHandler - added batch {r.Page}");
                }
            },
            onError: async e =>
            {
                _logger.Error($"MatchingIndexEngine - error building index: {indexName} | Message: {e.Message} | Inner Exception: {e.InnerException} | Data: {e.Data}");

                await ActivateIndexAsync(indexName).ConfigureAwait(false);
                var deprecatedIndexes = await GetDeprecatedMatchingIndexesAsync(indexName).ConfigureAwait(false);
                await DeleteIndexesAsync(deprecatedIndexes).ConfigureAwait(false);

                throw new ElasticException($"MatchingIndexEngine - error building index: {indexName}");
            },
            onCompleted: async () =>
            {
                await ActivateIndexAsync(indexName).ConfigureAwait(false);
                var deprecatedIndexes = await GetDeprecatedMatchingIndexesAsync(indexName).ConfigureAwait(false);
                await DeleteIndexesAsync(deprecatedIndexes).ConfigureAwait(false);

                _logger.Info($"Rebuild of new index was successful - switched alias to new index: {indexName}.");
            }
        );
        bulkAllObservable.Subscribe(observer);


        handle.WaitOne();
        handle.Dispose();
    }

0 个答案:

没有答案