如何在Python3中通过elasticsearch批量删除?

时间:2018-12-18 17:12:24

标签: python-3.x elasticsearch

我已经编写了一个代码片段,该代码片段将批量删除es id。我在try except中写过:

with open(delete_patch_destination) as delete_json_file:
            for line in delete_json_file:
                line_content = json.loads(line)
                for es_key in line_content.keys():
                    es_key = ast.literal_eval(es_key).decode()
                    es_indices_to_be_deleted.append(
                        {
                            '_op_type': 'delete',
                            '_index': self.index_name,
                            '_type': "latest",
                            "_id": es_key
                        }
                    )
                    if len(es_indices_to_be_deleted) >= 500:
                        try:
                            helpers.bulk(self.es, es_indices_to_be_deleted)
                            del es_indices_to_be_deleted[:]
                        except exceptions.NotFoundError:
                            pass
            if len(es_indices_to_be_deleted) > 0:
                try:
                    helpers.bulk(self.es, es_indices_to_be_deleted)
                    del es_indices_to_be_deleted[:]
                except exceptions.NotFoundError:
                    pass

相同的错误是通过Not found error,它是:

py3.5.egg/pallet/tasks/datatable.py", line 244, in delete_from_es
        helpers.bulk(self.es, es_indices_to_be_deleted)
      File "/usr/local/lib/python3.5/dist-packages/elasticsearch/helpers/__init__.py", line 257, in bulk
        for ok, item in streaming_bulk(client, actions, **kwargs):
      File "/usr/local/lib/python3.5/dist-packages/elasticsearch/helpers/__init__.py", line 192, in streaming_bulk
        raise_on_error, **kwargs)
      File "/usr/local/lib/python3.5/dist-packages/elasticsearch/helpers/__init__.py", line 137, in _process_bulk_chunk
        raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
    elasticsearch.helpers.BulkIndexError: ('500 document(s) failed to index.', [{'delete': {'_index': 'f0e7d61886945c3b7cfcd894e9f552d0', 'status': 404, '_type': 'latest', '_id': 'f52193c6535e6417b0b51d74a6ca3f22', '_shards': {'total': 2, 'successful': 1, 'failed': 0}, '_version': 1, 'found': False}},

我应该除了引发的错误之外,还是要使此批量删除正常工作,还要做什么?

0 个答案:

没有答案