Question

我在服务器上运行了elasticsearch 5.5，其中包含一些索引数据。我想将此ES数据迁移到AWS elasticsearch集群。我如何执行此迁移。我知道一种方法是创建ES集群的快照，但我无法找到任何适当的文档。

Answer 1

迁移的最佳方式是使用快照。您需要将数据快照到Amazon S3，然后从那里继续进行还原。可以找到快照到S3的文档here。或者，您也可以重新索引数据，尽管这是一个较长的过程，并且根据AWS ES的版本存在限制。

我还建议查看AWS上的官方托管产品Elastic Cloud，其中包括其他X-Pack监控，管理和安全功能。迁移到Elastic Cloud的migration guide也会覆盖快照并重新编制索引。

Answer 2

我为此暂时创建了一个shell脚本-

Github-https://github.com/vivekyad4v/aws-elasticsearch-domain-migration/blob/master/migrate.sh

#!/bin/bash

#### Make sure you have Docker engine installed on the host ####
###### TODO - Support parameters ######

export AWS_ACCESS_KEY_ID=xxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxx
export AWS_DEFAULT_REGION=ap-south-1
export AWS_DEFAULT_OUTPUT=json
export S3_BUCKET_NAME=my-es-migration-bucket
export DATE=$(date +%d-%b-%H_%M)

old_instance="https://vpc-my-es-ykp2tlrxonk23dblqkseidmllu.ap-southeast-1.es.amazonaws.com"
new_instance="https://vpc-my-es-mg5td7bqwp4zuiddwgx2n474sm.ap-south-1.es.amazonaws.com"
delete=(.kibana)
es_indexes=$(curl -s "${old_instance}/_cat/indices" | awk '{ print $3 }')
es_indexes=${es_indexes//$delete/}
es_indexes=$(echo $es_indexes|tr -d '\n')

echo "index to be copied are - $es_indexes"

for index in $es_indexes; do

# Export ES data to S3 (using s3urls) 
docker run --rm -ti taskrabbit/elasticsearch-dump \
  --s3AccessKeyId "${AWS_ACCESS_KEY_ID}" \
  --s3SecretAccessKey "${AWS_SECRET_ACCESS_KEY}" \
  --input="${old_instance}/${index}" \
  --output "s3://${S3_BUCKET_NAME}/${index}-${DATE}.json"

# Import data from S3 into ES (using s3urls) 
docker run --rm -ti taskrabbit/elasticsearch-dump \
  --s3AccessKeyId "${AWS_ACCESS_KEY_ID}" \
  --s3SecretAccessKey "${AWS_SECRET_ACCESS_KEY}" \
  --input "s3://${S3_BUCKET_NAME}/${index}-${DATE}.json" \
  --output="${new_instance}/${index}"

new_indexes=$(curl -s "${new_instance}/_cat/indices" | awk '{ print $3 }')
echo $new_indexes
curl -s "${new_instance}/_cat/indices"

done

如何将elasticsearch数据迁移到AWS elasticsearch域？

2 个答案: