我有两个相同的服务器,我在一个服务器上设置了弹性搜索,并在其中导入了大约200万条记录。我现在希望其他服务器具有相同的记录集,并且这两个服务器作为一对工作,只需要一个服务器关闭我可以快速从群集中删除它,它将正常工作,没有插入新数据读。我按照我在网上找到的信息,并设置我的主机找到对方。看起来索引没有转移到我在复制的分片状态中看到的其他服务器,但它们的数据目录中的大小永远不会增加。
我用它作为指针。
How to add a new node to my Elasticsearch cluster
我的问题是,我可以scp将300gb以上的数据复制到数据目录中,还是弹性搜索需要这样做?
调试信息
curl 192.168.1.1:9200/_cat/recovery?v
index shard time type stage source_host target_host repository snapshot files files_percent bytes bytes_percent total_files total_bytes translog translog_percent total_translog
pelias 0 248094 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 76 13092952848 0 100.0% 0
pelias 0 333911 relocation index 192.168.1.2 192.168.1.1 n/a n/a 76 76.3% 13092952848 9.6% 76 13092952848 0 100.0% 0
pelias 1 248053 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 70 12924273847 0 100.0% 0
pelias 1 333915 relocation index 192.168.1.2 192.168.1.1 n/a n/a 70 98.6% 12924273847 98.0% 70 12924273847 0 100.0% 0
pelias 2 246409 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 88 13122252645 0 100.0% 0
pelias 3 247434 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 89 13020268507 0 100.0% 0
pelias 4 247109 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 91 13173644369 0 100.0% 0
pelias 5 247574 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 85 13133234395 0 100.0% 0
pelias 6 245236 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 12882617780 0 100.0% 0
pelias 7 246530 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 86 13098807983 0 100.0% 0
pelias 8 245089 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 12963045661 0 100.0% 0
pelias 9 245149 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 94 13072471481 0 100.0% 0
pelias 10 248994 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 85 12950316547 0 100.0% 0
pelias 11 247667 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 13329893241 0 100.0% 0
pelias 12 248055 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 12930959145 0 100.0% 0
pelias 13 249255 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 106 11586141290 0 100.0% 0
pelias 14 247759 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 15107208138 0 100.0% 0
pelias 15 247888 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 13112198101 0 100.0% 0
pelias 16 247675 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 91 14672189956 0 100.0% 0
pelias 17 247205 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 13048452056 0 100.0% 0
pelias 18 246910 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 13095873096 0 100.0% 0
pelias 19 246325 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 70 15019530308 0 100.0% 0
pelias 20 247924 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 77 12898451788 0 100.0% 0
pelias 21 248663 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 76 13035468700 0 100.0% 0
pelias 22 249105 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 95 13045543380 0 100.0% 0
pelias 23 246119 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 13052787518 0 100.0% 0
curl 192.168.1.1:9200/_cat/shards?v
index shard prirep state docs store ip node
pelias 22 p STARTED 21505006 12.1gb 192.168.1.2 ronin-tile02
pelias 5 p STARTED 21503012 12.2gb 192.168.1.2 ronin-tile02
pelias 2 p STARTED 21499705 12.2gb 192.168.1.2 ronin-tile02
pelias 15 p STARTED 21504176 12.2gb 192.168.1.2 ronin-tile02
pelias 4 p STARTED 21512286 12.2gb 192.168.1.2 ronin-tile02
pelias 16 p STARTED 21508361 13.6gb 192.168.1.2 ronin-tile02
pelias 20 p STARTED 21499566 12gb 192.168.1.2 ronin-tile02
pelias 6 p STARTED 21504264 11.9gb 192.168.1.2 ronin-tile02
pelias 14 p STARTED 21511288 14gb 192.168.1.2 ronin-tile02
pelias 23 p STARTED 21513431 12.1gb 192.168.1.2 ronin-tile02
pelias 13 p STARTED 21505815 10.7gb 192.168.1.2 ronin-tile02
pelias 12 p STARTED 21506998 12gb 192.168.1.2 ronin-tile02
pelias 10 p STARTED 21511704 12gb 192.168.1.2 ronin-tile02
pelias 9 p STARTED 21504736 12.1gb 192.168.1.2 ronin-tile02
pelias 1 p RELOCATING 21506366 12gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01
pelias 7 p STARTED 21508186 12.1gb 192.168.1.2 ronin-tile02
pelias 21 p STARTED 21511618 12.1gb 192.168.1.2 ronin-tile02
pelias 17 p STARTED 21511014 12.1gb 192.168.1.2 ronin-tile02
pelias 18 p STARTED 21512577 12.1gb 192.168.1.2 ronin-tile02
pelias 8 p STARTED 21511765 12gb 192.168.1.2 ronin-tile02
pelias 11 p STARTED 21504697 12.4gb 192.168.1.2 ronin-tile02
pelias 3 p STARTED 21509465 12.1gb 192.168.1.2 ronin-tile02
pelias 19 p STARTED 21503146 13.9gb 192.168.1.2 ronin-tile02
pelias 0 p RELOCATING 21512876 12.1gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01
curl 192.168.1.1:9200/_cat/indices?v
health status index pri rep docs.count docs.deleted store.size pri.store.size
green open pelias 24 0 516182058 89 295.5gb 295.5gb
答案 0 :(得分:0)
为了在两个(或更多)Elasticsearch节点之间复制数据,您需要添加replicas。对于索引(pelias
)和主机(192.168.1.1:9200
),可以使用以下curl
命令添加一个副本:
curl -XPUT 'http://192.168.1.1:9200/pelias/_settings' -d '{
"index": {
"number_of_replicas": 1
}
}'
顺便说一句,感谢您使用Pelias,我是核心维护者之一:)
我还维护了一组bash脚本来完成常见的Elasticsearch任务here。有一个专门针对adding replicas的脚本,您或其他人可能会觉得有用。