elasticsearch - 备份存储库大小远远大于索引大小

我每小时有一次Elasticsearch备份，经过大约370次备份（大约15天）后，我的备份库超过了15G！但总指数大小只有500M！ Elasticsearch是增量备份，但是15G VS 500M，差别是如此巨大！我想知道索引和备份库之间的大小不同是否正常？

是否是我频繁备份（每小时）造成的？我使用群集1中的每小时备份和群集2中的每小时恢复来保持两个ES群集数据实时相同。

我正在使用Elasticsearch备份API进行备份

设置回购： curl -XPUT＆＃34; http://IP:9200/_snapshot/backup＆＃34; -d＆＃34; {\＆＃34;输入\＆＃34;：\＆＃34; fs \＆＃34;，\＆＃34;设置\＆＃34;：{\＆＃34;压缩\＆＃34; ：true，\＆＃34; location \＆＃34;：\＆＃34; backup \＆＃34;}}＆＃34;
备份： CURTIME = date +"%Y-%m-%d %H:%M:%S" BKTIME = $ {CURTIME // [ - ：] /} curl -XPUT＆＃34; http://IP:9200/snapshot/backup/snapshot $ BKTIME？wait_for_completion = true＆＃34;

我的Elasticsearch设置：2个节点，12个分片/节点，2个索引，用于将快照存储到NAS的fs备份类型

在Elasticsearch数据目录中，索引大小为：

node1索引大小： [root @ esnode1 indices] $ du -sh

307M。

node2索引大小 [root @ esnode2 indices] $ du -sh

238M。

[root @ esnode1 indices] $ du -lh

8.0K ./index1/10/translog 8.0K ./index1/10/_state 2.9M ./index1/10/index 2.9M ./index1/10 12K ./index1/5/translog 8.0K ./index1/5/_state 1.5M ./index1/5/index 1.5M ./index1/5 8.0K ./index1/4/translog 8.0K ./index1/4/_state 2.9M ./index1/4/index 2.9M ./index1/4 8.0K ./index1/_state 8.0K ./index1/7/translog 8.0K ./index1/7/_state 2.9M ./index1/7/index 2.9M ./index1/7 8.0K ./index1/1/translog 8.0K ./index1/1/_state 2.9M ./index1/1/index 2.9M ./index1/1 8.0K ./index1/2/translog 8.0K ./index1/2/_state 2.9M ./index1/2/index 2.9M ./index1/2 8.0K ./index1/6/translog 8.0K ./index1/6/_state 3.0M ./index1/6/index 3.0M ./index1/6 8.0K ./index1/0/translog 8.0K ./index1/0/_state 1.5M ./index1/0/index 1.5M ./index1/0 8.0K ./index1/8/translog 8.0K ./index1/8/_state 1.5M ./index1/8/index 1.5M ./index1/8 8.0K ./index1/11/translog 8.0K ./index1/11/_state 2.9M ./index1/11/index 2.9M ./index1/11 12K ./index1/9/translog 8.0K ./index1/9/_state 3.0M ./index1/9/index 3.0M ./index1/9 8.0K ./index1/3/translog 8.0K ./index1/3/_state 3.0M ./index1/3/index 3.0M ./index1/3 31M ./index1 16K ./index2/10/translog 8.0K ./index2/10/_state 16M ./index2/10/index 16M ./index2/10 36K ./index2/5/translog 8.0K ./index2/5/_state 43M ./index2/5/index 43M ./index2/5 20K ./index2/4/translog 8.0K ./index2/4/_state 17M ./index2/4/index 18M ./index2/4 8.0K ./index2/_state 40K ./index2/7/translog 8.0K ./index2/7/_state 32M ./index2/7/index 32M ./index2/7 68K ./index2/1/translog 8.0K ./index2/1/_state 21M ./index2/1/index 21M ./index2/1 64K ./index2/2/translog 8.0K ./index2/2/_state 19M ./index2/2/index 19M ./index2/2 116K ./index2/6/translog 8.0K ./index2/6/_state 22M ./index2/6/index 22M ./index2/6 24K ./index2/0/translog 8.0K ./index2/0/_state 17M ./index2/0/index 17M ./index2/0 128K ./index2/8/translog 8.0K ./index2/8/_state 34M ./index2/8/index 34M ./index2/8 72K ./index2/11/translog 8.0K ./index2/11/_state 20M ./index2/11/index 20M ./index2/11 88K ./index2/9/translog 8.0K ./index2/9/_state 22M ./index2/9/index 22M ./index2/9 76K ./index2/3/translog 8.0K ./index2/3/_state 16M ./index2/3/index 16M ./index2/3 277M ./index2 307M。

在备份库中，大小： [root @ esnode1 backup] $ du -lh

114M ./backup/indices/index1/0

112M ./backup/indices/index1/5

114M ./backup/indices/index1/11

114M ./backup/indices/index1/10

111M ./backup/indices/index1/8

116M ./backup/indices/index1/4

120M ./backup/indices/index1/9

118M ./backup/indices/index1/3

114M ./backup/indices/index1/2

115M ./backup/indices/index1/7

115M ./backup/indices/index1/1

112M ./backup/indices/index1/6

1.4G ./backup/indices/index1

747M ./backup/indices/index2/0

1.6G ./backup/indices/index2/5

887M ./backup/indices/index2/11

743M ./backup/indices/index2/10

2.1G ./backup/indices/index2/8

801M ./backup/indices/index2/4

1.3G ./backup/indices/index2/9

878M ./backup/indices/index2/3

951M ./backup/indices/index2/2

1.2G ./backup/indices/index2/7

953M ./backup/indices/index2/1

943M ./backup/indices/index2/6

13G ./backup/indices/index2

15G ./backup/indices

15G ./backup

1.1M ./backuplogs

15G。

====== https://www.elastic.co/blog/introducing-snapshot-restore 备份和还原操作都是增量操作，这意味着只有自上次快照以来更改的文件才会复制到存储库或还原到索引中。 增量快照允许根据需要频繁执行快照操作，而不会产生太多的磁盘空间开销。用户现在可以在升级之前轻松创建快照或在群集中进行风险更改，并快速回滚到先前的索引状态事情出错了。快照/恢复机制还可用于在“热”群集与不同地理区域中的远程“冷”备份群集之间同步数据，以实现快速灾难恢复。

从上面来看，我的案子确实是一个问题，任何人都可以帮助我吗？提前谢谢！

备份存储库大小远远大于索引大小

2 个答案: