Question

我在单个磁盘上安装了一个非常简单的ES6，该磁盘足够大（用于我的需要）：

root@ip-172-17-20-222:/home/admin# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  155M  1.4G  10% /run
/dev/nvme0n1p1   59G   34G   24G  59% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           1.6G     0  1.6G   0% /run/user/1000

如您所见，磁盘上有足够的可用空间。

我使用http API配置了ES：

curl -XPUT -H "Content-Type: application/json"  "http://172.17.20.222:9200/_cluster/settings" -d'
{
  "transient": {
    "cluster.routing.allocation.disk.watermark.low": "85%",
    "cluster.routing.allocation.disk.watermark.high": "90%",
    "cluster.routing.allocation.disk.watermark.flood_stage": "95%",
    "cluster.info.update.interval": "1m"
  }
}'
curl -XPUT -H "Content-Type: application/json" http://172.17.20.222:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

但几分钟后：

[2019-11-22T16:02:46,889][INFO ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] low disk watermark [85%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 6gb[10.2%], replicas will not be assigned to this node
[2019-11-22T16:03:46,890][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.6gb[4.4%], all indices on this node will be marked read-only
[2019-11-22T16:04:46,892][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.5gb[4.3%], all indices on this node will be marked read-only
[2019-11-22T16:05:46,897][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.5gb[4.2%], all indices on this node will be marked read-only
[2019-11-22T16:06:46,899][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.4gb[4.1%], all indices on this node will be marked read-only
[2019-11-22T16:07:46,901][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.3gb[4%], all indices on this node will be marked read-only
[2019-11-22T16:08:46,903][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.3gb[3.9%], all indices on this node will be marked read-only
[2019-11-22T16:09:46,905][WARN ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] flood stage disk watermark [95%] exceeded on [C25At0XLRAyaZ00zegY20A][C25At0X][/var/lib/elasticsearch/nodes/0] free: 2.2gb[3.8%], all indices on this node will be marked read-only
[2019-11-22T16:10:46,908][INFO ][o.e.c.r.a.DiskThresholdMonitor] [C25At0X] rerouting shards: [one or more nodes has gone under the high or low watermark]

如果我理解正确，ES将暂停索引编制，因为它认为磁盘中没有足够的可用空间，但这似乎不正确。

按照我的说法，我也不认为该问题与inode相关：

root@ip-172-17-20-222:/home/admin# ulimit
unlimited
root@ip-172-17-20-222:/home/admin# df -ih
Filesystem     Inodes IUsed IFree IUse% Mounted on
udev             2.0M   269  2.0M    1% /dev
tmpfs            2.0M   683  2.0M    1% /run
/dev/nvme0n1p1   3.8M   44K  3.8M    2% /
tmpfs            2.0M     1  2.0M    1% /dev/shm
tmpfs            2.0M     2  2.0M    1% /run/lock
tmpfs            2.0M    15  2.0M    1% /sys/fs/cgroup
tmpfs            2.0M    11  2.0M    1% /run/user/1000

还有什么可能导致这种行为？

Answer 1

free: 2.5gb[4.3%]

这些数字与df来自同一来源，但变化很快。似乎每分钟消耗约100MB的磁盘空间。您还需要跟踪磁盘使用情况，因为它会随着时间的推移而发生变化，以便也可以通过df来捕获这种情况。

由于所谓的磁盘可用空间不足，ElasticSearch错误地暂停了索引编制

1 个答案: