我正在尝试摆脱heketi中的一堆孤立卷。尝试时,出现“错误”,然后显示有关我刚尝试删除的卷的信息,序列化为JSON。没有别的了。我试图挖掘日志,但它们什么也没透露。
这是我用来尝试删除卷的命令:
heketi-cli -s $HEKETI_CLI_SERVER --user admin --secret ${KEY} volume delete 22f1a960651f0f16ada20a15d68c7dd6
Error: {"size":30,"name":"vol_22f1a960651f0f16ada20a15d68c7dd6","durability":{"type":"none","replicate":{},"disperse":{}},"gid":2008,"glustervolumeoptions":["","cluster.post-op-delay-secs 0"," performance.client-io-threads off"," performance.open-behind off"," performance.readdir-ahead off"," performance.read-ahead off"," performance.stat-prefetch off"," performance.write-behind off"," performance.io-cache off"," cluster.consistent-metadata on"," performance.quick-read off"," performance.strict-o-direct on"," storage.health-check-interval 0",""],"snapshot":{"enable":true,"factor":1},"id":"22f1a960651f0f16ada20a15d68c7dd6","cluster":"e924a50aa93d9eae1132c60eb1f36310","mount":{"glusterfs":{"hosts":["<SECRET>"],"device":"<SECRET>:vol_22f1a960651f0f16ada20a15d68c7dd6","options":{"backup-volfile-servers":""}}},"blockinfo":{},"bricks":[{"id":"0f4c6d7f605e9368bfe3dc7cc117b69a","path":"/var/lib/heketi/mounts/vg_970f0faf60f8dfc6f6a0d6bd25bdea7c/brick_0f4c6d7f605e9368bfe3dc7cc117b69a/brick","device":"970f0faf60f8dfc6f6a0d6bd25bdea7c","node":"107894a855c9d2c34509b18272e6c298","volume":"22f1a960651f0f16ada20a15d68c7dd6","size":31457280}]}
请注意,第二行仅包含Error,然后是有关序列化为json的卷的信息。
该卷在gluster中不存在。我使用以下命令来验证该卷不再存在:
kubectl -n default exec -t -i glusterfs-rgz9g bash
gluster volume info
<shows volume i did not delete>
Kubernetes没有显示PersistentVolumeClaim或PersistentVolume:
kubectl get pvc -A
No resources found.
kubectl get pv -A
No resources found.
我尝试查看heketi日志,但它仅报告该卷的GET
kubectl -n default logs heketi-56f678775c-nrbwd
[negroni] 2019-11-25T21:29:19Z | 200 | 1.407715ms | <SECRET>:8080 | GET /volumes/22f1a960651f0f16ada20a15d68c7dd6
[negroni] 2019-11-25T21:29:19Z | 200 | 1.111984ms | <SECRET>:8080 | GET /volumes/22f1a960651f0f16ada20a15d68c7dd6
[negroni] 2019-11-25T21:29:19Z | 200 | 1.540357ms | <SECRET>:8080 | GET /volumes/22f1a960651f0f16ada20a15d68c7dd6
我尝试设置更详细的日志级别,但设置不成立:
heketi-cli -s $HEKETI_CLI_SERVER --user admin --secret ${KEY} loglevel set debug
Server log level updated
heketi-cli -s $HEKETI_CLI_SERVER --user admin --secret ${KEY} loglevel get
info
我的CLI使用
heketi-cli -v
heketi-cli v9.0.0
Heketi服务器正在运行:
kubectl -n default exec -t -i heketi-56f678775c-nrbwd bash
heketi -v
Heketi v9.0.0-124-gc2e2a4ab
基于日志,我认为heketi-cli出现问题,然后再也没有实际将POST或DELETE请求发送到heketi服务器。
如何进行调试?在这一点上,我唯一的解决方法是重新创建群集,但是我想避免这种情况,特别是在类似情况再次出现时。
答案 0 :(得分:0)
好像heketi-cli
中有一个错误,因为如果我使用ruby和curl手动创建请求,则可以删除该卷:
TOKEN=$(ruby makeToken.rb DELETE /volumes/22f1a960651f0f16ada20a15d68c7dd6)
curl -X DELETE -H "Authorization: Bearer $TOKEN" http://10.233.21.178:8080/volumes/22f1a960651f0f16ada20a15d68c7dd6
有关如何生成jwt令牌的信息,请参见https://github.com/heketi/heketi/blob/master/docs/api/api.md#authentication-model。
我手动创建了请求,期望得到更好的错误消息,命令行工具将其吞下。原来cli确实被破坏了。
用于制作jwt令牌的ruby代码(makeToken.rb)。您需要填写通行证和服务器。
#!/usr/bin/env ruby
require 'jwt'
require 'digest'
user = "admin"
pass = "<SECRET>"
server = "http://localhost:8080/"
method = "#{ARGV[0]}"
uri = "#{ARGV[1]}"
payload = {}
headers = {
iss: 'admin',
iat: Time.now.to_i,
exp: Time.now.to_i + 600,
qsh: Digest::SHA256.hexdigest("#{method}&#{uri}")
}
token = JWT.encode headers, pass, 'HS256'
print("token for\n")
print("#{method} #{uri}\n")
print("#{token}\n")