使用SSL / TLS加密的GlusterFS存储群集出错了。
我已经在很多方面尝试了好几天。我希望有人可以帮助我。
首先，我启用了GlusterFS的SSL并在每个对等节点中重新启动了glusterd。
然后，GlusterFS卷afr_vol_1的SSL将被启用并由GlusterFS本机客户端挂载在/ mnt / gluster上。
通常情况下，如果我的mount目录中只有子文件，我可以重复读取或写入。
但是，如果我的mount目录中的sub-dirs被重复读取或写入一段时间，我的mount dir会出错。
在这里，我使用了命令“find”。重现它。
错误信息如下：
find: ‘./gluster’: Transport endpoint is not connected
最后，如果我禁用了SSL并重新启动了glusterd，一切都会好的。
那么，我的SSL配置有什么问题吗？
我的操作系统是CentOS 7.2。
GlusterFS版本是glusterfs-3.13.1（也尝试过glusterfs-4.0.0rc1）。
OpenSSL版本是OpenSSL 1.0.1e-fips（也尝试过OpenSSL 1.1.0g）。
我创建了我的glusterfs.pem和gluster.ca并将它们复制到其他对等节点中：
# cd /etc/ssl/
# sudo openssl genrsa -out glusterfs.key 2048
# sudo openssl req -new -x509 -key glusterfs.key –days 3650 -subj "/CN=glusterfs" -out glusterfs.pem
# cat /etc/ssl/glusterfs.pem > glusterfs.ca
我启用了GlusterFS的SSL并在每个对等节点中重新启动了glusterd：
# sudo touch /var/lib/glusterd/secure-access
# systemctl restart glusterd.service
我启用了GlusterFS卷afr_vol_1的SSL：
# gluster volume set afr_vol_1 server.ssl on
# gluster volume set afr_vol_1 client.ssl on
# gluster volume set afr_vol_1 auth.ssl-allow '*'
# gluster volume set afr_vol_1 ssl.cipher-list 'ALL'
# gluster volume stop afr_vol_1
# gluster volume start afr_vol_1
GlusterFS卷afr_vol_1由GlusterFS本机客户端挂载在/ mnt / gluster上：
# mount -t glusterfs localhost:/afr_vol_1 /mnt/gluster/
mnt-gluster.log如下：
[2018-03-05 06:18:13.440850] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-afr_vol_1-client-2: error returned while attempting to connect to host:(null), port:0
[2018-03-05 06:18:13.441577] E [socket.c:2167:__socket_read_frag] 0-rpc: wrong MSG-TYPE (20538) received from 192.168.8.99:24007
[2018-03-05 06:18:13.442245] E [rpc-clnt.c:350:saved_frames_unwind] (--> /usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x12a)[0x7f1584f4e43a] (--> /usr/local/lib/libgfrpc.so.0(+0xc452)[0x7f1584d1b452] (--> /usr/local/lib/libgfrpc.so.0(+0xc55e)[0x7f1584d1b55e] (--> /usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f1584d1caed] (--> /usr/local/lib/libgfrpc.so.0(+0xe638)[0x7f1584d1d638] ))))) 0-afr_vol_1-client-2: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-03-05 06:18:13.441030 (xid=0x505d)
[2018-03-05 06:18:14.454846] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-afr_vol_1-client-1: error returned while attempting to connect to host:(null), port:0
[2018-03-05 06:18:14.455487] E [socket.c:2167:__socket_read_frag] 0-rpc: wrong MSG-TYPE (13199) received from 192.168.8.98:24007
[2018-03-05 06:18:14.456464] E [rpc-clnt.c:350:saved_frames_unwind] (--> /usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x12a)[0x7f1584f4e43a] (--> /usr/local/lib/libgfrpc.so.0(+0xc452)[0x7f1584d1b452] (--> /usr/local/lib/libgfrpc.so.0(+0xc55e)[0x7f1584d1b55e] (--> /usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f1584d1caed] (--> /usr/local/lib/libgfrpc.so.0(+0xe638)[0x7f1584d1d638] ))))) 0-afr_vol_1-client-1: forced unwinding frame type(GF-DUMP) op(DUMP(1)) called at 2018-03-05 06:18:14.455147 (xid=0x33ac)
[2018-03-05 06:18:15.043577] W [socket.c:592:__socket_rwv] 0-afr_vol_1-client-1: writev on 192.168.8.98:49152 failed (No data available)
[2018-03-05 06:18:15.043944] E [socket.c:2777:socket_poller] 0-afr_vol_1-client-1: poll error on socket
[2018-03-05 06:18:15.044065] I [MSGID: 114018] [client.c:2227:client_rpc_notify] 0-afr_vol_1-client-1: disconnected from afr_vol_1-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2018-03-05 06:18:15.044103] W [MSGID: 108001] [afr-common.c:5383:afr_notify] 0-afr_vol_1-replicate-0: Client-quorum is not met
[2018-03-05 06:18:15.044856] E [rpc-clnt.c:350:saved_frames_unwind] (--> /usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x12a)[0x7f17d9c7644a] (--> /usr/local/lib/libgfrpc.so.0(+0xc452)[0x7f17d9a43452] (--> /usr/local/lib/libgfrpc.so.0(+0xc55e)[0x7f17d9a4355e] (--> /usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f17d9a44aed] (--> /usr/local/lib/libgfrpc.so.0(+0xe638)[0x7f17d9a45638] ))))) 0-afr_vol_1-client-1: forced unwinding frame type(GlusterFS 3.3) op(RELEASEDIR(42)) called at 2018-03-05 06:18:15.043227 (xid=0x2ddf)
[2018-03-05 06:18:15.045142] E [rpc-clnt.c:350:saved_frames_unwind] (--> /usr/local/lib/libglusterfs.so.0(_gf_log_callingfn+0x12a)[0x7f17d9c7644a] (--> /usr/local/lib/libgfrpc.so.0(+0xc452)[0x7f17d9a43452] (--> /usr/local/lib/libgfrpc.so.0(+0xc55e)[0x7f17d9a4355e] (--> /usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f17d9a44aed] (--> /usr/local/lib/libgfrpc.so.0(+0xe638)[0x7f17d9a45638] ))))) 0-afr_vol_1-client-1: forced unwinding frame type(GlusterFS 3.3) op(RELEASEDIR(42)) called at 2018-03-05 06:18:15.043690 (xid=0x2de0)
[2018-03-05 06:18:15.105207] W [fuse-bridge.c:2922:fuse_readdirp_cbk] 0-glusterfs-fuse: 28837: READDIRP => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.161590] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.161676] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.161730] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28839: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.162301] I [MSGID: 108031] [afr-common.c:2530:afr_local_discovery_cbk] 0-afr_vol_1-replicate-0: selecting local read_child afr_vol_1-client-0
[2018-03-05 06:18:15.163540] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.163619] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.163689] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28840: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.230220] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.230331] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.230454] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28841: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.231788] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.231870] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.231942] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28842: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.295447] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.295537] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.295590] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28843: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.296719] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.296777] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.296820] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28844: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-03-05 06:18:15.358443] W [MSGID: 108027] [afr-common.c:2255:afr_attempt_readsubvol_set] 0-afr_vol_1-replicate-0: no read subvols for /
[2018-03-05 06:18:15.358538] E [MSGID: 101046] [dht-common.c:1904:dht_revalidate_cbk] 0-afr_vol_1-dht: dict is null
[2018-03-05 06:18:15.358590] W [fuse-bridge.c:896:fuse_attr_cbk] 0-glusterfs-fuse: 28845: LOOKUP() / => -1 (Transport endpoint is not connected)
另一个对等节点中的glusterd.log如下：
[2018-03-05 06:18:14.931907] E [socket.c:2777:socket_poller] 0-socket.management: poll error on socket
[2018-03-05 06:18:34.555460] E [socket.c:2785:socket_poller] 0-socket.management: socket_poller 192.168.8.97:49141 failed (Input/output error)
答案 0 :(得分：0)
嗯，这个问题自己解决了...... -_- b
如果按源代码安装GlusterFS，除非正确修改了configure，否则会出现与上面相同的错误。
但是，如果你按rpm安装GlusterFS 就可以了！
感谢。