Question

我遇到LVMETAD在并行lvm操作期间崩溃的问题（vgscan，pvscan / lvscan / lvs / vgs）

在lvmetad上应用调试后，我可以看到vg_list操作与另一个线程重叠，导致链表中的泄漏，导致内存崩溃＆＃34;对于LVMETAD。

>Jul 17 15:04:14 NODE1 lvmeta.sh[4623]: token_update end len 3 pid 5763 new token filter:3239235440
>Jul 17 15:04:14 NODE1 lvmeta.sh[4623]: 5763 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 17 15:04:14 NODE1 lvmeta.sh[4623]: 5763 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:06:46 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:06:46 NODE1 lvmeta.sh[4623]: pv_list
>Jul 18 12:06:47 NODE1 lvmeta.sh[4623]: vg_lookup vgid wxfglR-s3f8-yc3z-sg8K-JcBi-o0Qy-Ic8z0w name VG_NODE1
>Jul 18 12:06:47 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid wxfglR-s3f8-yc3z-sg8K-JcBi-o0Qy-Ic8z0w
>Jul 18 12:06:48 NODE1 lvmeta.sh[4623]:   PV /dev/sda4   VG VG_NODE1       lvm2 [<106.71 GiB / 47.75 GiB free]
>Jul 18 12:06:48 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:06:48 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:06:48 NODE1 lvmeta.sh[4623]:   PV /dev/sda3   VG VG_System       lvm2 [29.29 GiB / 9.60 GiB free]
>Jul 18 12:06:48 NODE1 lvmeta.sh[4623]:   Total: 2 [136.00 GiB] / in use: 2 [136.00 GiB] / in no VG: 0 [0   ]
>Jul 18 12:07:01 NODE1 lvmeta.sh[4623]: 9936 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:01 NODE1 lvmeta.sh[4623]: 9936 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:01 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 9977 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 9977 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10003 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10003 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10030 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10030 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10055 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10055 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10083 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10083 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10102 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10102 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10129 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10129 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10153 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: 10153 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:02 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10177 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10177 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10200 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10200 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10225 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10225 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10250 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10250 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10276 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10276 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10297 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10297 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10324 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10324 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10344 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10344 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10360 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10360 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10375 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: 10375 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:03 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10395 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10395 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10424 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10424 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10426 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10426 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10438 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10438 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10446 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10446 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10456 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10456 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10467 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10467 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10484 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: 10484 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:04 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10502 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10502 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10536 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10536 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10551 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10551 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_clear_outdated_pvs vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: set info VG name none uuid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 cache_version 351 new_version 352
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10574 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10574 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: set info VG name none uuid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 cache_version 351 new_version 353
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10590 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10590 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_list
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_update vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: update_metadata begin arg_vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 arg_name VG_System pvid none
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: update_metadata existing vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 and existing name VG_System
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: update_metadata for VG_System ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 from 351 to 353
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: moving PV tpUO22-Ra0R-sXa4-bFH4-eFH1-z2Dr-Nf46dV to VG #orphan
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: moving PV tpUO22-Ra0R-sXa4-bFH4-eFH1-z2Dr-Nf46dV to VG ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: update_metadata is done for VG_System ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: pv_found pvid tpUO22-Ra0R-sXa4-bFH4-eFH1-z2Dr-Nf46dV on device 2051 matches existing
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: vg_lookup vgid ziVnMV-iS3r-PMBW-VVKO-pE9F-Q49h-5LWne2 name VG_System
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: 10649 global info flags none reason none token filter:3239235440 update_pid 0
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: *** Error in `/usr/bin/lvmetad': malloc(): smallbin double linked list corrupted: 0x000000ffdc0008c0 ***
>Jul 18 12:07:05 NODE1 lvmeta.sh[4623]: lvmetad: mm/dbg_malloc.c:271: dm_bounds_check_debug: Assertion `!"Memory smash"' failed.
>Jul 18 12:07:06 NODE1 lvmeta.sh[4623]: /usr/bin/lvmeta.sh: line 5:  5761 Aborted                 (core dumped) /usr/bin/lvmetad -f -s /run/lvm/lvmetad.socket -l debug

对我来说，每次重新启动主机时都可以重现。但它崩溃了，因为systemd启动脚本请求并行vgs / lvs加载到软件。

复制这一点非常简单：

for i in `seq 1 100`; do lvs & done

这会导致lvmetad崩溃。在以下版本

LVM版本：2.02.171（2）（2017-05-03）图书馆版本：1.02.140（2017-05-03）驱动程序版本：4.34.0 lvmetad版本：2.02.171（2）（2017-05-03）

可以帮助解决这个问题吗？

lvmetad与＆＃34;内存粉碎＆＃34;崩溃在lvm并行操作期间

0 个答案: