为什么在docker中运行openstack容器时会出现“Watchdog:BUG:Soft lockup CPU”错误。

时间:2017-11-10 05:17:09

标签: docker cpu openstack openstack-nova openstack-neutron

我刚刚使用kolla和kolla-ansible stable / pike版本部署了openstack。我部署在一个控制节点和一个计算节点上。它工作但是当我尝试使用地平线仪表板时,控制节点非常慢并且锁定并且终端输出以下内容:

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [runc:[2:INIT]:10527]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [fluentd:5005]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [keepalived:10664]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#15 stuck for 23s! [java:5604]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [neutron-openvsw:3101]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [cinder-schedule:3193]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [java:6186]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [docker-containe:6601]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#12 stuck for 22s! [keepalived:4295]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [keepalived:10666]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#9 stuck for 22s! [gmain:939]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [node:3261]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [neutron-l3-agen:5071]

Message from syslogd@openstackcontroller at Nov  7 21:46:39 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [irqbalance:898]

每当我停止在控制节点上运行的所有docker容器时,似乎没有任何更多的CPU锁定错误,但是一旦我再次启动所有在控制节点上运行openstack服务的docker容器,它就会启动再次运行非常慢,锁定错误再次开始

1 个答案:

答案 0 :(得分:0)

你能使用更新的Linux内核> 4.6.X?

使内核对软锁定发生恐慌并在恐慌时重新启动将有助于在某种程度上恢复系统。他们为我工作。

sysctl kernel.softlockup_panic=1  # this will panic on soft lockup
sysctl kernel.panic=60            # reset system on panic after 60 seconds

上述内核设置适用于每个https://github.com/moby/moby/issues/19758#issuecomment-272430028的几个人。