Question

我们最近开始遇到厨师 - 客户在跑步过程中死亡的麻烦，因为他们花了很多时间停留在通常进行得更快的运行列表的各个部分。我一直在我的家庭wifi上，我的同事一直在工作wifi，它一直有自己的连接问题。

如果你的ssh连接在chef-client运行时被机器中断，那是否会以看似无法解释的方式崩溃？我正在使用PutTY从我的Win7连接，而我的同事正在使用Apple终端应用程序。

我们一直运行的所有机器都是Ubuntu 12.04（在EC2中）并且剩余大量磁盘空间 - 它们仅使用~1GB且可以免费使用~5GB。

以下是来自/var/log/chef/client.log的日志输出（使用log_location中的/etc/chef/client.rb指令设置为described here）。

[2014-01-08T00:27:07+00:00] WARN: Nodejs user is nodejs
[2014-01-08T00:27:07+00:00] WARN: Cloning resource attributes for group[nodejs] from prior resource (CHEF-3694)
[2014-01-08T00:27:07+00:00] WARN: Previous group[nodejs]: /var/chef/cache/cookbooks/nodejs/recipes/default.rb:26:in `from_file'
[2014-01-08T00:27:07+00:00] WARN: Current  group[nodejs]: /var/chef/cache/cookbooks/spicoli-app/recipes/default.rb:38:in `from_file'
[2014-01-08T00:27:07+00:00] WARN: Cloning resource attributes for user[nodejs] from prior resource (CHEF-3694)
[2014-01-08T00:27:07+00:00] WARN: Previous user[nodejs]: /var/chef/cache/cookbooks/nodejs/recipes/default.rb:34:in `from_file'
[2014-01-08T00:27:07+00:00] WARN: Current  user[nodejs]: /var/chef/cache/cookbooks/spicoli-app/recipes/default.rb:46:in `from_file'
[2014-01-08T00:27:30+00:00] WARN: Environment is _default
[2014-01-08T00:27:30+00:00] WARN: Nodejs user is nodejs
[2014-01-08T02:04:54+00:00] ERROR: Running exception handlers
[2014-01-08T02:04:54+00:00] ERROR: Exception handlers complete
[2014-01-08T02:04:54+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2014-01-08T02:04:55+00:00] ERROR: Input/output error - <STDOUT>
[2014-01-08T02:04:57+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

错误堆栈跟踪就是这样：

Generated at 2014-01-08 02:04:54 +0000
Errno::EIO: Input/output error - <STDOUT>
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/base.rb:91:in `write'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/base.rb:91:in `puts'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/base.rb:91:in `puts'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:61:in `display_section'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:44:in `block (2 levels) in display'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:43:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:43:in `block in display'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:42:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/error_descriptor.rb:42:in `display'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/base.rb:130:in `display_error'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/base.rb:161:in `resource_failed'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/formatters/doc.rb:159:in `resource_failed'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/event_dispatch/dispatcher.rb:29:in `block in resource_failed'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/event_dispatch/dispatcher.rb:29:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/event_dispatch/dispatcher.rb:29:in `resource_failed'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource.rb:637:in `rescue in run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource.rb:643:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/resource_collection.rb:96:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/client.rb:433:in `converge'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/client.rb:500:in `do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/client.rb:199:in `block in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/client.rb:193:in `fork'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/client.rb:193:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/application.rb:208:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/application/client.rb:312:in `block in run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/application/client.rb:304:in `loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/application/client.rb:304:in `run_application'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/lib/chef/application.rb:66:in `run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.8.0/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:23:in `load'
/usr/bin/chef-client:23:in `<main>'

这是一个非常普遍的错误！但它确实表明STDOUT输出中断，这种情况在客户端断开时有意义。

编辑：根据要求，以下是client.rb文件的内容（名称模糊，当然。）

$ cat /etc/chef/client.rb
log_level        :auto
log_location     "/var/log/chef/client.log"
chef_server_url  "https://api.opscode.com/organizations/myapp"
validation_client_name "my-validator"
node_name "my-app-node"

编辑2 ：尝试使用sudo su -s /bin/bash root -c "screen chef-client"

我在午餐时屏幕终止，并在ShellOut npm install命令上记录超时。这是在厨师 - 客户坐在这个操作上超过一个小时之后。

[2014-01-09T16:39:07+00:00] WARN: Environment is _default
[2014-01-09T16:39:07+00:00] WARN: Nodejs user is nodejs
[2014-01-09T18:16:28+00:00] ERROR: Running exception handlers
[2014-01-09T18:16:28+00:00] ERROR: Exception handlers complete
[2014-01-09T18:16:28+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2014-01-09T18:16:31+00:00] ERROR: execute[npm-install-app] (spicoli-app::default line 110) had an error: Mixlib::ShellOut::CommandTimeout: command timed out:
---- Begin output of npm --registry http://my.npm.repo.amazonaws.com:5984/registry/_design/app/_rewrite install --cache /home/nodejs/.npm --tmp /home/nodejs/tmp

--- snip: install messages from npm ---

[2014-01-09T18:16:33+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

这是一个完全不同的错误。 stacktrace.out文件也明确提到ShellOut，因此它也完全不同。最奇怪的是，当我从命令行运行相同的npm命令时，在一分钟内完成。

所以我不确定是否有办法进一步诊断以前的失败，但我欢迎其他建议。有关此新故障的输入，我问this followup question。

Answer 1

如果你的ssh连接在chef-client运行的时候被机器中断了，是否会以看似无法解释的方式使运行崩溃？

好吧，堆栈跟踪似乎意味着类似的事情正在发生。消息显示"Errno::EIO: Input/output error - <STDOUT>"，这与我期望看到STDOUT是否通过已关闭的SSH通道的情况一致。

我建议两件事：

运行chef-client，将所有控制台输出重定向到文件;例如将> /tmp/log 2>&1添加到命令的末尾。（重定向需要在远程计算机上发生。）

在命令中添加-l debug以提高日志记录级别，如Opscode's technical FAQ所述。这可以揭示当前隐藏的线索。

看看你的第二次更新，它有某种防火墙或网络相关问题的标志。

厨师 - 客户中途奄奄一息

1 个答案: