在CircleCI上Postgres死锁

时间:2014-12-03 22:44:38

标签: postgresql circleci

最近,我将CircleCI上的单一构建并发增加到了11个实例。我只是间歇性地

  

PG :: TRDeadlockDetected

某些构建实例中的错误。我对CircleCI架构知之甚少,无法对此进行故障排除。有没有人对此有所了解?

PG::TRDeadlockDetected: ERROR:  deadlock detected
DETAIL:  Process 11634 waits for AccessExclusiveLock on relation 19091 of database 18000; blocked by process 11840.
Process 11840 waits for AccessShareLock on relation 18632 of database 18000; blocked by process 11634.
HINT:  See server log for query details.
: TRUNCATE TABLE "addresses", "users", "hits" RESTART IDENTITY CASCADE; (ActiveRecord::StatementInvalid)
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:128:in `exec'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:128:in `block in execute'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:442:in `block in log'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.10/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/abstract_adapter.rb:437:in `log'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:127:in `execute'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/active_record/truncation.rb:148:in `truncate_tables'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/active_record/truncation.rb:202:in `block in clean'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.10/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:17:in `disable_referential_integrity'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/active_record/truncation.rb:198:in `clean'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/base.rb:86:in `clean'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/configuration.rb:79:in `block in clean'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/configuration.rb:79:in `each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/database_cleaner-1.2.0/lib/database_cleaner/configuration.rb:79:in `clean'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-rails-1.4.0/lib/cucumber/rails/hooks/database_cleaner.rb:9:in `After'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/rb_support/rb_hook.rb:14:in `invoke'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:114:in `invoke'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:108:in `block in execute_after'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:107:in `reverse_each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:107:in `execute_after'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:19:in `after'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:112:in `block in fire_hook'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:111:in `each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:111:in `fire_hook'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:112:in `after'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:99:in `before_and_after'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:81:in `block in with_hooks'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:120:in `call'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:94:in `call'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:94:in `block (3 levels) in execute_around'
/home/ubuntu/icn/features/support/env.rb:84:in `call'
/home/ubuntu/icn/features/support/env.rb:84:in `block in <top (required)>'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/rb_support/rb_hook.rb:14:in `invoke'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:114:in `invoke'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:93:in `block (2 levels) in execute_around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:97:in `call'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/language_support/language_methods.rb:8:in `around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:123:in `call'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime/support_code.rb:123:in `around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:93:in `around'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:80:in `with_hooks'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:13:in `execute'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/scenario.rb:32:in `block in accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/scenario.rb:31:in `accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/feature.rb:38:in `block in accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/feature.rb:37:in `each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/feature.rb:37:in `accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/features.rb:28:in `block in accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/features.rb:17:in `each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/features.rb:17:in `each'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/features.rb:27:in `accept'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/runtime.rb:48:in `run!'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/lib/cucumber/cli/main.rb:47:in `execute!'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/gems/cucumber-1.3.8/bin/cucumber:13:in `<top (required)>'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/bin/cucumber:23:in `load'
/home/ubuntu/icn/vendor/bundle/ruby/2.0.0/bin/cucumber:23:in `<main>'

1 个答案:

答案 0 :(得分:0)

当并行运行构建时,CircleCI使用独立的容器,每个容器都有自己的数据库实例,因此问题不会来自彼此交互的并行执行。

然而,有可能在整个机器上拆分测试会暴露出意外行为或测试之间的不干净设置或拆卸问题。

当它出现间歇性时显然更难,但是你可以在启用ssh的情况下重新运行构建,然后在出现错误并检查所有日志时连接到容器。这应该可以让你更好地了解死锁发生的位置。

此外,如果您需要帮助调试,CircleCI可以在app或sayhi@circleci.com上直接支持。