Question

我正试图在我的项目中引入一些并发性。

我现在正在使用parallel gem，并且在并行进程内创建对象时遇到问题。

Parallel.map(["a"], :in_processes => 8) do |i|
  ActiveRecord::Base.connection_pool.with_connection do
    User.create
  end
end

结果：

/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:in `dump': no _dump_data is defined for class Binding (TypeError)
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:in `process_incoming_jobs'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:257:in `block in worker'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in `fork'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in `worker'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:238:in `block in create_workers'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in `each'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in `create_workers'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:201:in `work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:106:in `map'
    from (irb):15:in `irb_binding'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in `eval'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in `evaluate'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/context.rb:254:in `evaluate'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:159:in `block (2 levels) in eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:273:in `signal_status'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:156:in `block in eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:243:in `block (2 levels) in each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `loop'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `block in each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `catch'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:155:in `eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:70:in `block in start'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in `catch'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:47:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:8:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands.rb:41:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'
Parallel::DeadWorker: Parallel::DeadWorker
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:51:in `rescue in work'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:48:in `work'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:216:in `block (4 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:352:in `with_instrumentation'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:215:in `block (3 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in `loop'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in `block (2 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:65:in `block (2 levels) in in_threads'

并行块的任何变化似乎都会导致相同的错误。删除对create的调用可使程序按预期运行。

Answer 1

看起来你要返回的内容是不可转储的，所以要么返回nil，要么使用Parallel.each或者也许with_connection实际上不会返回用户对象但有些奇怪......你也可以尝试返回user.attributes而不是完整的用户对象

Answer 2

http://ruby.zigzo.com/2012/01/29/the-parallel-gem-and-postgresql-oh-and-rails/

根据上述Mario的帖子，PostgreSQL不允许对多个线程使用相同的连接。我想知道MySQL是否相同，加上MySQL可能会在访问时锁定。以下是在PostgreSQL上并行工作的解决方案：ActiveRecord :: Base.connection.reconnect！

用平行宝石创建对象

2 个答案: