有效地插入嵌套对象

时间:2014-09-10 22:59:20

标签: sql ruby-on-rails ruby activerecord bulkinsert

如果我有以下架构:

class Sinbad < ActiveRecord::Base
  has_many :tinbads
  accepts_nested_attributes_for: :tinbads
end

class Tinbad < ActiveRecord::Base
  belongs_to :sinbad
  has_many :pinbads
  accepts_nested_attributes_for: :pinbads
end

class Pinbad < ActiveRecord::Base
  belongs_to :tinbad
end

并且Tinbad有几百Pinbads的情况并不少见,有没有一种常见的方法来创建嵌套的Sinbad而无需调用数百个查询?

我已经理解Active Record不支持批量插入,但有没有办法绕过这个不涉及手写SQL的方法?我查看了https://github.com/zdennis/activerecord-import,但它不支持嵌套对象。目前,SinbadController#create操作平均>400插入事务,这是最常用的操作。

以下是我不希望发生的事情的一个例子: https://gist.github.com/adamkuipers/12578343d31a651bee4a

我没有插入photos表N次,而只想插入一次。

1 个答案:

答案 0 :(得分:0)

我有完全相同的问题。我正在解析大型电子表格,并且用于存储数据的架构是嵌套的,因此我只插入一个&#34; Sinbad&#34;但成千上万的&#34; Pinbad&#34;可以立即插入...

我提出的加速插入的方法是批量插入模式的底部叶子(将模式可视化为树),因为这必须是具有最多实例数量的模型 - 在您的情况下, Pinbad实例。我们不能批量插入中间叶子,因为批量插入不允许获取插入的乘法模型的ID(例如,参见有关postgresql的讨论here)。所以它并不理想,但这是我发现使插入更有效的唯一方法(不改变架构本身)。

由于您需要自行保存对象,因此您必须删除accepts_nested_attributes_for,并且使用activerecord-import方便批量插入:

class Sinbad
  #
  # Let's imagine you still receive the params as if you were using accepts_nested_attributes,
  # Meaning :pinbads_attributes will be nested under :tinbads_attributes
  # that will be nested under :sinbad
  #
  def self.efficient_create params
    # I think AR doesn't like when attributes doesn't exist,
    # so we should keep the tinbads attributes somewhere else 
    tinbads_attributes = params[:tinbads_attributes]
    params.delete :tinbads_attributes
    sinbad = self.create! params

    # Array that will contain the attributes of the pinbads to bulk insert
    pinbads_to_save = []
    # ActiveRecords-Import needs to know which cols of Pinbad you insert
    pinbads_cols = [:tinbad_id, :name, :other]

    # We need to manually save the tinbads one by one,
    # but that's what happen when using accepts_nested_attributes_for
    tinbads_attributes.each do |attrs|
      pinbads_attribute = attrs[:pinbads_attributes]
      attrs.delete :pinbads_attibutes
      tinbad = sinbad.tinbads.create! attrs

      pinbads_attributes.each do |p_attrs|
        # Take care to put the attributes
        # in the same order than the pinbad_cols array
        pinbads_to_save << [tinbad.id, p_attrs[:name], p_attrs[:other]]
      end
    end

    # Now we can bulk insert the pinbads, using activerecord-import
    Pinbad.import_without_validations_or_callbacks pinbad_cols, pinbads_to_save
  end
end

这就是我在我的情况下所做的事情,并且由于架构层次结构中的最后一级创建了大多数实例,因此整体插入时间大大减少。在您的情况下,您将用1个大容量插入物替换大约400个Pinbad插入物。

希望有所帮助,我愿意接受任何建议或替代解决方案!