优化内存密集型脚本

时间:2011-08-18 12:19:32

标签: sql ruby-on-rails

我目前有这段代码:

  shop_category.shop_sub_categories.each do |sub_category|
     products << sub_category.products 
   end 

   products = products.flatten.uniq.compact


   data = products.first(100).map do |product|
    {
      :name => CGI.unescapeHTML(product.name),
      :manufacturer => product.manufacturer,
      :detail => product.description,
      :sales_rank => product.sales_rank,
      :price => product.price,
      :retailer => product.retailer
    }
   end

我发现这几乎占用大量内存,因为它占用了我96%的CPU:

   shop_category.shop_sub_categories.each do |sub_category|
     products << sub_category.products 
   end 

我正在考虑根据WHERE IN语句编写一些内容,以便我只能这样查询数据:

SELECT * FROM products JOIN shop_categorization ON product_id = products.id WHERE shop_sub_category_id IN(1,2,3,4,5)

我目前使用的脚本非常棒。我在想是否有一些事情可以做到更多Rails-y。

  sub_category_ids = shop_category.shop_sub_categories.map(&:id)
  sub_category_ids = sub_category_ids.join(',')
  products = Product.find_by_sql("SELECT * FROM products JOIN shop_sub_categorizations ON product_id = products.id WHERE shop_sub_categorizations.shop_sub_category_id IN (#{sub_category_ids}) LIMIT 100")

1 个答案:

答案 0 :(得分:1)

如果您只想要子类别ID,而不是:

  sub_category_ids = shop_category.shop_sub_categories.map(&:id)

您只需选择id列:

  sub_category_ids = shop_category.shop_sub_categories.select('id').map(&:id)

此外,要重新构建find_by_sql语句以使用ActiveRecord方法,您可以执行以下操作:

  products = Product.join("JOIN shop_sub_categorizations ON product_id = products.id").where(["shop_sub_categorizations.shop_sub_category_id IN (?)", sub_category_ids]).limit(100)

请记住,如果不指定要选择的列,则会增加选择返回所有列并将其映射到内存的计算开销。

编辑:

查看进程监视器以​​确定您的开销是Ruby脚本还是数据库。向数据库添加索引可能会降低数据库查询的CPU开销。有关SQL索引的更多信息,请访问:http://www.w3schools.com/sql/sql_create_index.asp