如何正确使用Sidekiq处理Rails中的后台任务

时间:2018-04-04 23:46:59

标签: ruby-on-rails heroku sidekiq shopify-app

所以,我' VE产生使用https://github.com/Shopify/shopify_app一个滑轨应用程序 - 和大部分具有该应用如预期工作 - 这'的目标是为从外部的库存管理API获得的产品数量,然后使用该库存管理系统中的最新数量更新Shopify中的变量数量。

我的问题是,对外部API的初始POST请求会响应大量产品 - 有时需要15秒。除此之外,我的应用程序的另一部分然后接受此响应,并且对于Shopify中也存在的响应中的每个产品,它将向Shopify发出PUT请求以更新变体数量。与初始请求一样,这也需要10-15秒。

我的问题是我在Heroku上托管应用程序,因此我已经达到了30秒的请求超时限制。因此,我需要使用后台工作程序将上述(可能两者)的至少一个请求偏移到工作队列。我已经推荐了广泛推荐的Sidekiq宝石 - https://github.com/mperham/sidekiq - 这很容易设置。

我的问题是,我不'知道如何从成品Sidekiq工人作业的结果,然后再使用该控制器内 - 我也不要'知道这是不是最好的做法(I& #39;对于Rails / App开发来说,这是一个新手。)

我'已经列入我的控制器(之前将它分解为工人),目前运行下面的应用程序 - 我想我只是需要一些建议 - 我会正确地做这个 - 应该有的这个逻辑是模型内如果是这样,那么该模型如何与Controller进行通信,那么Sidekiq将如何适应所有这些模型。

感谢任何建议或帮助,谢谢。

class StockManagementController < ShopifyApp::AuthenticatedController

require 'uri'
require 'net/http'
require 'json'
require 'nokogiri'
require 'open-uri'
require 'rexml/document'

def new
    @token = StockManagementController.new
end

def get_token

    url = URI('https://external.api.endpoint/api/v1/AuthToken')
    http = Net::HTTP.new(url.host, url.port)
    http.use_ssl = true
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE

    @HEROKU_ENV_USERNAME = ENV['HEROKU_ENV_USERNAME']
    @HEROKU_ENV_PASSWORD = ENV['HEROKU_ENV_PASSWORD']

    request = Net::HTTP::Post.new(url)
    request['content-type'] = 'application/x-www-form-urlencoded'
    request['cache-control'] = 'no-cache'
    request.body = 'username=' + @HEROKU_ENV_USERNAME + '&password=' + @HEROKU_ENV_PASSWORD + '&grant_type=password'
    response = http.request(request)
    responseJSON = JSON.parse(response.read_body)
    session[:accessToken] = responseJSON['access_token']

    if session[:accessToken]
        flash[:notice] = 'StockManagement token generation was successful.'
        redirect_to '/StockManagement/product_quantity'
    else
        flash[:alert] = 'StockManagement token generation was unsuccessful.'
    end
end

def product_quantity

    REXML::Document.entity_expansion_text_limit = 1_000_000

    @theToken = session[:accessToken]

    if @theToken

        url = URI('https://external.api.endpoint/api/v1/ProductQuantity')
        http = Net::HTTP.new(url.host, url.port)
        http.use_ssl = true
        http.verify_mode = OpenSSL::SSL::VERIFY_NONE

        request = Net::HTTP::Post.new(url)
        request['authorization'] = 'bearer ' + @theToken + ''
        request['content-type'] = 'application/xml'
        request['cache-control'] = 'no-cache'

        response = http.request(request)
        responseBody = response.read_body
        finalResponse = Hash.from_xml(responseBody).to_json
        resultQuantity = JSON.parse finalResponse

        @connectionType = resultQuantity['AutomatorResponse']['Type']
        @successResponse = resultQuantity['AutomatorResponse']['Success']
        @errorResponse = resultQuantity['AutomatorResponse']['ErrorMsg']

        productQuantityResponse = resultQuantity['AutomatorResponse']['ResponseString']
        xmlResponse = Hash.from_xml(productQuantityResponse).to_json
        jsonResponse = JSON.parse xmlResponse

        @fullResponse = jsonResponse['StockManagement']['Company']['InventoryQuantitiesByLocation']['InventoryQuantity']

        # This hash is used to store the final list of items that we need in order to display the item's we've synced, and to show the number of items we've sycned successfully.
        @finalList = Hash.new

        # This array is used to contain the available products - this is used later on as a way of only rendering
        @availableProducts = Array.new

        # Here we get all of the variant data from Shopify.
        @variants = ShopifyAPI::Variant.find(:all, params: {})

        # For each peace of variant data, we push all of the available SKUs in the store to the @availableProducts Array for use later
        @variants.each do |variant|
            @availableProducts << variant.sku
        end

        #Our final list of products which will contain details from both the Stock Management company and Shopify - we will use this list to run api calls against each item
        @finalProductList = Array.new

        puts "Final product list has #{@fullResponse.length} items."
        puts @fullResponse.inspect

        # We look through every item in the response from Company
        @fullResponse.each_with_index do |p, index|

            # We get the Quantity and Product Code
            @productQTY = p["QtyOnHand"].to_f.round
            @productCode = p["Code"].upcase

            # If the product code is found in the list of available products in the Shopify store...
            if @availableProducts.include? @productCode
                @variants.each do |variant|
                    if @productCode === variant.sku
                        if @productQTY != 0
                            @finalProductList << {
                                "sku" => variant.sku,
                                "inventory_quantity" => variant.inventory_quantity,
                                "old_inventory_quantity" => variant.old_inventory_quantity,
                                "id" => variant.id,
                                "company_sku" => @productCode,
                                "company_qty" => @productQTY
                            }
                        end
                    end
                end
            end
        end

        # If we get a successful response from StockManagement, proceed...
        if @finalProductList
            flash[:notice] = 'StockManagement product quantity check was successful.'

            puts "Final product list has #{@finalProductList.length} items."
            puts @finalProductList

            @finalProductList.each do |item|

                @productSKU = item["sku"]
                @productInventoryQuantity = item["inventory_quantity"]
                @productOldInventoryQuantity = item["old_inventory_quantity"]
                @productID = item["id"]
                @companySKU = item["company_sku"]
                @companyQTY = item["company_qty"]

                url = URI("https://example.myshopify.com/admin/variants/#{@productID}.json")

                http = Net::HTTP.new(url.host, url.port)
                http.use_ssl = true
                http.verify_mode = OpenSSL::SSL::VERIFY_NONE
                request = Net::HTTP::Put.new(url)
                request["content-type"] = 'application/json'
                request["authorization"] = 'Basic KJSHDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDF'
                request["cache-control"] = 'no-cache'
                request.body = "{\n\t\"variant\": {\n\t\t\"id\": #{@productID},\n\t\t\"inventory_quantity\": #{@companyQTY},\n\t\t\"old_inventory_quantity\": #{@productOldInventoryQuantity}\n\t}\n}"

                # This is the line that actually runs the put request to update the quantity.
                response = http.request(request)

                # Finally, we populate the finalList has with response information.
                @finalList[@companySKU] = ["","You had #{@productOldInventoryQuantity} in stock, now you have #{@companyQTY} in stock."]

            end

        else
            # If the overall sync failed, we flash an alert.
            flash[:alert] = 'Quantity synchronisation was unsuccessful.'

        end

        # Lastly we get the final number of items that were synchronised.
        @synchronisedItems = @finalList.length

        # We flash this notification, letting the user known how many products were successfully synchronised.
        flash[:notice] = "#{@synchronisedItems} product quantities were synchronised successfully."

        # We then pretty print this to the console for debugging purposes.
        pp @finalList

    else

        flash[:alert] = @errorResponse

    end
end
end

1 个答案:

答案 0 :(得分:1)

首先,您的product_quantity方法太长了。你应该把它分成更小的部分。 2,http.verify_mode = OpenSSL::SSL::VERIFY_NONE不应该在生产中完成。您提供的示例以及您的问题过于复杂,因此难以回答。听起来你需要对设计模式有一个基本的了解,这不是一个特定的红宝石问题。

如果您的应用需要在控制器内部进行实时API调用,那么这是一个糟糕的设计。您不希望保持任何类型的请求最多等待超过几秒钟。您应该首先考虑为什么需要提出这些请求。如果您需要快速访问它的数据,则应编写后台作业以按计划刮取数据并将其存储在您自己的数据库中。

如果您的应用的用户提出了需要等待API响应的请求,您可以编写一个工作人员来处理获取API数据,并最终使用actioncable向用户的浏览器发送响应。< / p>

对于您的常量定义,您可能应该在初始化程序中执行此操作,而{(1}}中的初始化程序将在运行时加载到您的应用程序中。您可以使用te my_app_root/config/initializers/constants.rb语法在需要的地方调用它们,但如果您更喜欢更简单的常量,请删除ENV[],因为ruby中的命名约定就是对象。

@