Question

我有一系列需要并行处理的客户。我尝试为此使用for (uint i=0; i<frequentLines.size(); i++) { for (int j=0; j<64; j++) { std::get<0>(frequentLines)[i][j] } }。结果非常缓慢，比顺序实现要慢得多。内部函数pmap有一个事务。显然，pmap一次启动所有事务，最终导致重试性能。您最好的并行化方法是什么？

process-customer

编辑： (defn process-customers [customers] (doall (pmap (fn [sub-customers] (doseq [customer sub-customers] (process-customer customer))) (partition-all 10 customers))))函数涉及以下步骤。为简洁起见，我写了一些步骤。所有步骤都在一个交易内部，以确保另一笔并行交易不会引起负库存之类的矛盾。

process-customer

编辑2：：以下版本的(defn- process-customer [customer] "Process `customer`. Consists of three steps: 1. Finding all stores in which the requested products are still available. 2. Sorting the found stores to find the cheapest (for the sum of all products). 3. Buying the products by updating the `stock`. )与上面的并行版本process-customers具有相同的性能。下面显然是顺序的。

process-customers

Answer 1

我假设您的交易在process-customer的整个生命周期内都处于锁定状态。由于所有客户都在争夺相同的商店，这将很慢。如果您可以将流程分为两个阶段：1）报价和2）履行并仅在（2）上应用事务，那么性能应该会好得多。或者，如果您购买agent编程，则会在消息级别自动为您定义事务边界。您可以考虑以下示例：

(defn get-best-deal
  "Returns the best deal for a given order with given stores (agent)"
  [stores order]
  ;;
  ;; request for quotation from 1000 stores (in parallel)
  ;;
  (doseq [store stores]
    (send store get-quote order))
  ;;
  ;; wait for reply, up to 0.5s
  ;;
  (apply await-for 500 stores)
  ;;
  ;; sort and find the best store
  ;;
  (when-let [best-store (->> stores
                             (filter (fn [store] (get-in @store [:quotes order])))
                             (sort-by (fn [store] (->> (get-in @store [:quotes order])
                                                       vals
                                                       (reduce +))))
                             first)]
    {:best-store best-store
     :invoice-id (do
                   ;; execute the order
                   (send best-store fulfill order)
                   ;; wait for the transaction to complete
                   (await best-store)
                   ;; get an invoice id
                   (get-in @best-store [:invoices order]))}))

，并从100种产品中的100个订单（总共289个订单项）中找到1,000家商店的最佳交易：

(->> orders
       (pmap (partial get-best-deal stores))
       (filter :invoice-id)
       count
       time)
;; => 57
;; "Elapsed time: 312.002328 msecs"

示例业务逻辑：

(defn get-quote
  "issue a quote by checking inventory"
  [store {:keys [order-items] :as order}]
  (if-let [quote (->> order-items
                   (reduce reduce-inventory
                           {:store store
                            :quote nil})
                   :quote)]
    ;; has inventory to generate a quote
    (assoc-in store [:quotes order] quote)
    ;; no inventory
    (update store :quotes dissoc order)))

(defn fulfill
  "fulfill an order if previuosly quoted"
  [store order]
  (if-let [quote (get-in store [:quotes order])]
    ;; check inventory again and generate invoice
    (let [[invoice inventory'] (check-inventory-and-generate-invoice store order)]
      (cond-> store
        invoice (->
                  ;; register invoice
                  (assoc-in [:invoices order] invoice)
                  ;; invalidate the quote
                  (update :quotes dissoc order)
                  ;; update inventory
                  (assoc :inventory inventory'))))
    ;; not quoted before
    store))

如何在Clojure中执行并行事务

1 个答案: