Question

我正在Clojure上测试Alia在6节点Cassandra集群上的性能。即使多线程我也只能获得大约400次写入/秒。使用Firebrand Cassandra驱动程序并在Java中手动处理线程，我们可以通过96个线程获得5000次写入/秒。

我在这里使用座席时做错了什么？在这台运行的机器上，CPU使用率仅为25％左右。这似乎非常低。

更新：在Alia建议的作者中，使用预准备语句而不是原始语句以同步，单线程方式实现高达2500 /秒的增益。我仍然需要通过Clojure的多线程测试这个，并分别利用Alia /底层Java驱动程序内置的异步函数来查看哪个更快。

更新2：我现在通过额外利用驱动程序内置的异步功能，看到类似于mpenet的结果。

(ns alia-perf-test.core
  (:gen-class)
  (:require [qbits.alia :as alia]
            [qbits.hayt :as hayt]))

(defn exec-query [session query]
  (alia/execute session (hayt/->raw query)))

(defmacro time-query
  [expr]
  `(let [start# (. System (nanoTime))
         ret# ~expr]
     (/ (double (- (. System (nanoTime)) start#)) 1000000.0)))

(defn write-entity
  [total-time session entity]
  (let [query (hayt/->raw (hayt/insert :entities (hayt/values entity) (hayt/using :timestamp 1234)))
        query-time (time-query (alia/execute session query))]
      (+ total-time query-time)))

(defn generate-entity []
  {:id (str (java.util.UUID/randomUUID)) :num 0})

(defn write-something
  [write-agent session]
  (send-off write-agent write-entity
        session 
        (generate-entity)))

(defn -main [& args]
  (let [cluster (alia/cluster ["server1"
                               "server2" 
                               "server3" 
                               "server4" 
                               "server5" 
                               "server6"]
                              :pooling-options {:core-connections-per-host [:local 16 :remote 16]
                                                :max-connections-per-host  [:local 1000 :remote 1000]
                                                :max-simultaneous-requests-per-connection [:local 32 :remote 32]
                                                :min-simultaneous-requests-per-connection [:local 16 :remote 16]})
        session (alia/connect cluster)]
    (alia/set-consistency! :any)
    (exec-query session (hayt/create-keyspace :aliaperftest
                                              (hayt/with {:replication
                                                          {:class "NetworkTopologyStrategy"
                                                           :dc1 3 :dc2 3}})))
    (exec-query session (hayt/use-keyspace :aliaperftest))
    (exec-query session (hayt/create-table :entities
                                           (hayt/column-definitions {:id :varchar
                                                                     :num :int
                                                                     :primary-key [:id]})))
    (let [num-entities 10000
          write-agent (agent 0)]
      (dotimes [n num-entities]
        (write-something write-agent session))
      (await write-agent)
      (println "Wrote" num-entities "entities in" @write-agent "ms -"
           (* (/ num-entities @write-agent) 1000.0) "ops/sec"))

    (exec-query session (hayt/drop-table :entities))
    (exec-query session (hayt/drop-keyspace :aliaperftest))
    (alia/shutdown session)
    (alia/shutdown cluster)
    (shutdown-agents)))

Answer 1

更新：我已经在异步模式下使用alia的成功处理程序（使用单个节点）在~8秒内完成100k请求，使用atom接收结果并等待所有响应到达。

根据您的设置和批处理，使用自定义执行程序也可以从中挤出更多性能，我没有走得那么远。见：https://github.com/mpenet/alia/blob/master/docs/guide.md#executors

Clojure Cassandra驾驶员表现

1 个答案: