我想使用自定义Comparator对原始Java数组进行排序,但是我遇到了类型错误。我认为comparator
函数正在创建Comparator<java.lang.Object>
而不是Comparator<Long>
,但我无法弄清楚如何解决这个问题。
这是一个最小的例子:
x.core=> (def x (double-array [4 3 5 6 7]))
#'x.core/x
x.core=> (java.util.Arrays/sort x (comparator #(> %1 %2)))
ClassCastException [D cannot be cast to [Ljava.lang.Object; x.core/eval1524 (form-init5588058267991397340.clj:1)
我已尝试在比较器功能中添加不同类型的提示,但坦率地说,我对该语言相对较新,基本上只是投掷飞镖。
我故意简化上面的例子,专注于关键问题,这是一个类型错误。在下面的部分中,我尝试提供一些更详细的信息来激发问题并说明我使用自定义Comparator的原因。
动机
我要做的是复制R&#39 order
函数,其功能如下:
> x = c(7, 2, 5, 3, 1, 4, 6)
> order(x)
[1] 5 2 4 6 3 7 1
> x[order(x)]
[1] 1 2 3 4 5 6 7
正如您所看到的,它返回将对其输入向量进行排序的索引的排列。
这是Clojure中的一个有效解决方案:
(defn order
"Permutation of indices sorted by x"
[x]
(let [v (vec x)]
(sort-by #(v %) (range (count v)))))
x.core=> (order [7 2 5 3 1 4 6])
(4 1 3 5 2 6 0)
(注意,当Clojure为0索引时,R是1索引的。)诀窍是将一个向量(即x [0, 1, ..., (count x)]
的索引除以向量x本身。
R vs. Clojure Performance
不幸的是,我对此解决方案的性能感到困扰。 R解决方案很多更快:
> x = runif(1000000)
> system.time({ y = order(x) })
user system elapsed
0.041 0.004 0.046
对应的Clojure代码:
x.core=> (def x (repeatedly 1000000 rand))
#'x.core/x
x.core=> (time (def y (order x)))
"Elapsed time: 2857.216452 msecs"
#'x.core/y
Primitive Arrays the Solution?
我发现原始数组倾向于在与R:
相当的时间内排序> x = runif(1000000)
> system.time({ y = sort(x) })
user system elapsed
0.061 0.005 0.069
VS。
x.core=> (def x (double-array (repeatedly 1000000 rand)))
#'x.core/x
x.core=> (time (java.util.Arrays/sort x))
"Elapsed time: 86.827277 msecs"
nil
这是我尝试将自定义Comparator与java.util.Arrays类一起使用的动机。我希望速度可与R相媲美。
我应该补充一点,我可以使用带有ArrayList的自定义Comparator,如下所示,但性能并不比我的启动功能好:
(defn order2
[x]
(let [v (vec x)
compx (comparator (fn [i j] (< (v i) (v j))))
ix (java.util.ArrayList. (range (count v)))]
(java.util.Collections/sort ix compx)
(vec ix)))
即使您只想提供一些一般的Clojure建议,我们也将不胜感激。我还在学习这门语言,并且很有乐趣。 : - )
修改的
Per Carcigenicate的答案如下,
(defn order
[x]
(let [ix (int-array (range (count x)))]
(vec (-> (java.util.Arrays/stream ix)
(.boxed)
(.sorted (fn [i j] (< (aget x i) (aget x j))))
(.mapToInt
(proxy [java.util.function.ToIntFunction] []
(applyAsInt [^long d] d)))
(.toArray)))))
将起作用:
x.core=> (def x (double-array [5 3 1 3.14 -10]))
#'x.core/x
x.core=> (order x)
[4 2 1 3 0]
x.core=> (map #(aget x %) (order x))
(-10.0 1.0 3.0 3.14 5.0)
不幸的是,它超级慢。我猜原始人可能不是答案。
答案 0 :(得分:1)
这是使用带有随机枢轴的快速排序的order
函数的Clojure实现。它与R相当接近:使用你的100万双打基准,我的定时时间大多在520-530毫秒范围内,而R一般在500毫秒左右徘徊。
更新:使用非常基本的双线程版本(2x快速排序,然后是合并步骤,生成输出向量)我的时间明显改善 - 最差基准平均值为415 ms,否则我倾向于在325-365 ms范围内得到结果。有关双线程版本,请参阅此消息的结尾,或者如果您更喜欢gist形式的版本,则它们是 - two-threaded, single-threaded
请注意,它将输入作为中间步骤倒入一个双精度数组中,最终返回long的向量。在我的盒子上将一百万个双打倒入一个向量似乎只需要超过30毫秒,所以如果你对数组结果感到满意,你可以不用这一步。
主要并发症是invokePrim
- 从Clojure 1.9.0-RC1开始,在该位置的常规函数调用将导致拳击。其他方法也是可行的,但这很有效,而且看起来很简单。
有关基准测试结果,请参阅此消息的结尾。第一次运行的较低分位数结果实际上是报告的最佳结果
(defn order2 [xs]
(let [rnd (java.util.Random.)
a1 (double-array xs)
a2 (long-array (alength a1))]
(dotimes [i (alength a2)]
(aset a2 i i))
(letfn [(quicksort [^long l ^long h]
(if (< l h)
(let [p (.invokePrim ^clojure.lang.IFn$LLL partition l h)]
(quicksort l (dec p))
(quicksort (inc p) h))))
(partition ^long [^long l ^long h]
(let [pidx (+ l (.nextInt rnd (- h l)))
pivot (aget a1 pidx)]
(swap1 a1 pidx h)
(swap2 a2 pidx h)
(loop [i (dec l)
j l]
(if (< j h)
(if (< (aget a1 j) pivot)
(let [i (inc i)]
(swap1 a1 i j)
(swap2 a2 i j)
(recur i (inc j)))
(recur i (inc j)))
(let [i (inc i)]
(when (< (aget a1 h) (aget a1 i))
(swap1 a1 i h)
(swap2 a2 i h))
i)))))
(swap1 [^doubles a ^long i ^long j]
(let [tmp (aget a i)]
(aset a i (aget a j))
(aset a j tmp)))
(swap2 [^longs a ^long i ^long j]
(let [tmp (aget a i)]
(aset a i (aget a j))
(aset a j tmp)))]
(quicksort 0 (dec (alength a1)))
(vec a2))))
基准测试结果(注意:第一次测试使用问题文本中定义的x
- (def x (repeatedly 1000000 rand))
;它也使用c/bench
,而以下运行使用c/quick-bench
) :
user> (c/bench (order2 x))
Evaluation count : 120 in 60 samples of 2 calls.
Execution time mean : 522.485408 ms
Execution time std-deviation : 33.490530 ms
Execution time lower quantile : 470.089782 ms ( 2.5%)
Execution time upper quantile : 575.687990 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order2 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 527.020004 ms
Execution time std-deviation : 14.846061 ms
Execution time lower quantile : 507.175127 ms ( 2.5%)
Execution time upper quantile : 543.675752 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order2 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 513.476501 ms
Execution time std-deviation : 12.828449 ms
Execution time lower quantile : 497.164534 ms ( 2.5%)
Execution time upper quantile : 525.094463 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order2 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 529.826816 ms
Execution time std-deviation : 21.454522 ms
Execution time lower quantile : 508.547461 ms ( 2.5%)
Execution time upper quantile : 552.592925 ms (97.5%)
Overhead used : 15.378363 ns
nil
来自同一个方框的一些R时间进行比较:
> system.time({ y = order(x) })
user system elapsed
0.512 0.004 0.514
> system.time({ y = order(x) })
user system elapsed
0.496 0.000 0.496
> system.time({ y = order(x) })
user system elapsed
0.508 0.000 0.510
> system.time({ y = order(x) })
user system elapsed
0.508 0.000 0.513
> system.time({ y = order(x) })
user system elapsed
0.496 0.000 0.499
> system.time({ y = order(x) })
user system elapsed
0.500 0.000 0.502
更新:双线程Clojure版本:
(defn order3 [xs]
(let [rnd (java.util.Random.)
a1 (double-array xs)
a2 (long-array (alength a1))]
(dotimes [i (alength a2)]
(aset a2 i i))
(letfn [(quicksort [^long l ^long h]
(if (< l h)
(let [p (.invokePrim ^clojure.lang.IFn$LLL partition l h)]
(quicksort l (dec p))
(quicksort (inc p) h))))
(partition ^long [^long l ^long h]
(let [pidx (+ l (.nextInt rnd (- h l)))
pivot (aget a1 pidx)]
(swap1 a1 pidx h)
(swap2 a2 pidx h)
(loop [i (dec l)
j l]
(if (< j h)
(if (< (aget a1 j) pivot)
(let [i (inc i)]
(swap1 a1 i j)
(swap2 a2 i j)
(recur i (inc j)))
(recur i (inc j)))
(let [i (inc i)]
(when (< (aget a1 h) (aget a1 i))
(swap1 a1 i h)
(swap2 a2 i h))
i)))))
(swap1 [^doubles a ^long i ^long j]
(let [tmp (aget a i)]
(aset a i (aget a j))
(aset a j tmp)))
(swap2 [^longs a ^long i ^long j]
(let [tmp (aget a i)]
(aset a i (aget a j))
(aset a j tmp)))]
(let [lim (alength a1)
mid (quot lim 2)
f1 (future (quicksort 0 (dec mid)))
f2 (future (quicksort mid (dec lim)))]
@f1
@f2
(loop [out (transient [])
i 0
j mid]
(cond
(== i mid)
(persistent!
(if (== j lim)
out
(reduce (fn [out j]
(conj! out (aget a2 j)))
out
(range j lim))))
(== j lim)
(persistent!
(reduce (fn [out i]
(conj! out (aget a2 i)))
out
(range i mid)))
:else
(let [ie (aget a1 i)
je (aget a1 j)]
(if (< ie je)
(recur (conj! out (aget a2 i)) (inc i) j)
(recur (conj! out (aget a2 j)) i (inc j))))))))))
这个的一些基准测试结果:
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order3 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 325.351056 ms
Execution time std-deviation : 3.511578 ms
Execution time lower quantile : 321.947510 ms ( 2.5%)
Execution time upper quantile : 330.375038 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order3 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 339.422989 ms
Execution time std-deviation : 19.929177 ms
Execution time lower quantile : 318.996436 ms ( 2.5%)
Execution time upper quantile : 366.113347 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order3 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 415.171336 ms
Execution time std-deviation : 13.624262 ms
Execution time lower quantile : 393.242455 ms ( 2.5%)
Execution time upper quantile : 428.881001 ms (97.5%)
Overhead used : 15.378363 ns
Found 1 outliers in 6 samples (16.6667 %)
low-severe 1 (16.6667 %)
Variance from outliers : 13.8889 % Variance is moderately inflated by outliers
nil
user> (let [x (repeatedly 1000000 rand)]
(c/quick-bench (order3 x)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 324.547827 ms
Execution time std-deviation : 5.196817 ms
Execution time lower quantile : 318.541727 ms ( 2.5%)
Execution time upper quantile : 331.878289 ms (97.5%)
Overhead used : 15.378363 ns
nil
user> (c/bench (order3 x))
Evaluation count : 180 in 60 samples of 3 calls.
Execution time mean : 361.529793 ms
Execution time std-deviation : 45.285047 ms
Execution time lower quantile : 307.535934 ms ( 2.5%)
Execution time upper quantile : 446.679687 ms (97.5%)
Overhead used : 15.378363 ns
Found 1 outliers in 60 samples (1.6667 %)
low-severe 1 (1.6667 %)
Variance from outliers : 78.9377 % Variance is severely inflated by outliers
nil