Question

我正在使用numpy.random.shuffle来计算2D数组的随机列上的统计量。 Python代码如下：

import numpy as np

def timeline_sample(series, num):
    random = series.copy()
    for i in range(num):
        np.random.shuffle(random.T)
        yield random

我得到的速度是这样的：

import numpy as np
arr = np.random.sample((50, 5000))

%%timeit
for series in timeline_sample(rnd, 100):
    np.sum(series)

1 loops, best of 3: 391 ms per loop

我尝试Cythonize这个函数，但我不知道如何替换np.random.shuffle的调用，并且函数慢了3倍。有谁知道如何加速或替换它？它目前是我的计划的瓶颈。

Cython代码：

cimport cython

import numpy as np
cimport numpy as np


@cython.boundscheck(False)
@cython.wraparound(False)
def timeline_sample2(double[:, ::1] series, int num):
    cdef double[:, ::1] random = series.copy()
    cdef int i
    for i in range(num):
        np.random.shuffle(random.T)
        yield random

Answer 1

这可能会提高速度：

from timeit import Timer

import numpy as np
arr = np.random.sample((50, 5000))

def timeline_sample(series, num):
    random = series.copy()
    for i in range(num):
        np.random.shuffle(random.T)
        yield random

 def timeline_sample_fast(series, num):
    random = series.T.copy()
    for i in range(num):
        np.random.shuffle(random)
        yield random.T

def timeline_sample_faster(series, num):
    length = arr.shape[1]
    for i in range(num):
        yield series[:, np.random.permutation(length)]

def consume(iterable):
    for s in iterable:
        np.sum(s)

min(Timer(lambda: consume(timeline_sample(arr, 1))).repeat(10, 10))
min(Timer(lambda: consume(timeline_sample_fast(arr, 1))).repeat(10, 10))
min(Timer(lambda: consume(timeline_sample_faster(arr, 1))).repeat(10, 10))
#>>> 0.2585161680035526
#>>> 0.2416607110062614
#>>> 0.04835709399776533

强迫它连续确实会增加时间，但不会增加时间：

def consume(iterable):
    for s in iterable:
        np.sum(np.ascontiguousarray(s))

min(Timer(lambda: consume(timeline_sample(arr, 1))).repeat(10, 10))
min(Timer(lambda: consume(timeline_sample_fast(arr, 1))).repeat(10, 10))
min(Timer(lambda: consume(timeline_sample_faster(arr, 1))).repeat(10, 10))
#>>> 0.2632228760048747
#>>> 0.25778737501241267
#>>> 0.07451769898761995

Answer 2

随机化行会更便宜，下面的代码在功能上是等效的但是在我的机器上快了大约3倍。

def timeline_sample_fast(series, num):
   random = series.T.copy()
   for i in range(num):
       np.random.shuffle(random)
       yield random.T



arr = np.random.sample((600, 50))

%%timeit                         
for s in timeline_sample(arr, 100):
    np.sum(s)

10 loops, best of 3: 55.5 ms per loop

%%timeit
for s in timeline_sample_fast(arr, 100):
   np.sum(s)

10 loops, best of 3: 18.6 ms per loop

是否有更快版本的numpy.random.shuffle？

2 个答案: