NumPy中的累积加法/乘法

时间:2015-12-09 15:34:41

标签: python performance numpy recursion vectorization

有一个相对简单的代码块,它循环遍历两个数组,相乘并累加:

su

有没有办法在没有迭代的情况下做到这一点?我想可以使用cumsum / cumprod,但我很难搞清楚如何使用。当你逐步分解发生的事情时,它看起来像这样:

import numpy as np

a = np.array([1, 2, 4, 6, 7, 8, 9, 11])
b = np.array([0.01, 0.2, 0.03, 0.1, 0.1, 0.6, 0.5, 0.9])
c = []
d = 0
for i, val in enumerate(a):
    d += val
    c.append(d)
    d *= b[i]

编辑以澄清:我对列表(或数组)感兴趣 c

4 个答案:

答案 0 :(得分:6)

在每次迭代中,你都有 -

d[n+1] = d[n] + a[n]
d[n+1] = d[n+1] * b[n]

因此,基本上 -

d[n+1] = (d[n] + a[n]) * b[n]

即。 -

d[n+1] = (d[n]* b[n]) + K[n]   #where `K[n] = a[n] * b[n]`

现在,如果你记下直到n = 2个案例的表达式,使用这个公式,你会有 -

  

d [1] = d [0] * b [0] + K [0]

     

d [2] = d [0] * b [0] * b [1] + K [0] * b [1] + K [1]

     

d [3] = d [0] * b [0] * b [1] * b [2] + K [0] * b [1] * b [2] + K [1] * b [ 2] + K [2]

Scalars      :    b[0]*b[1]*b[2]     b[1]*b[2]        b[2]       1 
Coefficients :         d[0]             K[0]          K[1]       K[2]

因此,你需要b的反向cumprod,用K数组执行元素乘法。最后,要获得c,请执行cumsum,然后存储c,然后再缩小b,这样您就需要缩小cumsum版本反转的b cumprod。

最终实施将如下所示 -

# Get reversed cumprod of b and pad with `1` at the end
b_rev_cumprod = b[::-1].cumprod()[::-1]
B = np.hstack((b_rev_cumprod,1))

# Get K
K = a*b

# Append with 0 at the start, corresponding starting d
K_ext = np.hstack((0,K))

# Perform elementwsie multiplication and cumsum and scale down for final c
sums = (B*K_ext).cumsum()
c = sums[1:]/b_rev_cumprod

运行时测试并验证输出

功能定义 -

def original_approach(a,b):
    c = []
    d = 0
    for i, val in enumerate(a):
        d = d+val
        c.append(d)
        d = d*b[i]
    return c

def vectorized_approach(a,b): 
    b_rev_cumprod = b[::-1].cumprod()[::-1]
    B = np.hstack((b_rev_cumprod,1))

    K = a*b
    K_ext = np.hstack((0,K))
    sums = (B*K_ext).cumsum()
    return sums[1:]/b_rev_cumprod

运行时和验证

案例#1:OP示例案例

In [301]: # Inputs
     ...: a = np.array([1, 2, 4, 6, 7, 8, 9, 11])
     ...: b = np.array([0.01, 0.2, 0.03, 0.1, 0.1, 0.6, 0.5, 0.9])
     ...: 

In [302]: original_approach(a,b)
Out[302]: 
[1,
 2.0099999999999998,
 4.4020000000000001,
 6.1320600000000001,
 7.6132059999999999,
 8.7613205999999995,
 14.256792359999999,
 18.128396179999999]

In [303]: vectorized_approach(a,b)
Out[303]: 
array([  1.        ,   2.01      ,   4.402     ,   6.13206   ,
         7.613206  ,   8.7613206 ,  14.25679236,  18.12839618])

案例#2:大输入案例

In [304]: # Inputs
     ...: N = 1000
     ...: a = np.random.randint(0,100000,N)
     ...: b = np.random.rand(N)+0.1
     ...: 

In [305]: np.allclose(original_approach(a,b),vectorized_approach(a,b))
Out[305]: True

In [306]: %timeit original_approach(a,b)
1000 loops, best of 3: 746 µs per loop

In [307]: %timeit vectorized_approach(a,b)
10000 loops, best of 3: 76.9 µs per loop

请注意,对于极其庞大的输入数组情况,如果b元素是如此小的分数,由于累积运算,b_rev_cumprod的初始数可能会出现zeros结果在NaNs的初始位置。

答案 1 :(得分:3)

让我们看看我们是否能够更快。我现在离开了纯粹的python世界,并表明这种纯粹的数字问题可以进一步优化。

这两名球员是@Divakar的快速矢量化版本:

def vectorized_approach(a,b): 
    b_rev_cumprod = b[::-1].cumprod()[::-1]
    B = np.hstack((b_rev_cumprod,1))

    K = a*b
    K_ext = np.hstack((0,K))
    sums = (B*K_ext).cumsum()
    return sums[1:]/b_rev_cumprod

和cython版本:

%%cython
import numpy as np
def cython_approach(long[:] a, double[:] b):
    cdef double d
    cdef size_t i, n
    n = a.shape[0]
    cdef double[:] c = np.empty(n)

    d = 0
    for i in range(n):
        d += a[i]
        c[i] = d
        d *= b[i]
    return c

cython版本比矢量化版本快5倍:

%timeit vectorized_approach(a,b) - > 10000 loops, best of 3: 43.4 µs per loop

%timeit cython_approach(a,b) - > 100000 loops, best of 3: 7.7 µs per loop

cython版本的另一个优点是它更具可读性。

最大的缺点是你要离开纯python并根据你的用例编译扩展模块可能不适合你。

答案 2 :(得分:2)

这对我有用,并且是矢量化的

b_mat = np.tile(b,(b.size,1)).T
b_mat = np.vstack((np.ones(b.size),b_mat))
np.fill_diagonal(b_mat,1)
b_mat[np.triu_indices(b.size)]=1

b_prod_mat = np.cumprod(b_mat,axis=0)
b_prod_mat[np.triu_indices(b.size)] = 0
np.fill_diagonal(b_prod_mat,1)

c = np.dot(b_prod_mat,a)
c

# output
array([  1.   ,   2.01 ,   4.402,   6.132,   7.613,   8.761,  14.257,
        18.128,  16.316])

我同意不容易看到最新情况。您的数组c可以写为矩阵向量乘法b_prod_mat * a,其中a是您的数组,b_prod_matb的特定产品组成。所有重点基本上都是创建b_prod_mat

答案 3 :(得分:1)

我不确定这比for循环好,但这是一种方式:

contains

它的作用是像这样创建一个大矩阵extension SequenceType where Generator.Element : Equatable

a.dot([np.concatenate((np.zeros(i), (1, ), b[i:-1])) for i in range(len(b))])

然后,您只需将向量A与矩阵1 b0 b0b1 b0b1b2 ... b0b1..bn-1 0 1 b1 b1b2 ... b1..bn-1 0 0 1 b2 ... ... 0 0 0 0 ... 1 相乘,即可获得预期结果。

相关问题