Python分析 - 内联和调用函数之间的差异

时间:2014-04-10 23:21:00

标签: python numpy function-call

我有这段代码:

def getNeighbors(cfg, cand, adj):
    c_nocfg = np.setdiff1d(cand, cfg)
    deg = np.sum(adj[np.ix_(cfg, c_nocfg)], axis=0)
    degidx = np.where(deg) > 0
    nbs = c_nocfg[degidx]
    deg = deg[degidx]
    return nbs, deg

从邻接矩阵中检索邻居(以及它们在cfg中的节点所跨越的子图中的度数)。 内联代码给出了合理的性能(~10kx10k邻接矩阵作为布尔数组,10k候选者在cand中,子图cfg跨越500个节点):0.02s 但是,调用函数getNeighbors会导致大约0.5s的开销。

模拟

deg = np.sum(adj[np.ix_(cfg, c_nocfg)], axis=0)

deg = np.random.randint(500, size=c_nocfg.shape[0])

将函数调用的运行时间降低到0.002秒。 有人可以解释一下是什么原因造成我案件的巨大开销 - 毕竟,总和操作本身并不是太昂贵。

分析输出

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1    0.466    0.466    0.488    0.488 /home/x/x/x/utils/benchmarks.py:15(getNeighbors)
    1    0.000    0.000    0.019    0.019 /usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:1621(sum)
    1    0.000    0.000    0.019    0.019 /usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:23(_sum)
    1    0.019    0.019    0.019    0.019 {method 'reduce' of 'numpy.ufunc' objects}
    1    0.000    0.000    0.002    0.002 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:410(setdiff1d)
    1    0.000    0.000    0.001    0.001 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:284(in1d)
    2    0.001    0.000    0.001    0.000 {method 'argsort' of 'numpy.ndarray' objects}
    2    0.000    0.000    0.001    0.000 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:93(unique)
    2    0.000    0.000    0.000    0.000 {method 'sort' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 {numpy.core.multiarray.where}
    4    0.000    0.000    0.000    0.000 {numpy.core.multiarray.concatenate}
    2    0.000    0.000    0.000    0.000 {method 'flatten' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 /usr/local/lib/python2.7/dist-packages/numpy/lib/index_tricks.py:26(ix_)
    5    0.000    0.000    0.000    0.000 /usr/local/lib/python2.7/dist-packages/numpy/core/numeric.py:392(asarray)
    5    0.000    0.000    0.000    0.000 {numpy.core.multiarray.array}
    1    0.000    0.000    0.000    0.000 {isinstance}
    2    0.000    0.000    0.000    0.000 {method 'reshape' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 {range}
    2    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
    2    0.000    0.000    0.000    0.000 {issubclass}
    2    0.000    0.000    0.000    0.000 {method 'ravel' of 'numpy.ndarray' objects}
    6    0.000    0.000    0.000    0.000 {len}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

内联版:

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1    0.000    0.000    0.019    0.019 /usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:1621(sum)
    1    0.000    0.000    0.019    0.019 /usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:23(_sum)
    1    0.019    0.019    0.019    0.019 {method 'reduce' of 'numpy.ufunc' objects}
    1    0.000    0.000    0.002    0.002 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:410(setdiff1d)
    1    0.000    0.000    0.001    0.001 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:284(in1d)
    2    0.001    0.000    0.001    0.000 {method 'argsort' of 'numpy.ndarray' objects}
    2    0.000    0.000    0.000    0.000 /usr/local/lib/python2.7/dist-packages/numpy/lib/arraysetops.py:93(unique)
    2    0.000    0.000    0.000    0.000 {method 'sort' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 {numpy.core.multiarray.where}
    4    0.000    0.000    0.000    0.000 {numpy.core.multiarray.concatenate}
    2    0.000    0.000    0.000    0.000 {method 'flatten' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 /usr/local/lib/python2.7/dist-packages/numpy/lib/index_tricks.py:26(ix_)
    5    0.000    0.000    0.000    0.000 /usr/local/lib/python2.7/dist-packages/numpy/core/numeric.py:392(asarray)
    5    0.000    0.000    0.000    0.000 {numpy.core.multiarray.array}
    1    0.000    0.000    0.000    0.000 {isinstance}
    2    0.000    0.000    0.000    0.000 {method 'reshape' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 {range}
    2    0.000    0.000    0.000    0.000 {method 'ravel' of 'numpy.ndarray' objects}
    2    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
    2    0.000    0.000    0.000    0.000 {issubclass}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    6    0.000    0.000    0.000    0.000 {len}

测试样本数据:

np.random.seed(0)
adj = np.zeros(10000*10000, dtype=np.bool)
adj[np.random.randint(low=0, high=10000*10000+1, size=100000)] = True
adj = adj.reshape((10000, 10000))
cand = np.arange(adj.shape[0])
cfgs = np.random.choice(cand, size=500)

0 个答案:

没有答案