Question

如果我尝试使用numba的jit-compiler编译包含一系列条件的函数，则需要很长时间。该程序看起来基本上像

from numba import jit
import numpy as np

@jit(nopython=True)
def foo(a, b):
    valid = [
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0),
        (a - 1 >= 0) and (b - 1 >= 0)
    ]

foo(1, 1)

我排除了所有不会显着改变编译时间的内容。如果我使用超过20个元素，就会出现问题。

| elements | time |
-------------------
|    21    | 2.7s |
|    22    | 5.1s |
|    23    |  10s |
|   ...    |  ... |
-------------------

尽管如此，该功能运作良好。有人知道，为什么需要这么长时间，用numba编译这样的函数？使用整数或浮点数组合以类似的方式创建数组不会产生任何问题。

Answer 1

您可能想在numba issue tracker上报告此情况，感觉编译器中的某些内容出现错误，因为它的扩展性很差。
您也可以考虑是否真的需要这样的大量数组语句，如果问题可以更明确地重构。例如。而不是布尔数组，valid可以是一个根据需要调用的函数吗？
众所周知，当前版本的numba中的解决方法正在展开这些条件。

对于你的例子：

# "codegen"
for i in range(23):
    print(f'    valid[{i}] = (a - 1 >= 0) and (b - 1 >= 0)')

@jit(nopython=True)
def foo(a, b):
    valid = np.empty(23, dtype=np.bool_)
    valid[0] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[1] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[2] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[3] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[4] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[5] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[6] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[7] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[8] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[9] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[10] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[11] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[12] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[13] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[14] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[15] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[16] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[17] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[18] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[19] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[20] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[21] = (a - 1 >= 0) and (b - 1 >= 0)
    valid[22] = (a - 1 >= 0) and (b - 1 >= 0)

%time foo(1,1)
Wall time: 274 ms

用numba.jit编译一系列条件需要很长时间

1 个答案: