使用numpy进行元素“in”的Pythonic和有效方法

时间:2015-07-24 19:27:26

标签: python arrays numpy boolean

我正在寻找一种有效获取布尔数组的方法,其中给定两个大小相等ab的数组,如果a的相应元素,则每个元素都为true } in b的相应元素显示a = numpy.array([1, 2, 3, 4]) b = numpy.array([[1, 2, 13], [2, 8, 9], [5, 6], [7]]) print(numpy.magic_function(a, b))

例如,以下程序:

[True, True, False, False]

应打印

[x in y for x, y in zip(a, b)]

请记住,此功能应相当于

numpy

a - 针对bb较大且Invalid column name 'Film_Id'. 的每个元素相当小的情况进行了优化。

3 个答案:

答案 0 :(得分:4)

要利用NumPy的broadcasting规则,您应首先将数组b平方,这可以使用itertools.izip_longest来实现:

from itertools import izip_longest

c = np.array(list(izip_longest(*b))).astype(float)

导致:

array([[  1.,   2.,   5.,   7.],
       [  2.,   8.,   6.,  nan],
       [ 13.,   9.,  nan,  nan]])

然后,通过np.isclose(c, a),您可以根据广播规则获得一个二维的布尔数组,显示每个c[:, i]a[i]之间的差异,并给出:

array([[ True,  True, False, False],
       [False, False, False, False],
       [False, False, False, False]], dtype=bool)

可以用来获得答案:

np.any(np.isclose(c, a), axis=0)
#array([ True,  True, False, False], dtype=bool)

答案 1 :(得分:3)

b中的小列表的长度是否有上限?如果是这样,也许你可以使b表示1000x5的矩阵,并使用nan来填补太短的子阵列的空白。然后,您可以使用numpy.any来获得所需的答案,如下所示:

In [42]: a = np.array([1, 2, 3, 4])
    ...: b = np.array([[1, 2, 13], [2, 8, 9], [5, 6], [7]])

In [43]: bb = np.full((len(b), max(len(i) for i in b)), np.nan)

In [44]: for irow, row in enumerate(b):
    ...:     bb[irow, :len(row)] = row

In [45]: bb
Out[45]: 
array([[  1.,   2.,  13.],
       [  2.,   8.,   9.],
       [  5.,   6.,  nan],
       [  7.,  nan,  nan]])

In [46]: a[:,np.newaxis] == bb
Out[46]: 
array([[ True, False, False],
       [ True, False, False],
       [False, False, False],
       [False, False, False]], dtype=bool)

In [47]: np.any(a[:,np.newaxis] == bb, axis=1)
Out[47]: array([ True,  True, False, False], dtype=bool)

不知道这对您的数据来说是否更快。

答案 2 :(得分:1)

摘要

Sauldo Castro的方法在迄今为止发布的内容中运行得最快。原始帖子中的生成器表达式是第二快的。

生成测试数据的代码:

import numpy
import random

alength = 100
a = numpy.array([random.randint(1, 6) for i in range(alength)])
b = []
for i in range(alength):
    length = random.randint(1, 5)
    element = []
    for i in range(length):
        element.append(random.randint(1, 6))
    b.append(element)
b = numpy.array(b)
print a, b

选项:

from itertools import izip_longest
def magic_function1(a, b): # From OP Martin Fixman
    return [x in y for x, y in zip(a, b)]  

def magic_function2(a, b): # What I thought might be better.
    bools = []
    for x, y in zip(a,b):
        found = False
        for j in y:
            if x == j:
                found=True
                break
        bools.append(found)

def magic_function3(a, b): # What I tried first
    bools = []
    for i in range(len(a)):
        found = False
        for j in range(len(b[i])):
            if a[i] == b[i][j]:
                found=True
                break
        bools.append(found)

def magic_function4(a, b): # From Bas Swinkels
    bb = numpy.full((len(b), max(len(i) for i in b)), numpy.nan)
    for irow, row in enumerate(b):
        bb[irow, :len(row)] = row
    a[:,numpy.newaxis] == bb
    return numpy.any(a[:,numpy.newaxis] == bb, axis=1)

def magic_function5(a, b): # From Sauldo Castro, revised version
    c = numpy.array(list(izip_longest(*b))).astype(float)
    return numpy.isclose(c, a), axis=0)  

时间n_executions

n_executions = 100
clock = timeit.Timer(stmt="magic_function1(a, b)", setup="from __main__ import magic_function1, a, b")
print clock.timeit(n_executions), "seconds"
# Repeat with each candidate function

结果:

  • magic_function1
  • 的0.158078225475秒
  • 对于magic_function2
  • ,为0.181080926835秒
  • 0.259621047822秒为magic_function3
  • 0.287054750224秒for magic_function4
  • magic_function5
  • 的0.0839162196207秒