Question

我有一个长度为3（x）的整数元组的numpy数组和一个长度为2（y）的整数元组的numpy数组。

x = numpy.array([[3, 4, 5], [5, 12, 13], [6, 8, 10], [7, 24, 25]]) #first 4 elem
y = numpy.array([[3, 4], [4, 5], [3, 5], [5, 12]]) # first 4 elem

我正在尝试比较数组y中的元素：[a，b] 和 [b，c] 和 [a，c]这些是单个子集数组x中的元素[a，b，c]。我叫这个函数联合。我找到联合的循环方法如下所示。这对于包含200K最小元素的数组来说并不太好。

def union(x, y):
for intx in range (len(x)):
    cond1 = cond2 = cond3 = 0
    for inty in range (len(y)):
        if (y[inty][0] == x[intx][0] and y[inty][1] == x[intx][1]): #[a, b] & [a, b, c]
            print ("condition 1 passed")
            cond1 = 1
        if (y[inty][0] == x[intx][1] and y[inty][1] == x[intx][2]): #[b, c] & [a, b, c]
            print ("condition 2 passed")
            cond2 = 1
        if (y[inty][0] == x[intx][0] and y[inty][1] == x[intx][2]): #[a, c] & [a, b, c]
            print ("condition 3 passed")
            cond3 = 1
        if (cond1 & cond2 & cond3):
            print("union found with ", x[intx])
            cond1 = cond2 = cond3 = 0
return

>>> union(x,y)
condition 1 passed
condition 2 passed
condition 3 passed
union found with  [3 4 5]
condition 1 passed

更新＃1：示例1：这组x和y没有联合：

x = numpy.array([[21, 220, 221]])
y = numpy.array([[21, 220], [20, 21], [220,3021], [1220,3621], [60,221]])

更新＃2：示例2：这组x和y没有联合：

x = numpy.array([[43, 924, 925]])
y = numpy.array([[43, 924], [924, 1643], [924,4307], [72, 925]])

示例3：这是一组x和y，其联合为[4,8,16]。

x = numpy.array([[4, 8, 16], [8, 4, 16]])
y = numpy.array([[4, 8], [8, 16], [4, 16]])

示例4：这是一组x和y，其联合为[12,14,15]。

x = numpy.array([[12, 13, 15], [12, 14, 15]])
y = numpy.array([[12, 14], [12, 13], [12, 15], [14, 15]])

总结：一般来说，如果

，数组x和y将具有[a，b，c]的并集

x = numpy.array([[a, b, c], ...])
y = numpy.array([[a, b], [b, c], [a, c],...])

或y中的随机排序

y = numpy.array([[...[b, c], [a, c], ... [a, b]])

所以我的问题：是否有一种简单的方法来进行数组操作？例如，numpy.logical_并建议x1和x2必须是相同的形状。用isdisjoint替换我的if语句并不简单，这是一种更快的方法。 https://stackoverflow.com/a/24478521/8275288

Answer 1

如果您只对符合条件的x“行感兴趣，可以使用：

import numpy as np

def union(x, y):
    # Create a boolean mask for the columns of "x"
    res = np.ones(x.shape[0], dtype=bool)
    # Mask containing the "x" rows that have one "partial match"
    res_tmp = np.zeros(x.shape[0], dtype=bool)
    # Walk through the axis-combinations
    # you could also use Divakars "(x[:,:2], x[:,::2], x[:,1:])" here.
    for cols in (x[:, [0, 1]], x[:, [1, 2]], x[:, [0, 2]]):
        # Check each row of y if it has a partial match
        for y_row in y:
            res_tmp |= (y_row == cols).all(axis=1)
        # Update the overall mask and then reset the partial match mask
        res &= res_tmp
        res_tmp[:] = 0
    return res

x = np.array([[3, 4, 5], [5, 12, 13], [6, 8, 10], [7, 24, 25]])
y = np.array([[3, 4], [4, 5], [3, 5], [5, 12]])
mask = union(x, y)
print(mask)     # array([ True, False, False, False], dtype=bool)
print(x[mask])  # array([[3, 4, 5]])

或者使用其他y：

y = np.array([[3, 4], [4, 5], [3, 5], [5, 12], [12, 13], [5, 13]])
mask = union(x, y)
print(x[mask])
# array([[ 3,  4,  5],
#        [ 5, 12, 13]])

它仍然需要循环两次，但内部操作y_row == x[:, ax]是矢量化的。这应该至少带来一些（可能是巨大的）速度提升。

还可以对for y_row in y循环进行矢量化（使用广播），但是如果您的x数组和y非常大，那么这将无法提高性能但会使用{ {1}}内存（在某些情况下，这可能需要比实际内存更多的内存 - 导致异常或性能非常差，因为您回退到交换内存）。

Answer 2

numpy_indexed包（免责声明：我是它的作者）可用于创建原始代码的相当简单的矢量化版本，这应该更有效：

from functools import reduce
import numpy_indexed as npi

def contains_union(x, y):
    """Returns an ndarray with a bool for each element in x, 
    indicating if it can be constructed as a union of elements in y"""
    idx = [[0, 1], [1, 2], [0, 2]]
    y = npi.as_index(y)   # not required, but a performance optimization
    return reduce(np.logical_and, (npi.in_(x[:, i], y) for i in idx))

Answer 3

如果你的x值最大小于最大整数表示的sqrt（使用int 64？），那么数字技巧可能有效

我使用int（1e6）作为一个可读的例子

import numpy

#rolled up all of the examples
x = numpy.array([[3, 4, 5], [5, 12, 13], [6, 8, 10], [7, 24, 25],
                [21, 220, 221],
                [43, 924, 925],
                [4, 8, 16], [8, 4, 16],
                [12, 13, 15], [12, 14, 15]]) #all examples

#and a numpy array of integer tuples of length 2:

y = numpy.array([[3, 4], [4, 5], [3, 5], [5, 12],
                [21, 220], [20, 21], [220,3021], [1220,3621], [60,221],
                [43, 924], [924, 1643], [924,4307], [72, 925],
                [4, 8], [8, 16], [4, 16],
                [12, 14], [12, 13], [12, 15], [14, 15]]) #all examples

#then make a couple of transform arrays

zx=numpy.array([[int(1e6), 1, 0],
                [0, int(1e6), 1],
                [int(1e6), 0, 1],
                ])
zy = numpy.array([[int(1e6)], [1]])


# and the magic is: np.intersect1d(zx @ x[ix], y @ zy)

# just to see part of what is being fed to intersect1d
print(zx @ x[0])
 [3000004 4000005 3000005]  

print(y[:4] @ zy)
[[3000004]
 [4000005]
 [3000005]
 [5000012]]


# if len of the intersection == 3 then you have your match

y_zy = y @ zy # only calc once
for ix in range(len(x)):
    matches = len(np.intersect1d(zx @ x[ix], y_zy))
    print(ix, matches, x[ix] if matches == 3 else '')
0 3 [3 4 5]
1 2 
2 0 
3 0 
4 1 
5 1 
6 3 [ 4  8 16]
7 2 
8 2 
9 3 [12 14 15]

我不知道intersect1d速度，但根据文档，如果可以设置unique=True标记，则可以改进，取决于您的数据

如何使用numpy在2D和3D数组之间找到联合？

3 个答案: