我正在使用here中的函数get_tuples(length, total)
要生成给定长度和总和的所有元组的数组,下面显示一个示例和函数。创建数组后,我需要找到一种方法来返回数组中给定数量的元素的索引。我可以通过将数组更改为列表来使用.index()
进行操作,如下所示。但是,此解决方案或另一种也基于搜索(例如,使用np.where
)的解决方案需要大量时间才能找到索引。由于数组中的所有元素(示例中的数组s
)都不同,所以我想知道是否可以构造一对一映射,即一个函数,使得给定数组中的元素,它返回通过对该元素的值进行一些加法和乘法来对该元素进行索引。有什么想法吗?谢谢!
import numpy as np
def get_tuples(length, total):
if length == 1:
yield (total,)
return
for i in range(total + 1):
for t in get_tuples(length - 1, total - i):
yield (i,) + t
#example
s = np.array(list(get_tuples(4, 20)))
# array s
In [1]: s
Out[1]:
array([[ 0, 0, 0, 20],
[ 0, 0, 1, 19],
[ 0, 0, 2, 18],
...,
[19, 0, 1, 0],
[19, 1, 0, 0],
[20, 0, 0, 0]])
#example of element to find the index for. (Note in reality this is 1000+ elements)
elements_to_find =np.array([[ 0, 0, 0, 20],
[ 0, 0, 7, 13],
[ 0, 5, 5, 10],
[ 0, 0, 5, 15],
[ 0, 2, 4, 14]])
#change array to list
s_list = s.tolist()
#find the indices
indx=[s_list.index(i) for i in elements_to_find.tolist()]
#output
In [2]: indx
Out[2]: [0, 7, 100, 5, 45]
答案 0 :(得分:2)
这里是一个仅根据元组计算索引的公式,即它不需要看到完整的数组。要计算N元组的索引,需要评估N-1个二项式系数。以下实现是(部分)矢量化的,它接受ND数组,但元组必须在最后一维。
import numpy as np
from scipy.special import comb
# unfortunately, comb with option exact=True is not vectorized
def bc(N,k):
return np.round(comb(N,k)).astype(int)
def get_idx(s):
N = s.shape[-1] - 1
R = np.arange(1,N)
ps = s[...,::-1].cumsum(-1)
B = bc(ps[...,1:-1]+R,1+R)
return bc(ps[...,-1]+N,N) - ps[...,0] - 1 - B.sum(-1)
# OP's generator
def get_tuples(length, total):
if length == 1:
yield (total,)
return
for i in range(total + 1):
for t in get_tuples(length - 1, total - i):
yield (i,) + t
#example
s = np.array(list(get_tuples(4, 20)))
# compute each index
r = get_idx(s)
# expected: 0,1,2,3,...
assert (r == np.arange(len(r))).all()
print("all ok")
#example of element to find the index for. (Note in reality this is 1000+ elements)
elements_to_find =np.array([[ 0, 0, 0, 20],
[ 0, 0, 7, 13],
[ 0, 5, 5, 10],
[ 0, 0, 5, 15],
[ 0, 2, 4, 14]])
print(get_idx(elements_to_find))
样品运行:
all ok
[ 0 7 100 5 45]
如何得出公式:
使用stars and bars将完整的分区数#part(N,k)
(N为总数,k为长度)表示为单个二项式系数(N + k - 1) choose (k - 1)
。
从头开始计数:不难验证OP生成器的外循环的第i次完整迭代之后,#part(N-i,k)
尚未被枚举。实际上,剩下的就是所有分区p1 + p2 + ... = N,其中p1> = i;我们可以这样写p1 = q1 + i,使得q1 + p2 + ... = N-i,后面的分区是无约束的,因此我们可以使用1.来计数。
答案 1 :(得分:1)
您可以使用二进制搜索使搜索更快。
二元搜索使搜索成为O(log(n))而不是O(n)(使用索引)
我们不需要对元组进行排序,因为生成器已经对它们进行了排序
import bisect
def get_tuples(length, total):
" Generates tuples "
if length == 1:
yield (total,)
return
yield from ((i,) + t for i in range(total + 1) for t in get_tuples(length - 1, total - i))
def find_indexes(x, indexes):
if len(indexes) > 100:
# Faster to generate all indexes when we have a large
# number to check
d = dict(zip(x, range(len(x))))
return [d[tuple(i)] for i in indexes]
else:
return [bisect.bisect_left(x, tuple(i)) for i in indexes]
# Generate tuples (in this case 4, 20)
x = list(get_tuples(4, 20))
# Tuples are generated in sorted order [(0,0,0,20), ...(20,0,0,0)]
# which allows binary search to be used
indexes = [[ 0, 0, 0, 20],
[ 0, 0, 7, 13],
[ 0, 5, 5, 10],
[ 0, 0, 5, 15],
[ 0, 2, 4, 14]]
y = find_indexes(x, indexes)
print('Found indexes:', *y)
print('Indexes & Tuples:')
for i in y:
print(i, x[i])
输出
Found indexes: 0 7 100 5 45
Indexes & Tuples:
0 (0, 0, 0, 20)
7 (0, 0, 7, 13)
100 (0, 5, 5, 10)
5 (0, 0, 5, 15)
45 (0, 2, 4, 14)
性能
方案1 -已经计算了重复数,我们只想找到某些元组的索引
例如x = list(get_tuples(4,20))已执行。
搜索
indexes = [[ 0, 0, 0, 20],
[ 0, 0, 7, 13],
[ 0, 5, 5, 10],
[ 0, 0, 5, 15],
[ 0, 2, 4, 14]]
二进制搜索
%timeit find_indexes(x, indexes)
100000 loops, best of 3: 11.2 µs per loop
仅基于元组计算索引(由@PaulPanzer方法提供)
%timeit get_idx(indexes)
10000 loops, best of 3: 92.7 µs per loop
在这种情况下,如果已经预先计算了元组,二进制搜索的速度将提高约8倍。
方案2 -元组尚未预先计算。
%%timeit
import bisect
def find_indexes(x, t):
" finds the index of each tuple in list t (assumes x is sorted) "
return [bisect.bisect_left(x, tuple(i)) for i in t]
# Generate tuples (in this case 4, 20)
x = list(get_tuples(4, 20))
indexes = [[ 0, 0, 0, 20],
[ 0, 0, 7, 13],
[ 0, 5, 5, 10],
[ 0, 0, 5, 15],
[ 0, 2, 4, 14]]
y = find_indexes(x, indexes)
100 loops, best of 3: 2.69 ms per loop
@PaulPanzer方法在这种情况下是相同的时机(92.97美元)
=>当不必计算元组时,@ PaulPanzer的速度快约29倍
方案3 -大量索引(@PJORR) 产生大量随机索引
x = list(get_tuples(4, 20))
xnp = np.array(x)
indices = xnp[np.random.randint(0,len(xnp), 2000)]
indexes = indices.tolist()
%timeit find_indexes(x, indexes)
#Result: 1000 loops, best of 3: 1.1 ms per loop
%timeit get_idx(indices)
#Result: 1000 loops, best of 3: 716 µs per loop
在这种情况下,我们@PaulPanzer的速度提高了53%