Python - 多处理 - 巨大的循环

时间:2017-05-14 15:15:34

标签: python list loops multiprocessing

下午好,

我对Python很陌生,我必须解决一个需要尝试数十亿个假设的问题...更具体地说,我需要迭代440个元素的列表,但我需要做8次。 ..(是的,我知道,迭代次数完全是疯了)。

我的机器非常好,所以我想使用多处理python功能来加速这一点。

您是否知道任何可以从我的机器处理功能中获利的简单解决方案?

输入:

ListPairs:

for ind1 in range(16,37):
    for ind2 in range(16,37):

        ListPairsAux = []

        ListPairsAux.append(ind1)
        ListPairsAux.append(ind2)

        ListPairs.append(ListPairsAux)

为了简化问题,你可以假设len(list1 [i])和len(list2 [i])都是整数,并且都等于198.(在实际问题中我们实际上会有21个不同的整数,但都是以相同的顺序 - 意味着它们不会超过198。

for循环如下:

for first in ListPairs:
    print(str(first))
    for second in ListPairs:
        for third in ListPairs:
            for fourth in ListPairs:
                for fifth in ListPairs:
                    for sixth in ListPairs:
                        for seventh in ListPairs:
                            sumA = first[0] + second[0] + third[0] + fourth[0] + fifth[0] + sixth[0] + seventh[0]
                            sumB = first[1] + second[1] + third[1] + fourth[1] + fifth[1] + sixth[1] + seventh[1]
                            for i in range(len(list1)):
                                if sumA == len(list1[i]) and sumB == len(list2[i]):
                                    List7 = []
                                    List7 = [first, second, third, fourth, fifth, sixth, seventh]
                                    ListsOut[i].append(List7)

                            for eighth in ListPairs:
                                sumA = first[0] + second[0] + third[0] + fourth[0] + fifth[0] + sixth[0] + seventh[0] + eighth[0]
                                sumB = first[1] + second[1] + third[1] + fourth[1] + fifth[1] + sixth[1] + seventh[1] + eighth[1]
                                for i in range(len(list1)):
                                    if sumA == len(list1[i]) and sumB == len(list2[i]):
                                        List8 = []
                                        List8 = [first, second, third, fourth, fifth, sixth, seventh, eighth]
                                        ListsOut[i].append(List8)

非常感谢你!

2 个答案:

答案 0 :(得分:1)

您发布的解决方案可能永远不会完成,因为它需要经过超过10 ^ 21个元素组合。您应该使用更快的算法,而不是使用多处理。

使用list1,list2和lists_out 你在问题中使用,我们正在寻找结合两者之间的整数的方法 在图16和36中,它们总和为list1和list2中序列的长度。 组合应该是[16,36]范围内的7或8个整数。

import itertools
def so43965562(list1, list2, lists_out, lower=16, upper=36):
    assert len(list1) == len(list2) == len(lists_out)
    for n in (7, 8):
        for i in range(len(list1)):
            # Find all combinations of n numbers in [lower, upper]
            # that sum to len(list1[i])
            combs1 = combinations_summing_to(lower, upper, n, len(list1[i]))
            # Find all combinations of n numbers in [lower, upper]
            # that sum to len(list2[i])
            combs2 = combinations_summing_to(lower, upper, n, len(list2[i]))
            for t1, t2 in itertools.product(combs1, combs2):
                result = [(v1, v2) for v1, v2 in zip(t1, t2)]
                lists_out[i].append(result)

以下函数将s写为nl之间u个整数的总和。

def combinations_summing_to(l, u, n, s, suffix=()):
    """In which ways can s be written as the sum of n integers in [l, u]?

    >>> # Write 2 as a sum of 4 integers between 0 and 5.
    >>> print(list(combinations_summing_to(0, 5, 4, 2)))
    [(0, 0, 0, 2), (0, 0, 1, 1)]
    >>> # Write 5 as a sum of 3 integers between 0 and 5.
    >>> print(list(combinations_summing_to(0, 5, 3, 5)))
    [(0, 0, 5), (0, 1, 4), (0, 2, 3), (1, 1, 3), (1, 2, 2)]
    >>> # Write 12 as a sum of 3 integers between 0 and 5.
    >>> print(list(combinations_summing_to(0, 5, 3, 12)))
    [(2, 5, 5), (3, 4, 5), (4, 4, 4)]
    >>> # Write 34 as a sum of 2 integers between 16 and 36.
    >>> print(list(combinations_summing_to(16, 36, 2, 34)))
    [(16, 18), (17, 17)]
    """
    if n == 0:
        return (suffix,) if s == 0 else ()
    elif n == 1:
        return ((s,) + suffix,) if l <= s <= u else ()
    else:
        return itertools.chain.from_iterable(
            # Combinations summing to s where the last element is k
            combinations_summing_to(l, k, n - 1, s - k, (k,) + suffix)
            for k in range(u, l-1, -1)
            # Early bailout if you can't make s with all elements <= k
            if l * n <= s <= k * n)

您可以按如下方式运行解决方案:

lists_out = [[]]
so43965562(list1=[[0]*(7*16+1)], list2=[[0]*(7*16+2)], lists_out=lists_out)
for result in lists_out[0]:
    print(result)
# Outputs the following two combinations:
# [(16, 16), (16, 16), (16, 16), (16, 16), (16, 16), (16, 16), (17, 18)]
# [(16, 16), (16, 16), (16, 16), (16, 16), (16, 16), (16, 17), (17, 17)]
lists_out = [[]]
n = 133
so43965562(list1=[[0]*n], list2=[[0]*n], lists_out=lists_out)
print(len(lists_out[0]))
# Outputs 1795769, takes about 2.5 seconds to run.

请注意,输出大小呈指数级增长,从...开始 当n = 7 * 16 = 112时没有任何东西,所以它仍然需要很长时间才能计算出来 当你在问题中写下n = 198时的所有组合。

答案 1 :(得分:0)

如果你想要组合(或排列), 检查python itertools

https://docs.python.org/2/library/itertools.html#itertools.combinations

否则请修改算法。

 for combination in itertools.combinations_with_replacement(ListPairs, 8):
     # combination is a tuple
     for i in combination:
           #check your condition