Question

我正在尝试成对测试，并希望使用基于Python的成对测试工具。我已经尝试过AllPairs（http://pypi.python.org/pypi/AllPairs/2.0.1）。当我在列中输入10个条目时，它有bug。目前使用Microsoft PICT生成成对组合。

Python中是否有任何工具可以为大型数组生成成对组合？

AllPairs中的错误如果我给这个

parameters = [ [ "Brand X", "Brand Y","Brand A","Brand B","Brand C","Brand D" ]
             , [ "98", "NT", "2000", "XP"]
             , [ "Internal", "Modem","A","B","C","D","E","F","G","H","I","J","K","L","M" ]
             , [ "Salaried", "Hourly", "Part-Time", "Contr.","AA","BB","CC","DD","EE","FF","GG","HH","II" ]
             , [ 6, 10, 15, 30, 60, 70, 80, 90, 100, 110, 120, 130, 140 ]
             ]

输出

Brand X count is 16
Brand Y count is 122
Brand A count is 16
Brand B count is 16
Brand C count is 16
Brand D count is 15

这个

parameters = [ [ "Brand X", "Brand Y","Brand A","Brand B","Brand C","Brand D" ]
             , [ "98", "NT", "2000", "XP"]
             , [ "Internal", "Modem" ]
             , [ "Salaried", "Hourly", "Part-Time", "Contr." ]
             , [ 6, 10, 15, 30, 60 ]
             ]

输出

Brand X count is 5
Brand Y count is 5
Brand A count is 5
Brand B count is 5
Brand C count is 5
Brand D count is 6

我认为，对于更大的阵列工作是不正确的。

Answer 1

这样的事情怎么样？

from itertools import chain, combinations, product

def pairwiseGen(*sequences):
    unseen = set(chain.from_iterable(product(*i) for i in combinations(sequences, 2)))
    for path in product(*sequences):
        common_pairs = set(combinations(path, 2)) & unseen
        if common_pairs:
            yield path
            unseen.difference_update(common_pairs)

用法（使用您在问题中定义的parameters列表）：

>>> pairs = list(pairwiseGen(*parameters))
>>> len(pairs)
846

我认为仍然有一些优化空间（上面的发电机产生的结果比预期的要多一些），但我认为你会同意它比使用笛卡尔积更短：< / p>

>>> all_possible = list(product(*parameters))
>>> len(all_possible)
60480

Answer 2

TL; DR 即使Brand和Windows版本的分布不均匀，AllPairs也会返回有效的成对测试集。

示例

以下是您的示例：

../proj2/proj2.html

<强>有效期：

如果您检查，您会发现PICT和AllPairs都包含所有993个2元组。意味着PICT和AllPairs都返回有效集。

以上示例中的2元组总数为：

Brand : Brand X, Brand Y, Brand A, Brand B, Brand C, Brand D Version : 98, NT, 2000, XP Network : Internal, Modem, A, B, C, D, E, F, G, H, I, J, K, L, M Wage : Salaried, Hourly, Part-Time, Contr., AA, BB, CC, DD, EE, FF, GG, HH, II Time : 6, 10, 15, 30, 60, 70, 80, 90, 100, 110, 120, 130, 140

<强>性能：

典型的测试用例数是O（nm），其中n和m是两个最大的参数。在这种情况下：15 * 13 = 195是绝对最小的测试集数。

AllPairs返回200个5元组

PICT返回206 5元组

因此，PICT和AllPairs都会返回接近测试集的最小数量。

<强>分发：

Raj指出，AllPairs返回的分布不均匀。

品牌：

6*4 + 6*15 + 6*13 + 6*13 + 4*15 + 4*13 + 4*13 + 15*13 + 15*13 + 13*13 = 993

版本：

Brand A : 16 Brand B : 16 Brand C : 16 Brand D : 15 Brand X : 16 Brand Y : 122

PICT返回更均匀分布的测试集

品牌：

98 : 155 2000 : 15 NT : 16 XP : 15

版本：

Brand A : 26 Brand B : 32 Brand C : 40 Brand D : 35 Brand X : 40 Brand Y : 33

<强>解释

品牌和版本的不均匀分布仍然有效的原因是这两个字段的基数最低（6和4）。所以AllPairs似乎正在填补所有网络和工资对所需的5元组，其中有很多98 : 48 2000 : 55 NT : 52 XP : 51和Brand Y

以下是每个字段的基数：

Windows 98

Answer 3

我刚刚创建了一个围绕MS pict实现的包装器，这可能对你有用。显然有依赖性，但可以节省重建轮子。

import os
import tempfile
import subprocess


def all_pairs(*sequence):
    """
    Calculate the all pairs testing sequence from the Microsoft PICT all pairs application
    that is compiled from: https://github.com/Microsoft/pict

    >>> Type = ['Single', 'Span', 'Stripe', 'Mirror', 'RAID-5']
    >>> Size = [10, 100, 500, 1000, 5000, 10000, 40000]
    >>> Format_method = ['Quick', 'Slow']
    >>> File_system = ['FAT', 'FAT32', 'NTFS']
    >>> Cluster_size = [512, 1024, 2048, 4096, 8192, 16384, 32768, 65536]
    >>> Compression = ['On', 'Off']
    >>> combinations = all_pairs(Type, Size, Format_method, File_system, Cluster_size, Compression)
    >>> assert len(combinations) == 56
    """
    exe = '/opt/pict/bin/pict'
    assert os.path.exists(exe), 'Make sure you have install the PICT executable to: {}'.format(exe)
    assert len(sequence) > 0
    for s in sequence:
        assert isinstance(s, list)
        assert len(s) > 0

    # create input file
    lines = list()
    for i, s in enumerate(sequence):
        seq = ', '.join([str(r) for r in range(len(s))])
        new_line = '{}: {}'.format(i, seq)
        lines.append(new_line)

    with tempfile.NamedTemporaryFile(mode='w') as tmp:
        tmp.write(os.linesep.join(lines))
        tmp.flush()
        result = subprocess.check_output([exe, tmp.name])

    options = list()
    for line in result.split(os.linesep)[1:]:  # skip the header
        if len(line) == 0:
            continue
        indices = [int(i) for i in line.split('\t')]
        options.append([s[i] for s, i in zip(sequence, indices)])
    return options

Answer 4

使用allpairspy怎么样？它是AllPairs的一个分支，具有更多功能（可能是错误修复）和文档。调整到您的参数的basic usage sample code返回200个5元组：

from allpairspy import AllPairs

parameters = [
    [ "Brand X", "Brand Y","Brand A","Brand B","Brand C","Brand D" ],
    [ "98", "NT", "2000", "XP"],
    [ "Internal", "Modem","A","B","C","D","E","F","G","H","I","J","K","L","M" ],
    [ "Salaried", "Hourly", "Part-Time", "Contr.","AA","BB","CC","DD","EE","FF","GG","HH","II" ],
    [ 6, 10, 15, 30, 60, 70, 80, 90, 100, 110, 120, 130, 140 ]
]

print("PAIRWISE:")
for i, pairs in enumerate(AllPairs(parameters)):
    print("{:2d}: {}".format(i, pairs))

如果您想测试Python代码或使用基于pytest的测试框架，您可以integrate allpairspy with pytest via its test parametrization feature。

Python中的成对测试组合生成器

4 个答案: