测试咖啡馆：即使在必须通过抛出未处理的承诺拒绝而失败的情况下，第一次测试中未处理的承诺拒绝也会使第二次测试通过

Question

我正在使用通过Test Cafe中的Resemble.JS库使用的函数来比较实际屏幕截图和基本屏幕截图。在我的装置中，我有两个测试，由于屏幕截图不匹配，两个测试都应该在报告中失败，但是只有第一个测试显示为失败，而第二个测试在报告中通过了

请帮助我如何处理这种情况并将两个测试都标记为失败。

用于比较屏幕截图的功能：

const peformVisualRegression = async (testFixture, testName) => {
  // take actual screenshot
  await t.takeScreenshot(path.join('actual', testFixture, `${testName}.png`));

  const actualScreenshotAbsolutePath = getAbsolutePathForScreenshot(
    'actual',
    testFixture,
    testName
  );
  const isActualScreenshotTaken = fs.existsSync(actualScreenshotAbsolutePath);

  const baseScreenshotAbsolutePath = getAbsolutePathForScreenshot(
    'base',
    testFixture,
    testName
  );
  const isBaseScreenshotTaken = fs.existsSync(baseScreenshotAbsolutePath);

  if (isActualScreenshotTaken && isBaseScreenshotTaken) {
    await resemble(baseScreenshotAbsolutePath)
      .compareTo(actualScreenshotAbsolutePath)
      .scaleToSameSize()
      .outputSettings({
        errorColor: {
          blue: 255,
          green: 0,
          red: 255
        },
        errorType: 'movement',
        largeImageThreshold: 1200,
        outputDiff: true,
        transparency: 0.3,
        useCrossOrigin: false
      })
      .onComplete(async data => {
        if (data.rawMisMatchPercentage > 0) {
          logger.error(
            `Mismatch percentage for ${testFixture}/${testName} between actual and base screenshot is ${
              data.rawMisMatchPercentage
            }`
          );
          // write a diff image
          fs.writeFileSync(
            path.join(
              path.dirname(actualScreenshotAbsolutePath),
              `${path.basename(
                actualScreenshotAbsolutePath,
                path.extname(actualScreenshotAbsolutePath)
              )}-diff.png`
            ),
            data.getBuffer()
          );

          // fail test
          throw new Error(
            `Visual mismatch detected in test: ${testFixture}/${testName}. Please investigate.`
          );
        }
      });
  }

固定装置：

fixture('Test Duckduckgo search fixture 1').page('https://duckduckgo.com');

test('Testcafe case 1 | @TestrailID:1094986', async t => {
  await t.expect(samplePage.searchInputField.exists).ok();
  await samplePage.enterText('dog');
  const location = await getWindowLocation();
  await t.expect(location.href).contains('q=dog');
  await peformVisualRegression(
    'Test Duckduckgo search fixture 1',
    'Testcafe case 1'
  );
});

test('Testcafe case 2 | @TestrailID:1094987', async t => {
  await t.expect(samplePage.searchInputField.exists).ok();
  await peformVisualRegression(
    'Test Duckduckgo search fixture 1',
    'Testcafe case 2'
  );
});

Answer 1

我能够重现该问题，因此我在TestCafe存储库中创建了一个单独的bug report，以详细研究该问题。