我进行单元测试(使用TestStream
和PAssert
)DoFn
重置事件计时器。如果DoFn
重置计时器,则测试将永久挂起,并且此行为似乎特定于事件域计时器。
这是光束测试设施中的错误还是预期的计时器行为?
这是一个玩具示例,我可以使用beam 2.3 SDK重现此行为。
static class KeyElements extends DoFn<String, KV<String, String>> {
@ProcessElement
public void processElement(ProcessContext context) {
final String[] parts = context.element().split(":");
if (parts.length == 2) {
context.output(KV.of(parts[0], parts[1]));
}
}
}
static class TimerDoFn extends DoFn<KV<String, String>, KV<String, String>> {
@TimerId("expiry")
private final TimerSpec timerSpec = TimerSpecs.timer(TimeDomain.EVENT_TIME);
@ProcessElement
public void processElement(ProcessContext context, @TimerId("expiry") Timer timer) {
timer.set(context.timestamp().plus(Duration.standardHours(1)));
final KV<String, String> e = context.element();
context.output(KV.of(e.getKey(), e.getValue() + "_output"));
}
@OnTimer("expiry")
public void onExpiry(OnTimerContext context) {
// do nothing
}
}
@Rule
public TestPipeline p = TestPipeline.create();
@Test
public void testTimerDoFn() {
TestStream<String> stream = TestStream
.create(StringUtf8Coder.of())
.addElements(
TimestampedValue.of("a:0", new Instant(0)),
TimestampedValue.of("a:1", new Instant(1)),
TimestampedValue.of("a:2", new Instant(2)),
TimestampedValue.of("a:3", new Instant(3)))
.advanceWatermarkToInfinity();
PCollection<KV<String, String>> result = p
.apply(stream)
.apply(ParDo.of(new KeyElements()))
.apply(ParDo.of(new TimerDoFn()));
PAssert.that(result).containsInAnyOrder(
KV.of("a", "0_output"),
KV.of("a", "1_output"),
KV.of("a", "2_output"),
KV.of("a", "3_output"));
p.run();
}
如果输入元素为a:1, b:2, c:3, d:4
,则上述测试将停止。