Flink是否在合并流中支持原始通用类型?

时间:2019-07-17 09:48:33

标签: java apache-flink flink-streaming

我可以合并/加入2个包含通用类的流,其中每个流具有不同的参数化类型吗?我想将值的组合流作为单个原始类型进行处理,并根据类型中的一些信息将其转换为特定类型以进行详细处理。

我无法分享确切的代码,但以下示例显示了我想做的事情。

因此,例如,给定以下通用类型:

class MyGenericContainer<IN> extends Tuple3<String, IN, SomeOtherClass> {
    ...
    private final String myString;
    private final IN value;
    private final Class<IN> clazz; // created by constructor
    private SomeOtherClass someOtherClass;
    ...
}

和2个流,我希望能够做类似的事情:

DataStream<MyGenericContainer<String>> stream1 = ...
DataStream<MyGenericContainer<Integer>> stream2 = ...

DataStream<...> merged = stream1.union(stream2).process(new MyProcessFunction());

// within an operator, such as a MyProcessFunction:
MyGenericContainer container = raw generic container passed to function;
Object rawValue = container.getValue();
performProcessing((container.getClazz())rawValue); // safely cast rawValue

但是,执行此操作时出现错误:

Caused by: org.apache.flink.api.common.functions.InvalidTypesException: Type of TypeVariable 'IN' in 'class com.example.MyGenericTuple3' could not be determined. This is most likely a type erasure problem. The type extraction currently supports types with generic variables only in cases where all variables in the return type can be deduced from the input type(s). Otherwise the type has to be specified explicitly using type information.
    at org.apache.flink.api.java.typeutils.TypeExtractor.createSubTypesInfo(TypeExtractor.java:1133)
    at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:853)
    at org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:803)
    at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:587)
    at org.apache.flink.streaming.api.datastream.DataStream.process(DataStream.java:633)

如果我尝试向代码中添加returns(),如下所示:

DataStream<...> merged = stream1.union(stream2)
    .process(...)
    .returns(new TypeHint<MyGenericContainer>() {})

然后我得到了另一个异常:

Exception in thread "main" org.apache.flink.util.FlinkRuntimeException: The TypeHint is using a generic variable.This is not supported, generic types must be fully specified for the TypeHint.

是否支持这种情况?或者是否存在另一种将多个流合并为单个流的方式,其中每个流对象将具有特定的通用泛型类型?

0 个答案:

没有答案