需要使用parallelCamTasks(以args形式传递)通过cameraWithCube.getCam()1进行分区。
然后在每个分区内,需要通过cameraWithCube.getTs()再次进行分区,但需要确保每个 getTS()的第二个分区在同一物理节点上运行?
我该如何实现?
DataStream<CameraWithCube> cameraWithCubeDataStream = env
.addSource(new Source(....))
.keyBy((cameraWithCube) -> cameraWithCube.getCam() )
.process(new ProcessFunction<CameraWithCube, CameraWithCube>() {
public void processElement(CameraWithCube cameraWithCube, Context context, Collector<CameraWithCube> collector) throws Exception {
//do nothing
}
})
.slotSharingGroup("camSharingGroup")//TODO: how to add camera# of the partition
.setParallelism(parallelCamTasks)
.keyBy((cameraWithCube) -> cameraWithCube.getTs())
.process(new ProcessFunction<CameraWithCube, CameraWithCube>() {
public void processElement(CameraWithCube cameraWithCube, Context context, Collector<CameraWithCube> collector) throws Exception {
//TODO: process code
}
})
.setParallelism(noOfSlotsInEachPhysicalNode)//TODO: how many parallel tasks within physical node
.slotSharingGroup("??");//TODO: in same physical node
TIA