Spark DStream - Is Receiver to Executor relationship 1:1?

时间:2019-05-31 11:34:59

标签: apache-spark spark-streaming

As mentioned in the Performance Tuning section each input DStream creates a single receiver (running on a worker machine) that receives a single stream of data.

Does that mean that you'd only have one receiver per machine and therefore to maximise the parallelism of your job to an optimum level you'd ideally want to split your data on the sending side into as many topics as the number of machines you have in your Spark cluster?

0 个答案:

没有答案