Question

我需要一些帮助来理解为什么合并两个流会阻塞类FixedBatchSpout的一个spout。

简短说明：我正在尝试合并两个流s1和s2，但是调用topology.merge(s1, s2)会阻止s1发起的FixedBatchSpout（三叉笛），而来自s2的BaseRichSpout（风暴鲸鱼喷水）似乎运作正常。

详细信息：在以下主要方法中，只需添加第topology.merge(s1, s2);行即可阻止FixedBatchSpout发出第一批。这也发生在multireduce上。

FixedBatchSpout spout1 = new FixedBatchSpout(new Fields("sentence"), 2,
                new Values("the cow jumped over the moon"),
                new Values("the man went to the store and bought some candy”));

FixedLoopSpout spout2 = new FixedLoopSpout(new Fields("sentence"),
                new Values("THE COW JUMPED OVER THE MOON"),
                new Values("THE MAN WENT TO THE STORE AND BOUGHT SOME CANDY"));

Stream s1 = topology.newStream("hello", spout1);
Stream s2 = topology.newStream("world", spout2);
topology.merge(s1, s2);

public class FixedLoopSpout extends BaseRichSpout {

    Values[] values;
    List<Values> loop = new LinkedList<Values>();
    Iterator<Values> head;
    private SpoutOutputCollector collector;
    private final Fields outputFields;

    private long emitted = 0;

    public FixedLoopSpout(Fields outputFields, Values... values) {
        this.outputFields = outputFields;
        this.values = values;
    }

    public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
        this.collector = collector;
        for (Values value: this.values) {
            this.loop.add(value);
        }
        this.head = this.loop.iterator();
    }

    public void nextTuple() {
        if (!this.head.hasNext()) {
            // wrap
            this.head = this.loop.iterator();
        }
        this.collector.emit(this.head.next(), this.emitted++);
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    public void declareOutputFields(OutputFieldsDeclarer declarer) {
        declarer.declare(this.outputFields);
    }
}

感谢帮助，谢谢！雅克

合并三叉戟流阻塞三叉戟喷口，而风暴喷口继续工作

0 个答案: