使用maven在本地运行拓扑时,Apache Storm版本2.0问题

时间:2019-07-03 13:51:09

标签: maven pom.xml apache-storm apache-storm-topology

我正在尝试运行新版本的Storm(v2.0),但由于某些原因,在本地启动它时,控制台中未显示我的所有输出注释,导致我得出以下结论:未在以下位置执行Storm拓扑所有。 Storm的输出非常巨大,我看不出问题出在哪里。

这是我的pom.xml文件的内容:

<groupId>com.example.exclamationTopologyEx</groupId>
<artifactId>ExclamationTopologtEx</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-client</artifactId>
        <version>2.0.0</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-server</artifactId>
        <version>2.0.0</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.5.0</version>
            <executions>
                <execution>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <executable>java</executable>
                <includeProjectDependencies>true</includeProjectDependencies>
                <includePluginDependencies>false</includePluginDependencies>
                <classpathScope>compile</classpathScope>
                <mainClass>com.stormExample.SquareStormTopology</mainClass>
                <cleanupDaemonThreads>false</cleanupDaemonThreads>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.6.1</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>
</build>

这是启动拓扑的主要类别:

public static void main(String[] args) throws Exception {
    //used to build the toplogy
    TopologyBuilder builder = new TopologyBuilder();
    //add the spout with name 'spout' and parallelism hint of 5 executors
    builder.setSpout("spout", new DataSpout(),5);
    //add the Emitter bolt with the name 'emitter'
    builder.setBolt("emitter",new EmitterBolt(),8).shuffleGrouping("spout");

    Config conf = new Config();
    //set to false to disable debug when running on production cluster
    conf.setDebug(false);
    //if there are arguments then we are running on a cluster
    if(args!= null && args.length > 0){
        //parallelism hint to set the number of workers
        conf.setNumWorkers(3);
        //submit the toplogy
        StormSubmitter.submitTopology(args[0], conf, builder.createTopology());
    }else{//we are running locally
        //the maximum number of executors
        conf.setMaxTaskParallelism(3);

        //local cluster used to run locally
        LocalCluster cluster = new LocalCluster();
        //submitting the topology
        cluster.submitTopology("emitter-topology",conf, builder.createTopology());
        //sleep
        Thread.sleep(10000);
        //shut down the cluster
        cluster.shutdown();
    }
}

另外,这些是我的喷嘴和螺栓实现:

public class DataSpout extends BaseRichSpout {

    SpoutOutputCollector _collector;
    int nextNumber;

    @Override
    public void open(Map map, TopologyContext topologyContext, SpoutOutputCollector spoutOutputCollector) {
        _collector = spoutOutputCollector;
        nextNumber = 2;
    }

    @Override
    public void nextTuple() {
        if(nextNumber > 20){
            return;
        }else{
            _collector.emit(new Values(nextNumber));

            nextNumber = nextNumber + 2;
        }
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
        outputFieldsDeclarer.declare(new Fields("number"));
    }
}
public class EmitterBolt extends BaseBasicBolt {

    int sumNumbers;

    public EmitterBolt(){

    }

    @Override
    public void prepare(Map stormConf, TopologyContext context) {
        sumNumbers = 0;
        System.out.println("com.example.exclamationTopologyEx.EmitterBolt has been initialized");
    }

    @Override
    public void execute(Tuple tuple, BasicOutputCollector basicOutputCollector) {
        int nextNumber = tuple.getIntegerByField("number");

        sumNumbers+=nextNumber;

        System.out.println("The new sum is: " + sumNumbers);
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {

    }
}

我使用了maven命令:

mvn compile exec:java -Dstorm.topology=com.example.exclamationTopologyEx 

用于运行拓扑,但是正如我所说,控制台中未显示任何打印行。我试图从网站和git的文档中寻找解决方案,但是找不到任何可以使这种简单拓扑正常工作的解决方案。

我认为问题可能出在 pom.xml 文件中,我可能需要一些其他依赖项,或者我有一些错误的依赖项,但是正如我所说,我无法弄清楚什么是正确的pom.xml的内容。

我正在Mac OS上工作,并使用最新版本的Java(12.0.1),作为IDE,我正在使用IntelliJ IDEA。

0 个答案:

没有答案