在flink中使用空的自定义源时收到错误

时间:2018-10-26 09:20:06

标签: apache-flink flink-streaming

我要做的是:

1.download flink-1.6.1-bin-scala_2.11.tgz并解压缩

2.open eclipse,使用jdk 1.8创建一个新的Java项目,将flink-1.6.1-bin-scala_2.11中的lib方向的jar添加到该项目的类路径中

3。创建一个空的自定义源:

package flinkSrc;

import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.functions.source.RichSourceFunction;


public class ShopSourceFromMongo extends RichSourceFunction<String>{
    public void cancel() {}

    public void run(SourceContext<String> sourceContext) throws Exception {}

    @Override
    public void open(Configuration parameters) throws Exception {

    }

    @Override
    public void close() throws Exception {}



}

4。编写一个主程序以运行它:

package flinkSrc;

import java.util.Iterator;

import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.DataStreamUtils;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;


public class Main2 {
    public static void main(String[] args) throws Exception {
           StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

           DataStreamSource<String> shops = env.addSource(new ShopSourceFromMongo());

           Iterator<String> i = DataStreamUtils.collect(shops);
           while(i.hasNext()){
               System.out.println(i.next());
           }

            env.execute();
    }
}

运行main方法时,出现错误:

log4j:WARN No appenders could be found for logger (org.apache.flink.streaming.api.graph.StreamGraphGenerator).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: java.io.IOException: Cannot connect to the client to send back the stream
    at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:623)
    at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
    at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1511)
    at flinkSrc.Main2.main(Main2.java:23)
Caused by: java.io.IOException: Cannot connect to the client to send back the stream
    at org.apache.flink.streaming.experimental.CollectSink.open(CollectSink.java:85)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
    at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.DualStackPlainSocketImpl.connect0(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at java.net.Socket.connect(Socket.java:538)
    at java.net.Socket.<init>(Socket.java:434)
    at java.net.Socket.<init>(Socket.java:244)
    at org.apache.flink.streaming.experimental.CollectSink.open(CollectSink.java:80)
    ... 7 more

怎么了?事实上,我写了一个复杂的代码,但是却遇到了同样的错误,所以我删除了所有代码。

我添加的罐子是:

flink-1.6.1-bin-scala_2.11\flink-1.6.1\lib\flink-dist_2.11-1.6.1.jar
flink-1.6.1-bin-scala_2.11\flink-1.6.1\lib\flink-python_2.11-1.6.1.jar
flink-1.6.1-bin-scala_2.11\flink-1.6.1\lib\log4j-1.2.17.jar
flink-1.6.1\lib\slf4j-log4j12-1.7.7.jar

0 个答案:

没有答案