客户端未连接到Flink中的任何Elasticsearch节点

时间:2017-09-04 03:34:09

标签: maven elasticsearch apache-flink elasticsearch-plugin flink-streaming

我正在使用Flink 1.1.2并在Maven中添加了ElesticSearch依赖关系,如下所示

 <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-elasticsearch2_2.10</artifactId>
            <version>1.2.0</version>
 </dependency>

我的程序包含以下代码,用于从Kafka读取数据并插入弹性搜索

public class ReadFromKafka {


    public static void main(String[] args) throws Exception {
        // create execution environment
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        Properties properties = new Properties();
        properties.setProperty("bootstrap.servers", "localhost:9092");
        properties.setProperty("zookeeper.connect", "localhost:2181");
        properties.setProperty("group.id", "test");


        DataStream<JoinedStreamEvent> message = env.addSource(new FlinkKafkaConsumer09<JoinedStreamEvent>("test",
                new JoinSchema(), properties));

        System.out.println("reading form kafka ");

        message.print();


        Map<String, String> config = new HashMap<>();
        config.put("bulk.flush.max.actions", "1");   // flush inserts after every event
        config.put("cluster.name", "elasticsearch_amar"); // default cluster name

        List<InetSocketAddress> transports = new ArrayList<>();
// set default connection details
        transports.add(new InetSocketAddress(InetAddress.getByName("127.0.0.1"), 9300));

       message.addSink(new ElasticsearchSink<>(config,transports,new ElasticInserter()));

        env.execute();


    } //main

    public static class ElasticInserter implements ElasticsearchSinkFunction<JoinedStreamEvent>{


        @Override
        public void process(JoinedStreamEvent record, RuntimeContext runtimeContext, RequestIndexer requestIndexer) {

            Map<String, Integer> json = new HashMap<>();

            json.put("Time", record.getPatient_id());
            json.put("heart Rate ", record.getHeartRate());
            json.put("resp rete", record.getRespirationRate());


            IndexRequest rqst = Requests.indexRequest()
                    .index("nyc-places")           // index name
                    .type("popular-locations")     // mapping name
                    .source(json);

            requestIndexer.add(rqst);

        } //process


    } //ElasticInserter

} //ReadFromKafka

我使用homebrew安装了ElesticSearch,然后使用elesticsearch命令启动它,如下所示

enter image description here

然而,当我开始我的程序时,我得到了以下错误 enter image description here

1 个答案:

答案 0 :(得分:0)

我的声望低于50,无法发表评论。

我有一点建议:

  • 首先检查ES是否已启动, 见Can't Connect to Elasticsearch (through Curl)
  • 建议使用docker容器启动ES,例如。 docker run -d --name es -p 9200:9200 elasticsearch:2 -Des.network.host = 0.0.0.0
  • BTW,您可以尝试:在ES配置es.network.host中将0.0.0.0值修改为elasticsearch.yml