无法使用Flink API插入mongoDB

时间:2018-09-21 05:52:19

标签: java mongodb apache-flink

我正在尝试使用flink将数据写入引用https://github.com/okkam-it/flink-mongodb-test到mongoDB

这是我尝试过的一段代码

DataSet<Tuple2<Integer, Integer>> in = env.fromElements(new Tuple2<Integer, Integer>(1,2),
            new Tuple2<Integer, Integer>(2,3)
            );

    DataSet<Tuple2<Text, BSONWritable>> mongoResult = in.map(new MapFunction<Tuple2<Integer, Integer>, Tuple2<Text, BSONWritable>>() {
        @Override
        public Tuple2<Text, BSONWritable> map(Tuple2<Integer, Integer> integerPointTuple2) throws Exception {
            DBObject builder = BasicDBObjectBuilder.start()
                    .add("id", integerPointTuple2.getField(0))
                    .add("type", "this is created from flink")
                    .get();

            BSONWritable w = new BSONWritable(builder);



            return new Tuple2<Text, BSONWritable>(new Text("Ravali"), w);
        }
    });


    mongoResult.print();

    JobConf conf = new JobConf();
    conf.set("mongo.output.uri", "mongodb://localhost:27017/test.demo");
    MongoOutputFormat<Text, BSONWritable> mongoOutputFormat = new MongoOutputFormat<Text, BSONWritable>();

    MongoConfigUtil.setOutputURI(hdIf.getJobConf() , "mongodb://localhost:27017/test.demo");

    mongoResult.output(new HadoopOutputFormat<Text, BSONWritable>(mongoOutputFormat, conf));
env.execute();

我能够读取数据,但是无法将数据写入mongoDB。

如果我做错了任何事情,请提供帮助。

0 个答案:

没有答案