如何将flink表保存到csv文件?

时间:2018-05-27 17:35:48

标签: apache-flink

我正在使用Flink 1.4.0

我试图将Table API查询的结果保存到CSV文件中,但我还是 得到一个错误。 以下是详细信息:

我的输入文件如下所示:

id,species,color,weight,name 
311,canine,golden,75,dog1 
312,canine,brown,22,dog2 
313,feline,gray,8,cat1

我对此运行查询仅选择犬科动物,我想将其保存到 csv文件:

ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment(); 
BatchTableEnvironment tableEnv = 
TableEnvironment.getTableEnvironment(env); 

String inputPath = "location-of-source-file"; 
CsvTableSource petsTableSource = CsvTableSource.builder() 
  .path(inputPath) 
  .ignoreFirstLine() 
  .fieldDelimiter(",") 
  .field("id", Types.INT()) 
  .field("species", Types.STRING()) 
  .field("color", Types.STRING()) 
  .field("weight", Types.DOUBLE()) 
  .field("name", Types.STRING()) 
  .build(); 

// Register our table source 
tableEnv.registerTableSource("pets", petsTableSource); 
Table pets = tableEnv.scan("pets"); 

Table counts = pets 
  .groupBy("species") 
  .select("species, species.count as count") 
  .filter("species === 'canine'"); 

// Convert to Dataset and display results
DataSet<Row> result = tableEnv.toDataSet(counts, Row.class); 
result.print(); 

// Write Results to File 
TableSink<Row> sink = new CsvTableSink("/home/hadoop/output/pets.csv", ","); 
counts.writeToSink(sink); 

当我运行它时,我看到输出DataSet的结果:
犬科动物,2

但是我在输出文件中没有得到任何结果,我看到了这些 以下错误。我该怎么做才能解决这个问题?谢谢!

2018-05-27 13:29:17,040 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1873) - class org.apache.flink.types.Row does not contain a getter for field fields
2018-05-27 13:29:17,040 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1876) - class org.apache.flink.types.Row does not contain a setter for field fields
2018-05-27 13:29:17,040 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1911) - class org.apache.flink.types.Row is not a valid POJO type because not all fields are valid POJO fields.
2018-05-27 13:29:17,047 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1873) - class org.apache.flink.types.Row does not contain a getter for field fields
2018-05-27 13:29:17,047 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1876) - class org.apache.flink.types.Row does not contain a setter for field fields
2018-05-27 13:29:17,047 INFO  [main] typeutils.TypeExtractor (TypeExtractor.java:1911) - class org.apache.flink.types.Row is not a valid POJO type because not all fields are valid POJO fields.

1 个答案:

答案 0 :(得分:0)

你可能错过了

env.execute(); 

之后

counts.writeToSink(sink);

与立即触发执行的print()相比,writeToSink()只需附加一个接收器操作符,并且需要显式触发执行。

INFO的{​​{1}}消息“只是”告诉您,TypeExtractor不能用作POJO类型,但这里没问题。