如何在Zeppelin中将Flink var的内容写入屏幕?

时间:2015-09-04 15:14:39

标签: apache-flink apache-zeppelin

我尝试在Apache Zeppelin中运行以下简单命令。

%flink

var rabbit = env.fromElements(
"ARTHUR:  What, behind the rabbit?",
"TIM:  It is the rabbit!", 
"ARTHUR:  You silly sod!  You got us all worked up!",
"TIM:  Well, that's no ordinary rabbit.  That's the most foul, cruel, and bad-tempered rodent you ever set eyes on.",
"ROBIN:  You tit!  I soiled my armor I was so scared!", 
"TIM:  Look, that rabbit's got a vicious streak a mile wide, it's a killer!")

var counts = rabbit.flatMap { _.toLowerCase.split("\\W+")}.map{ (_,1)}.groupBy(0).sum(1) 

counts.print()

我尝试在笔记本中打印出结果。但不幸的是,我只得到以下输出。

rabbit: org.apache.flink.api.scala.DataSet[String] = org.apache.flink.api.scala.DataSet@37fdb65c
counts: org.apache.flink.api.scala.AggregateDataSet[(String, Int)] = org.apache.flink.api.scala.AggregateDataSet@1efc7158
res103: org.apache.flink.api.java.operators.DataSink[(String, Int)] = DataSink '<unnamed>' (Print to System.out)

如何在Zeppelin中将计数内容泄漏到笔记本中?

2 个答案:

答案 0 :(得分:5)

在Zeppelin中打印此类计算结果的方法是:

%flink
counts.collect().foreach(println(_))

//or one might prefer
//counts.collect foreach println 

输出:

(a,3)
(all,1)
(and,1)
(armor,1)
...

答案 1 :(得分:4)

观察到的行为的原因在于Apache Zeppelin和Apache Flink之间的相互作用。 Zeppelin捕获System.out的所有标准输出。但是,Flink还会将输出打印到counts.print(),这恰恰是您拨打Console时发生的情况。 bzz解决方案的工作原理是它使用counts.print()打印结果。

我打开了一个JIRA问题[1]并打开了一个拉取请求[2]来纠正这种行为,这样你也可以使用 public void addData(final String idSeries, final X x, final Y y, String kindOfSymbol) { //valido que exista la serie boolean seriesExist = seriesMap.containsKey(idSeries); //obtengo el index de la serie, como fue almacenada en la grafica Integer indexSerie = seriesMap.get(idSeries); if (seriesExist) { Platform.runLater(() - > { XYChart.Series < X, Y > currentSeries = null; currentSeries = chart.getData().get(indexSerie.intValue()); int lastNodePosition = currentSeries.getData().size() - 1; XYChart.Data < X, Y > point = new XYChart.Data < > (x, y); //The problem is here when I try to convert "Y" to "double" because the object JFXStackPaneSymbol require the double and Dimension parameters point.setNode(new JFXStackPaneSymbol(y, new Dimension(15, 15))); currentSeries.getData().add(point); currentSeries.getData().get(lastNodePosition).getNode().setStyle(kindOfSymbol); }); } }