从Java Application启动Flume Agent

时间:2015-02-16 02:58:11

标签: java apache hadoop flume flume-ng

我编写了一个Java应用程序,用于构建flume配置文件并将它们写入Linux机器上的磁盘。然后,应用程序构建flume命令并尝试通过运行以下代码中的命令来启动flume代理:

  try {
                    Process p = Runtime.getRuntime().exec(flumeStartCommand.toString());
                    p.waitFor(); 
                    BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
                    String line = null;
                    while ((line = in.readLine()) != null) {
                        System.out.println(line);
                    }                       

如果我使用flume命令并通过终端手动执行它,它可以正常工作。当java应用程序尝试执行flume命令时,没有任何反应。这是一个类路径问题吗?

Flume Command在这里: /root/flume-flume-1.6/flume-ng-dist/target/apache-flume-1.6.0-SNAPSHOT-bin/apache-flume-1.6.0-SNAPSHOT-bin/bin/flume-ng agent --conf conf --conf-file /root/flumeconfs/ConsumerGroup4.conf --name Agent_ConsumerGroup4 -Dflume.root.logger = INFO,console -Dflume.monitoring.type = http -Dflume.monitoring.port = 34548

我知道嵌入式代理是另一种选择,但这些仅适用于对我来说不是很有用的avro接收器。欣赏任何想法。 干杯, 科尔曼

1 个答案:

答案 0 :(得分:0)

对于其他任何想要解决这个问题的人来说,需要将flume命令分解为String数组中的参数,如下所示:

String[] flumeCommand = new String[]{"/root/flume-flume-1.6/flume-ng-dist/target/apache-flume-1.6.0-SNAPSHOT-bin/apache-flume-1.6.0-SNAPSHOT-bin/bin/flume-ng",
                                   "agent",
                                   "--conf",
                                   "conf",
                                   "--conf-file",
                                   _flumeConfigurationDir+Active.getFlumeConfName(),
                                   "--name",
                                   Active.getFlumeAgentName(),
                                   "-Dflume.root.logger=INFO,console",
                                   "-Dflume.monitoring.type=http",
                                   "-Dflume.monitoring.port="+Active.getFlumeMetricsPort(),
};

然后使用以下代码:

ProcessBuilder flumeCommandBuilder = new ProcessBuilder(flumeCommand);
File flumeConfigurationDirFile = new File(_flumeConfigurationDir);
flumeCommandBuilder.directory(flumeConfigurationDirFile);                       
System.out.println("Working Directory: "+ flumeCommandBuilder.directory());
flumeCommandBuilder.redirectErrorStream(true);
Process process = flumeCommandBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
    System.out.println("tasklist: " + line);