Question

我正在编写一个程序来管理文件，即将文件（如对象）从directory1移动到2.我在级联中编写了一段代码，我在想如何编写“管道”来创建一个流动。

有人可以帮助我吗？

感谢。

// access path to the file input and output and archiving
String inputPath = args[0];         //Directory in HDFS of the Input
String outputPath = args[1];        //Directory in HDFS of the Input

File inFile = new File(inputPath);
File outFile = new File(outputPath);

FileUtils.moveFile(inFile, outFile);

//Set up the configuration Properties
Properties properties = new Properties();
AppProps.setApplicationJarClass(properties, MoveToArchive.class);
FlowConnector flowConnector = new Hadoop2MR1FlowConnector(properties);

//Create Sources and Sinks Taps
Tap inputTap = new Hfs(new TextLine(), inputPath);
Tap outputTap = new Hfs(new TextLine(), outputPath);

Pipe copyPipe = new Pipe("copy");
copyPipe = new Each(copyPipe, SelectFileFunction(inFile));

FlowDef flowDef = FlowDef.flowDef()
            .setName("archive")
            .addSource("input", inputTap)
            .addSource("output", outputTap);

flowConnector.connect(flowDef).complete();

try{
    if(outFile.exists()){
        System.err.println("file has been moved successfully!");
    }
} catch(Exception e){
        System.err.println("file not found in Archive Directory");
}

Answer 1

您的管道未连接到任何源或接收器。想想管道。

替换

        .addSource("input", inputTap)
        .addSource("output", outputTap);

与

        .addSource(copyPipe, inputTap)
        .addTailSink(copyPipe, outputTap);

我如何创建管道？

1 个答案: