级联 - 根据字段值加入两个文件

Question

我正在尝试根据一些常见字段值加入两个文件并获取所有匹配的记录。

我有两个public static void PayTutionFees(IStudent student) { //Design problem (Student)student.ID; var realStudent = student as Student; //to avoid wrong cast check for type before if(student.GetType() == typeof(Student)) {//jippy student is a Student!} } 两个文件正在阅读。我想使用Tap no加入文件并获取匹配的记录。

如何加入文件和Field管道以创建assemble？

示例代码：

Flow

Answer 1

构建连接到水龙头的管道，连接它们，然后将输出管道连接到水槽。

Tap outTap = new MultiSinkTap(); // just saying, create your own tap
Pipe custFilePipe = new Pipe("custFilePipe");
Pipe tsctnFilePipe = new Pipe("tsctnFilePipe");

Fields groupFields = new Fields("no"); // fields used as joining keys
Pipe outPipe = new CoGroup(custFilePipe, groupFields, tsctnFilePipe, groupFields, new InnerJoin());

// build flow definition
FlowDef flowDef = FlowDef.flowDef().setName("myFlow")
 .addSource(custFilePipe, custFileTap)
 .addSource(tsctnFilePipe, tsctnFileTap)
 .addTailSink(outPipe, outTap);

Flow flow = flowConnector.connect(flowDef); // now you build the flow
flow.complete(); // run flow

Cascading for the Impatient是一个很好的教程，我推荐给Cascading初学者。