Question

我从mapper发出了2 D双数组作为值，并试图在reducer.converted中将其访问为double，以获得所有2D数组的总和。

public static class DoubleTwoDArrayWritable extends TwoDArrayWritable {
        public DoubleTwoDArrayWritable () { 
             super (DoubleWritable.class) ;
        }
    }

减速

public class ReducerSvm extends Reducer<Text, DoubleTwoDArrayWritable, Text, Text>{
    public void reduce(Text key,Iterable<DoubleTwoDArrayWritable> values,Context context){
        System.out.println("key------"+key.toString());
        Writable [][] getArray = null;
        double C[][] = new double[3][1];
        for (DoubleTwoDArrayWritable value : values)
        {
            getArray = value.get();
            for (int i=0; i<3 ; i++ )
            {
                for (int j=0 ; j<1 ; j++ ){
                    System.out.println("v--> "+((DoubleWritable)getArray[i][j]).get());
                    C[i][j] = ((DoubleWritable)getArray[i][j]).get();
                }
            }

            System.out.println("C array");
            for (int i=0; i<3 ; i++ ){
                for (int j=0 ; j<1 ; j++ ){
                    System.out.println(C[i][j]+" ");
                }
                System.out.println("");
            }
        }

我能够在Reducer中获取我的双数组。但是我hardcoded我的行和值。使用row

时，如何在reducer中计算column和TwoDArrayWritable

修改

正如Balduz所说，我编辑了代码

public void reduce(Text key,Iterable<DoubleTwoDArrayWritable> values,Context context){

        for (DoubleTwoDArrayWritable value : values) {
            Writable[][] currentArray = value.get();
            int rowSize = currentArray.length;
            int columnSize = currentArray[0].length;
            System.out.println("row size: "+rowSize);
            double[][] myArray = new double[rowSize][columnSize];

            for (int i = 0; i < currentArray.length; i++) {
                for (int j = 0; i < currentArray[i].length; j++) {
                     myArray[i][j] = ((DoubleWritable)currentArray[i][j]).get();
                }
            }
            System.out.println("myArray array");
            for (int i=0; i<myArray.length ; i++ ){
                for (int j=0 ; j<myArray[0].length ; j++ ){
                    System.out.println(myArray[i][j]+" ");
                }
                System.out.println("");
            }

        }
}
}

我能够正确获取行大小。

但显示

java.lang.ArrayIndexOutOfBoundsException: 1
    at edu.am.bigdata.svmmodel.ReducerTrail.reduce(ReducerTrail.java:26)
    at edu.am.bigdata.svmmodel.ReducerTrail.reduce(ReducerTrail.java:1)
    at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:164)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:610)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:444)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:449)

Answer 1

首先，请不要调用变量getArray，因为它看起来像是方法名称而导致混淆。要迭代每个矩阵，您需要执行以下操作：

for (DoubleTwoDArrayWritable value : values) {
    Writable[][] currentArray = value.get();
    for (int i = 0; i < currentArray.length; i++) {
        for (int j = 0; j < currentArray[i].length; j++) {
             DoubleWritable valueYouWant = (DoubleWritable)currentArray[i][j];
        }
    }
}

修改要将整个矩阵存储在变量中，我假设每行具有相同的列数。在这种情况下，您可以像这样初始化它：

for (DoubleTwoDArrayWritable value : values) { Writable[][] currentArray = value.get(); int rowSize = currentArray.length; int columnSize = currentArray[0].length; double[][] myArray = new double[rowSize][columnSize]; for (int i = 0; i < currentArray.length; i++) { for (int j = 0; j < currentArray[i].length; j++) { myArray[i][j] = ((DoubleWritable)currentArray[i][j]).get(); } } }

Answer 2

在可写密钥DoubleTwoDArrayWritable中公开两个方法，并从reducer中调用这两个方法来获取信息。

public static class DoubleTwoDArrayWritable extends TwoDArrayWritable {
    public DoubleTwoDArrayWritable () { 
         super (DoubleWritable.class) ;
    }

    public int getRow() {
        Writable[][] 2dArray = super.get();
        return 2dArray.length;
    }

    public int getColumn() {
        Writable[][] 2dArray = super.get();
        return 2dArray[0].length;
    }
}

希望它有所帮助。

TwoDArrayWritable中的行列值

2 个答案: