Question

我正在做大学作业，我们必须使用hadoop mapreduce。我试图创建一个新的自定义可写，因为我想输出键值对（键，（doc_name，1））。

public class Detector {

    private static final Path TEMP_PATH = new Path("temp");
    private static final String LENGTH = "gramLength";
    private static final String THRESHOLD = "threshold";


    public class Custom implements Writable {

        private Text document;
        private IntWritable count;

        public Custom(){
            setDocument("");
            setCount(0);
        }

        public Custom(String document, int count) {
            setDocument(document);
            setCount(count);
        }

        @Override
        public void readFields(DataInput in) throws IOException {
            // TODO Auto-generated method stub
            document.readFields(in);
            count.readFields(in);
        }

        @Override
        public void write(DataOutput out) throws IOException {
            document.write(out);
            count.write(out);
        }

        public int getCount() {
            return count.get();
        }

        public void setCount(int count) {
            this.count = new IntWritable(count);
        }

        public String getDocument() {
            return document.toString();
        }

        public void setDocument(String document) {
            this.document = new Text(document);
        }

    }

    public static class NGramMapper extends Mapper<Text, Text, Text, Text> {
        private int gramLength;
        private Pattern space_pattern=Pattern.compile("[ ]");
        private StringBuilder gramBuilder= new StringBuilder();

        @Override
        protected void setup(Context context) throws IOException,      InterruptedException{
            gramLength=context.getConfiguration().getInt(LENGTH, 0);
        }

        public void map(Text key, Text value, Context context) throws IOException, InterruptedException {
            String[] tokens=space_pattern.split(value.toString());
            for(int i=0;i<tokens.length;i++){
                gramBuilder.setLength(0);
                if(i+gramLength<=tokens.length){
                    for(int j=i;j<i+gramLength;j++){
                        gramBuilder.append(tokens[j]);
                        gramBuilder.append(" ");
                    }
                    context.write(new Text(gramBuilder.toString()), key);
                }
            }
        }
    }


    public static class OutputReducer extends Reducer<Text, Text, Text, Custom> {

        public void reduce(Text key, Iterable<Text> values, Context context)
                throws IOException, InterruptedException {
            for (Text val : values) {
                context.write(key,new Custom(val.toString(),1));
            }
        }
    }

    public static void main(String[] args) throws Exception {

        Configuration conf = new Configuration();
        FileSystem fs = FileSystem.get(conf);
        conf.setInt(LENGTH, Integer.parseInt(args[0]));
        conf.setInt(THRESHOLD, Integer.parseInt(args[1]));

        // Setup first MapReduce phase
        Job job1 = Job.getInstance(conf, "WordOrder-first");
        job1.setJarByClass(Detector.class);
        job1.setMapperClass(NGramMapper.class);
        job1.setReducerClass(OutputReducer.class);
        job1.setMapOutputKeyClass(Text.class);
        job1.setMapOutputValueClass(Text.class);
        job1.setOutputKeyClass(Text.class);
        job1.setOutputValueClass(Custom.class);
        job1.setInputFormatClass(WholeFileInputFormat.class);
        FileInputFormat.addInputPath(job1, new Path(args[2]));
        FileOutputFormat.setOutputPath(job1, new Path(args[3]));

        boolean status1 = job1.waitForCompletion(true);
        if (!status1) {
            System.exit(1);
        }
    }
}

当我将代码编译为类文件时，我收到此错误：

Detector.java:147: error: non-static variable this cannot be referenced from a static context
context.write(key,new Custom(val.toString(),1));

我遵循了有关自定义可写的不同教程，我的解决方案与其他解决方案相同。有什么建议吗？

Answer 1

静态字段和方法与所有实例共享。它们用于特定于类的值，而不是特定的实例。尽可能远离他们。

要解决您的问题，您需要实例化类的实例（创建对象），以便运行时可以为实例保留内存;或者更改您访问它的部分以获得static访问权限（不推荐！）。

关键字this用于引用确实是实例的内容（因此此事物），而不是static的内容，在这种情况下，应该由类名引用。您正在 static 上下文中使用它，这是不允许的。

Hadoop mapreduce自定义可写静态上下文

1 个答案: