Question

我有以下工作，还有许多其他工作，像这样推出

String jobName = MR.class
            .getSimpleName();
    Configuration config = HBaseConfiguration.create();

    config.set(
            "hbase.master", HBASE_MASTER);
    config.set(
            "hbase.zookeeper.quorum", HBASE_ZOOKEEPERS);
    config.set(
            "hbase.zookeeper.property.clientPort", ZK_CLIENT_PORT);
    Job job = Job.getInstance(config, jobName);

    job.setJarByClass(MR.class);
    Scan scan = new Scan();

    //gets the raw band data cells
    scan.addColumn("gc".getBytes(), "s".getBytes());
    System.out.println("Job=" + jobName);
    System.out.println("\tHBASE_MASTER=" + HBASE_MASTER);
    System.out.println(
            "\tHBASE_ZOOKEEPERS=" + HBASE_ZOOKEEPERS);
    System.out.println(
            "\tINPUT_TABLE=" + INPUT_TABLE);
    System.out.println(
            "\tOUTPUT_TABLE= " + OUTPUT_TABLE);
    System.out.println(
            "\tCACHING_LEVEL= " + CACHING_LEVEL);
    // TODO: make caching size configurable
    scan.setCaching(CACHING_LEVEL);

    scan.setCacheBlocks(
            false);

    // null for output key/value since we're not sending anything to reduce
    TableMapReduceUtil.initTableMapperJob(INPUT_TABLE, scan,
            MR.MapClass.class,
            null,
            null,
            job);

    TableMapReduceUtil.initTableReducerJob(
            "PIXELS", // output table
            null, // reducer class
            job);
    job.setNumReduceTasks(0);
    // at least one, adjust as required

    boolean b = job.waitForCompletion(true);

但我一直收到此错误，甚至无法将数据读入映射器

Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000009_2, Status : FAILED
Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000018_2, Status : FAILED
Error: java.io.BufferedReader.lines()Ljava/util/stream/Stream;
14/09/22 22:11:13 INFO mapreduce.Job: Task Id : attempt_1410880772411_0045_m_000002_2, Status : FAILED

之前从未见过这个，一直使用HBase .... 这是我尝试过的：

I can scan the table via the hbase shell no problem in the same way this job does
I can use the java api to scan in the same way no problem
I built and rebuilt my jar thinking some files were corrupt... not the problem
I have at least 5 other jobs setup the same way
I disabled, enabled, compacted, rebuilt everything you can imagine with the table
I am using maven shade to uber jar
I am running HBase 0.98.1-cdh5.1.0

非常感谢任何帮助或想法。

Answer 1

看起来我应该等一会儿问这个问题。问题是我实际上是在获取数据，但我的一个外部jar是使用Java 8构建的，并且.lines（）在java 7的BufferedReader中不存在，我正在运行java 7，因此错误。与HBase或Map Reduce无关。

地图减少超过HBase的错误，为什么？

1 个答案: