从hadoop中的deprecations.getDeprecatedKeyMap()。get(name)返回null

时间:2019-09-02 01:56:26

标签: hadoop mapreduce

我试图在简单的map reduce上运行hadoop程序,并且正在JobJob.java中调用以下方法

public RawComparator getOutputKeyComparator() {
    Class<? extends RawComparator> theClass = getClass(
      JobContext.KEY_COMPARATOR, null, RawComparator.class);
    if (theClass != null)
      return ReflectionUtils.newInstance(theClass, this);
    return WritableComparator.get(getMapOutputKeyClass().asSubclass(WritableComparable.class), this);
  }

现在上述方法最终会在Configuration.java上最终调用以下方法

private String[] handleDeprecation(DeprecationContext deprecations,
                                     String name) {
    if (null != name) {
      name = name.trim();
    }
    // Initialize the return value with requested name
    String[] names = new String[]{name};
    // Deprecated keys are logged once and an updated names are returned
    DeprecatedKeyInfo keyInfo = deprecations.getDeprecatedKeyMap().get(name);

现在,当尝试从depricatedkeymap中获取名称:mapreduce.job.output.key.comparator.class的值时,keyInfo返回null。请帮助我查找问题。

我收到以下异常:

java.lang.ClassCastException: interface javax.xml.soap.Text
    at java.lang.Class.asSubclass(Unknown Source)
    at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:903)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1018)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:408)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:82)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:710)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:782)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

0 个答案:

没有答案