我正在使用此hadoop mapreduce example application
:
command
hadoop jar e:/Apps/WordCount/hadoop-mapreduce-examples-2.2.0.jar wordcount /users/pavel/wordcount/input /users/pavel/wordcount/output7
我在日志中看到此错误:
13/12/14 12:54:17 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
13/12/14 12:54:17 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/12/14 12:54:20 INFO input.FileInputFormat: Total input paths to process : 2
13/12/14 12:54:20 INFO mapreduce.JobSubmitter: number of splits:2
13/12/14 12:54:20 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/12/14 12:54:20 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/12/14 12:54:20 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/12/14 12:54:20 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/12/14 12:54:20 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/12/14 12:54:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1979664780_0001
13/12/14 12:54:21 WARN conf.Configuration: file:/tmp/hadoop-Pavel/mapred/staging/Pavel1979664780/.staging/job_local1979664780_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
13/12/14 12:54:21 WARN conf.Configuration: file:/tmp/hadoop-Pavel/mapred/staging/Pavel1979664780/.staging/job_local1979664780_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
13/12/14 12:54:22 WARN conf.Configuration: file:/tmp/hadoop-Pavel/mapred/local/localRunner/Pavel/job_local1979664780_0001/job_local1979664780_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
13/12/14 12:54:22 WARN conf.Configuration: file:/tmp/hadoop-Pavel/mapred/local/localRunner/Pavel/job_local1979664780_0001/job_local1979664780_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
13/12/14 12:54:22 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
13/12/14 12:54:22 INFO mapreduce.Job: Running job: job_local1979664780_0001
13/12/14 12:54:22 INFO mapred.LocalJobRunner: OutputCommitter set in config null
13/12/14 12:54:22 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
13/12/14 12:54:22 INFO mapred.LocalJobRunner: Waiting for map tasks
13/12/14 12:54:22 INFO mapred.LocalJobRunner: Starting task: attempt_local1979664780_0001_m_000000_0
13/12/14 12:54:22 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
13/12/14 12:54:23 INFO mapreduce.Job: Job job_local1979664780_0001 running in uber mode : false
13/12/14 12:54:23 INFO mapreduce.Job: map 0% reduce 0%
13/12/14 12:55:13 INFO mapred.LocalJobRunner: Starting task: attempt_local1979664780_0001_m_000001_0
13/12/14 12:55:13 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
13/12/14 12:57:14 INFO mapred.LocalJobRunner: Map task executor complete.
13/12/14 12:57:14 WARN mapred.LocalJobRunner: job_local1979664780_0001
java.lang.Exception: java.lang.NullPointerException
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
Caused by: java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:441)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:404)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.util.WindowsBasedProcessTree.isAvailable(WindowsBasedProcessTree.java:57)
at org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree.getResourceCalculatorProcessTree(ResourceCalculatorProcessTree.java:153)
at org.apache.hadoop.mapred.Task.initialize(Task.java:579)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:235)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
13/12/14 12:57:15 INFO mapreduce.Job: Job job_local1979664780_0001 failed with state FAILED due to: NA
13/12/14 12:57:15 INFO mapreduce.Job: Counters: 0
我尝试调试此问题并注意到无效的命令列表从ProcessBuilder object
传递给Shell object
:String[2]: {null, 'help'}
那null
是问题的原因。
但我对如何解决这个问题没有任何想法......你能帮忙吗?
环境: Windows 8,Java 6.应用程序已从cygwin
启动。