我正在将Linux用于大数据主题的大学项目。我使用要通过Hadoop分析的Intellij构建了一个jar。该项目包括3个职位。分析完第一份工作后，shell上出现以下错误：
> > 2019-11-12 01:29:18,203 INFO mapreduce.Job: Job job_1573503622064_0002 completed successfully 2019-11-12 01:29:33,594
> INFO mapreduce.Job: Counters: 55
> File System Counters
> FILE: Number of bytes read=1116078848
> FILE: Number of bytes written=2283517013
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> HDFS: Number of bytes read=27162033259
> HDFS: Number of bytes written=38
> HDFS: Number of read operations=686
> HDFS: Number of large read operations=0
> HDFS: Number of write operations=2
> HDFS: Number of bytes read erasure-coded=0
> Job Counters
> Killed map tasks=1
> Launched map tasks=227
> Launched reduce tasks=1
> Data-local map tasks=227
> Total time spent by all maps in occupied slots (ms)=7571323
> Total time spent by all reduces in occupied slots (ms)=1386257
> Total time spent by all map tasks (ms)=7571323
> Total time spent by all reduce tasks (ms)=1386257
> Total vcore-milliseconds taken by all map tasks=7571323
> Total vcore-milliseconds taken by all reduce tasks=1386257
> Total megabyte-milliseconds taken by all map tasks=7753034752
> Total megabyte-milliseconds taken by all reduce tasks=1419527168
> Map-Reduce Framework
> Map input records=305975157
> Map output records=111607883
> Map output bytes=892863064
> Map output materialized bytes=1116080192
> Input split bytes=29026
> Combine input records=0
> Combine output records=0
> Reduce input groups=1
> Reduce shuffle bytes=1116080192
> Reduce input records=111607883
> Reduce output records=1
> Spilled Records=223215766
> Shuffled Maps =227
> Failed Shuffles=0
> Merged Map outputs=227
> GC time elapsed (ms)=140246
> CPU time spent (ms)=457710
> Physical memory (bytes) snapshot=51324133376
> Virtual memory (bytes) snapshot=576444985344
> Total committed heap usage (bytes)=38742728704
> Peak Map Physical memory (bytes)=233918464
> Peak Map Virtual memory (bytes)=2528382976
> Peak Reduce Physical memory (bytes)=718159872
> Peak Reduce Virtual memory (bytes)=2536824832
> Shuffle Errors
> BAD_ID=0
> CONNECTION=0
> IO_ERROR=0
> WRONG_LENGTH=0
> WRONG_MAP=0
> WRONG_REDUCE=0
> File Input Format Counters
> Bytes Read=27162004233
> File Output Format Counters
> Bytes Written=38 2019-11-12 01:29:35,986 INFO ipc.Client: Retrying connect to server: ubuntu/127.0.1.1:44561. Already tried 0
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:36,988 INFO ipc.Client: Retrying
> connect to server: ubuntu/127.0.1.1:44561. Already tried 1 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:37,989 INFO ipc.Client:
> Retrying connect to server: ubuntu/127.0.1.1:44561. Already tried 2
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:40,523 INFO
> mapred.ClientServiceDelegate: Application state is completed.
> FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> 2019-11-12 01:29:41,858 INFO ipc.Client: Retrying connect to server:
> 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:42,860 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:43,863 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:44,866 INFO ipc.Client: Retrying
> connect to server: 0.
> 0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:45,867 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:46,869 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:47,872 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:48,874 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:49,876 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:50,878 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:51,012 INFO
> mapred.ClientServiceDelegate: Application state is completed.
> FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> 2019-11-12 01:29:52,013 INFO ipc.Client: Retrying connect to server:
> 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:53,016 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:54,021 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:55,023 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:56,025 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:57,026 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:29:58,028 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:29:59,030 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:00,031 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:30:01,032 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:03,208 INFO
> mapred.ClientServiceDelegate: Application state is completed.
> FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> 2019-11-12 01:30:04,214 INFO ipc.Client: Retrying connect to server:
> 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:3 0:05,217 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:06,218 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:30:07,219 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:08,221 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:30:09,223 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:10,225 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:30:11,228 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) 2019-11-12 01:30:12,232 INFO ipc.Client:
> Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8
> time(s); retry policy is
> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS) 2019-11-12 01:30:13,233 INFO ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s);
> retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS) Exception in thread "main"
> java.io.IOException: java.net.ConnectException: Your endpoint
> configuration is wrong; For more details see:
> http://wiki.apache.org/hadoop/UnsetHostnameOrPort
> at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:345)
> at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:430)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:871)
> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:331)
> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:328)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:328)
> at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:624)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1603)
> at FirstProject.main(FirstProject.java:33)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Your endpoint configuration is wrong; For
> more details see: http://wiki.apache.org/hadoop/UnsetHostnameOrPort
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:833)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:753)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1549)
> at org.apache. hadoop.ipc.Client.call(Client.java:1491)
> at org.apache.hadoop.ipc.Client.call(Client.java:1388)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
> at com.sun.proxy.$Proxy14.getJobReport(Unknown Source)
> at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:326)
> ... 17 more Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:533)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:700)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:804)
> at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:421)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1606)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> ... 26 more
Hadoop配置是正确的，因为我分析了同一个jar和其他数据。此外，YARN和DFS也将启动。我删除输出文件夹，然后创建另一个。你能帮我理解吗？
答案 0 :(得分：0)
能否再次检查Hadoop配置？
这项工作正在尝试连接0.0.0.0/0.0.0.0:10020，这似乎不是正确的模式。应该像IP:Port
也许此作业将覆盖您的默认配置。