Question

我正在使用Sqoop将数据从oracle导入到HDFS。当Job启动时，它停留在5％的进度中大约1个小时，并且此信息是输出：

INFO mapreduce.Job: Task Id : attempt_1535519556038_0015_m_000037_0, Status : FAILED
Container launch failed for container_1535519556038_0015_01_000043 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
This token is expired. current time is 1536133107764 found 1536133094775
Note: System times on machines may be out of sync. Check system time and time zones.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168)
        at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:155)
        at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:375)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

，然后继续，直到作业成功终止并导入所有数据。所以，我的问题是将工作挂在进度的5％上的原因是什么？为什么会自我校正？正常吗如果不是，是否有可能与该发布的信息相关？我该如何解决？

Answer 1

该错误消息清楚地说明了“未经授权的容器启动请求。该令牌已过期”。

其中一种选择是通过设置以下内容来延长容器的使用寿命： yarn.resourcemanager.rm.container-allocation.expiry-interval-ms（默认情况下为 10分钟）。

注意：如果您在yarn.resourcemanager.rm.container-allocation.expiry-interval-ms配置文件中增加yarn-site.xml，则这些作业将起作用。

<property>
   <name>yarn.resourcemanager.rm.container-allocation.expiry-interval-ms</name>
   <value>1000000</value>
</property>

Sqoop卡在进度的5％

1 个答案: