我可以在Tez上运行hive,但是在tez ui中找不到任何工作。
它会让我疯狂!
配置是打击: TEZ-site.xml中
<property>
<name>tez.history.logging.service.class</name>
<value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value>
</property>
<property>
<description>URL for where the Tez UI is hosted</description>
<name>tez.tez-ui.history-url.base</name>
<value>http://10.0.0.51:8080/tez-ui</value>
</property>
和yarn-site.xml
<property>
<name>yarn.timeline-service.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.timeline-service.hostname</name>
<value>0.0.0.0</value>
</property>
<property>
<name>yarn.timeline-service.http-cross-origin.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.timeline-service.generic-application-history.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.timeline-service.webapp.address</name>
<value>${yarn.timeline-service.hostname}:8188</value>
</property>
<property>
<name>yarn.timeline-service.webapp.https.address</name>
<value>${yarn.timeline-service.hostname}:2191</value>
</property>
和网址:
http://10.0.0.51:8188/ws/v1/timeline/TEZ_DAG_ID
http://10.0.0.51:8188/ws/v1/timeline/TEZ_APPLICATION_ATTEMPT
http://10.0.0.51:8188/ws/v1/timeline/TEZ_APPLICATION
所有这些,我只是得到以下相同的响应:
{
entities: [ ]
}
答案 0 :(得分:0)
据我在时间轴服务器UI中看到,您没有关于作业的信息。在Timeline和Tez UI中,您无法查看有关在启用Timeline Server之前运行的作业的信息,因为Timeline Server使用LevelDB存储,并且应该在作业运行期间发布信息。
答案 1 :(得分:0)
在我的情况下,发现这是YARN ACL问题。 因此以下内容对我有所帮助:
yarn.acl.enable = false
or
yarn.admin.acl = activity_analyzer,yarn,dr.who,admin
作为奖励,收集了HDP 3.1 + TEZ 0.9.2的完整配置:
YARN配置:
yarn.timeline-service.enabled = true
yarn.acl.enable = false
yarn.admin.acl = activity_analyzer,yarn,dr.who,admin
yarn.timeline-service.webapp.address = <host>:8188
yarn.timeline-service.version = 2,0f
yarn.timeline-service.hostname = <host>
yarn.timeline-service.http-cross-origin.enabled = true
yarn.timeline-service.http-cross-origin.allowed-origins = *
yarn.resourcemanager.system-metrics-publisher.enabled = true
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes = org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl
TEZ配置:
yarn.timeline-service.enabled = true
tez.tez-ui.history-url.base = http://<host>/tez-ui/
tez.am.tez-ui.history-url.template = __HISTORY_URL_BASE__?viewPath=/#/tez-app/__APPLICATION_ID__
tez.history.logging.service.class = org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService
tez.dag.history.logging.enabled = true
tez.am.history.logging.enabled = true
tez.allow.disabled.timeline-domains = true
配置配置:
hive_timeline_logging_enabled = true
hive.exec.pre.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook
hive.exec.post.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
hive.exec.failure.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook
HDFS配置:
hadoop.http.filter.initializers = org.apache.hadoop.security.HttpCrossOriginFilterInitializer