特定配置后,Dataproc重新启动配置单元服务

时间:2018-10-18 07:33:43

标签: mysql hive google-cloud-dataproc

与Dataproc合作,我正在设置不同的配置以优化配置单元查询:

<property>
    <name>hive.vectorized.execution.enabled</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.cbo.enable</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.compute.query.using.stats</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.stats.fetch.column.stats</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.stats.fetch.partition.stats</name>
    <value>true</value>
   </property>

应用这些更改并重新启动hive meta-store和hiveserver后,由于不同节点上的连接问题,作业开始失败:

 ERROR, exception: java.lang.RuntimeException: org.apache.thrift.transport.TTransportException, result: %text org.apache.thrift.transport.TTransportException

但是,这些似乎是由于蜂巢mysql连接而发生的:

Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27552]: Upgrading MySQL tables if necessary.
Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27555]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27555]: Looking for 'mysql' as: /usr/bin/mysql
Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27555]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27555]: This installation of MySQL is already upgraded to 10.2.18-MariaDB, use --force if you still need to run mysql_upgrade
Oct 18 06:55:23 xxx-spark-m /etc/mysql/debian-start[27567]: Checking for insecure root accounts.
Oct 18 06:55:23 xxxx-spark-m /etc/mysql/debian-start[27571]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables
Oct 18 06:55:24 xxxx-spark-m mysqld[27488]: 2018-10-18  6:55:24 139690715899648 [Note] InnoDB: Buffer pool(s) load completed at 181018  6:55:24
Oct 18 07:05:46 xxxx-spark-m mysqld[27488]: 2018-10-18  7:05:46 139691201689344 [Warning] Aborted connection 42 to db: 'metastore' user: 'hive' host: 'host-xxx' (Got timeout reading communication packets)

我的问题首先显示为什么:

This installation of MySQL is already upgraded to 10.2.18-MariaDB, use --force if you still need to run mysql_upgrade

我也愿意访问mysql db来为hive用户创建特权,但是我无法弄清楚如何在dataproc上访问mysql db。

0 个答案:

没有答案