在连接到mysql rds时启动时配置单元3.1 metastore错误

时间:2018-08-24 02:43:10

标签: mysql amazon-web-services apache-spark hadoop hive

尝试启动连接到新MySQL RDS的Hive Metastore 3.1版本并得到此错误。使用Hive 2.3.2时,它可以正常工作(在单独的MySQL RDS实例上)。有谁知道是什么原因造成的?在启动HiveMetastore进程之前,已经对mysql进行了初始化:

  • echo“如果不存在元存储,则创建数据库;” > hive_metastore.sql 回显“ USE metastore;” >> hive_metastore.sql echo“ SOURCE $ HIVE_HOME / scripts / metastore / upgrade / mysql / hive-schema-3.1.0.mysql.sql;“

      
        

    hive_metastore.sql echo“如果不存在'hive'@'%',则通过'$ hive_pasw'标识创建用户;” >> hive_metastore.sql echo“全部撤销     特权,允许从'hive'@'%'选择;“ >> hive_metastore.sql echo     “将所有特权授予metastore。*为'hive'@'%';” >>     hive_metastore.sql echo“如果不存在,则创建用户'ro_hive'@'%'     IDENTIFIED BY'$ ro_hive_pasw';“ >> hive_metastore.sql回显” REVOKE     所有特权,可从'ro_hive'@'%'中获得许可;“ >>     hive_metastore.sql echo“ GRANT SELECT,SHOW VIEW,PROCESS,     上的复制客户端到'ro_hive'@'%';“ >> hive_metastore.sql     回显“冲洗特权”; >> hive_metastore.sql回显“退出” >>     hive_metastore.sql

      

    18/08/24 00:17:48 INFO metastore.ObjectStore:使用hive.metastore.cache.pinobjtypes =“ Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order”设置MetaStore对象引脚类 18/08/24 00:17:48 INFO metastore.MetaStoreDirectSql:使用直接SQL,底层数据库是MYSQL 18/08/24 00:17:48 INFO metastore.ObjectStore:初始化的ObjectStore 18/08/24 00:17:49 WARN DataNucleus.MetaData:元数据的jdbc类型为null,但这无效。被忽略 18/08/24 00:17:53 WARN metastore.ObjectStore:无法获取数据库hive.default,返回NoSuchObjectException 18/08/24 00:17:53 WARN metastore.HiveMetaStore:错误后重试创建默认数据库:抛出异常,刷新对数据存储的更改 javax.jdo.JDODataStoreException:刷新数据存储区时引发异常         在org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)         在org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:171)         在org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:765)         在org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:953)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)         com.sun.proxy。$ Proxy22.createDatabase(未知来源)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultDB_core(HiveMetaStore.java:751)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultDB(HiveMetaStore.java:769)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.init(HiveMetaStore.java:538)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler。(RetryingHMSHandler.java:80)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)         在org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)         在org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)         在org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)         在org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.util.RunJar.run(RunJar.java:221)         在org.apache.hadoop.util.RunJar.main(RunJar.java:136) NestedThrowablesStackTrace: java.sql.BatchUpdateException:“字段列表”中的未知列“ CATALOG_NAME”         在sun.reflect.NativeConstructorAccessorImpl.newInstance0(本机方法)处         在sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)         在sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)         在java.lang.reflect.Constructor.newInstance(Constructor.java:423)         在com.mysql.jdbc.Util.handleNewInstance(Util.java:425)         在com.mysql.jdbc.Util.getInstance(Util.java:408)         在com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)         在com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1778)         在com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1262)         在com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)         在com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)         在com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)         在org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeBatch(ParamLoggingPreparedStatement.java:366)         在org.datanucleus.store.rdbms.SQLController.processConnectionStatement(SQLController.java:676)         在org.datanucleus.store.rdbms.SQLController.processStatementsForConnection(SQLController.java:644)         在org.datanucleus.store.rdbms.SQLController $ 1.transactionFlushed(SQLController.java:731)         在org.datanucleus.store.connection.AbstractManagedConnection.transactionFlushed(AbstractManagedConnection.java:89)         在org.datanucleus.store.connection.ConnectionManagerImpl $ 2.transactionFlushed(ConnectionManagerImpl.java:450)         在org.datanucleus.TransactionImpl.flush(TransactionImpl.java:210)         在org.datanucleus.TransactionImpl.commit(TransactionImpl.java:274)         在org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:107)         在org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:765)         在org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:953)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)         com.sun.proxy。$ Proxy22.createDatabase(未知来源)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultDB_core(HiveMetaStore.java:751)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.createDefaultDB(HiveMetaStore.java:769)         在org.apache.hadoop.hive.metastore.HiveMetaStore $ HMSHandler.init(HiveMetaStore.java:538)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler。(RetryingHMSHandler.java:80)         在org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)         在org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8661)         在org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8656)         在org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8926)         在org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8843)         在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处         在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         在java.lang.reflect.Method.invoke(Method.java:498)         在org.apache.hadoop.util.RunJar.run(RunJar.java:221)         在org.apache.hadoop.util.RunJar.main(RunJar.java:136)

1 个答案:

答案 0 :(得分:1)

https://issues.apache.org/jira/browse/HIVE-19740 在hive-site.xml上将hive.metastore.event.db.notification.api.auth设置为false之后。蜂巢可以启动并正常运行。