我们将Hive 2.3.3与Hadoop 2.7.7和Spark 2.4.4一起使用。
我们将MariaDB用作Metastore的后端数据库。
Metastore服务的启动很好,我能够访问Hive CLI并执行查询操作。我没有启动HS2(试图找出问题所在)。
但是过了一段时间,Hive Metastore服务突然停滞了,甚至对简单的查询show databases
或show tables
也没有响应。
我们要对其执行其他查询的所有表都是空的(到目前为止,尚未创建任何分区),并且正在重新创建此环境。
hive.log错误文件:
2020-10-09T18:43:56,971 DEBUG [IPC Client (1223050066) connection to master/10.28.66.65:8020 from hadoopuser] ipc.Client: IPC Client (1223050066) connection to master/10.28.66.65:8020 from hadoopuser: closed
2020-10-09T18:43:56,971 DEBUG [IPC Client (1223050066) connection to master/10.28.66.65:8020 from hadoopuser] ipc.Client: IPC Client (1223050066) connection to master/10.28.66.65:8020 from hadoopuser: stopped, remaining connections 0
2020-10-09T18:53:53,769 WARN [9edaa669-999e-41a9-a7b3-bcee9d6198f4 main] metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect (1 of 1) after 5s. getAllFunctions
org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_all_functions(ThriftHiveMetastore.java:3812) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_all_functions(ThriftHiveMetastore.java:3800) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllFunctions(HiveMetaStoreClient.java:2393) ~[hive-exec-2.3.3.jar:2.3.3]
....
2020-10-09T18:53:58,777 INFO [9edaa669-999e-41a9-a7b3-bcee9d6198f4 main] hive.metastore: Closed a connection to metastore, current connections: 0
2020-10-09T18:53:58,777 INFO [9edaa669-999e-41a9-a7b3-bcee9d6198f4 main] hive.metastore: Trying to connect to metastore with URI thrift://master:9083
2020-10-09T18:53:58,779 INFO [9edaa669-999e-41a9-a7b3-bcee9d6198f4 main] hive.metastore: Opened a connection to metastore, current connections: 1
2020-10-09T18:53:58,805 INFO [9edaa669-999e-41a9-a7b3-bcee9d6198f4 main] hive.metastore: Connected to metastore.