错误无法使用pyhive连接到Hive数据库

时间:2018-02-14 13:24:48

标签: python hive cursor pyhive

这是用于连接到我们的hive数据库的代码,它在一周后运行良好,但现在似乎甚至无法打开会话并获得游标来执行查询。当我明确添加了cursor.close()方法时,问题暂时得到解决,但现在它又回来了。我无法使用python

访问配置单元数据库

我尝试过使用pyhs2和pyhive,这两个库都无法连接到hive数据库。到目前为止,集群上没有任何变化。这可能是什么原因?

我知道Hive不是关系数据库所以游标的概念没有意义但是Hive数据库有没有办法记住使用pyhive库创建的游标?如果是这样,我怎么能删除当前未使用的游标?

以下是执行时引发的代码和异常

from pyhive import hive
import contextlib

class Hive():
    def __init__(self,host="[hostnamehere]",db="default",port="10000",auth="KERBEROS",kerberos_service_name="hive"):
        self.host = host
        self.db = db
        self.port = port
        self.auth = auth
        self.kerberos_service_name = kerberos_service_name

    def connect(self):
        return(hive.connect(host=self.host, port=self.port, database=self.db, auth=self.auth, kerberos_service_name=self.kerberos_service_name))

    def query_one(self,sql):
        with contextlib.closing(self.connect()) as connection:
            with contextlib.closing(connection.cursor()) as cursor:
                cursor.execute(sql)
                result = cursor.fetch_one()
                cursor.close()
        return(result)


if __name__ == "__main__":
    connector = Hive()
    print("running query")
    print(connector.query_one("SELECT * FROM [tablenamehere]"))

    提出OperationalError(响应) pyhive.exc.OperationalError:TOpenSessionResp(status = TStatus(statusCode = 3,infoMessages = [' * org.apache.hive.service.cli.HiveSQLException:无法打开新会话:java.lang.RuntimeException:java。 lang.RuntimeException:无法实例化org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:13:12',' org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager。 java:289',' org.apache.hive.service.cli.CLIService:openSession:CLIService.java:199',' org.apache.hive.service.cli.thrift。 ThriftCLIService:getSessionHandle:ThriftCLIService.java:427',' org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:319',' org.apache。 hive.service.cli.thrift.TCLIService $ Processor $ OpenSession:getResult:TCLIService.java:1257',' org.apache.hive.service.cli.thrift.TCLIService $ Processor $ OpenSession:getResult:TCLIService .java:1242',' org.apache.thrift.ProcessFunction:process:ProcessFunction.java:3 9',' org.apache.thrift.TBaseProcessor:进程:TBaseProcessor.java:39',' org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge $ Server $ TUGIAssumingProcessor:进程: HadoopThriftAuthBridge.java:562' ;,' org.apache.thrift.server.TThreadPoolServer $ WorkerProcess:运行:TThreadPoolServer.java:286',' java.util.concurrent.ThreadPoolExecutor:runWorker: ThreadPoolExecutor.java:1149' ;,' java.util.concurrent.ThreadPoolExecutor $ Worker:run:ThreadPoolExecutor.java:624',' java.lang.Thread:run:Thread.java: 748',' * java.lang.RuntimeException:java.lang.RuntimeException:无法实例化org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:15:2',' org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:547',' org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:144& #39;,' org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:281',' * java.lang.RuntimeE xception:无法实例化org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient:21:6',' org.apache.hadoop.hive.metastore.MetaStoreUtils:newInstance:MetaStoreUtils.java:1566&# 39;,' org.apache.hadoop.hive.metastore.RetryingMetaStoreClient :: RetryingMetaStoreClient.java:92',' org.apache.hadoop.hive.metastore.RetryingMetaStoreClient:getProxy:RetryingMetaStoreClient.java :138',' org.apache.hadoop.hive.metastore.RetryingMetaStoreClient:getProxy:RetryingMetaStoreClient.java:110',' org.apache.hadoop.hive.ql.metadata.Hive :createMetaStoreClient:Hive.java:3510',' org.apache.hadoop.hive.ql.metadata.Hive:getMSC:Hive.java:3542',' org.apache.hadoop .hive.ql.session.SessionState:start:SessionState.java:528',' * java.lang.reflect.InvocationTargetException:null:25:4',' sun.reflect。 NativeConstructorAccessorImpl:newInstance0:NativeConstructorAccessorImpl.java:-2',' sun.reflect.NativeConstructorAccessorImpl:newInstance:Nat iveConstructorAccessorImpl.java:62',' sun.reflect.DelegatingConstructorAccessorImpl:newInstance:DelegatingConstructorAccessorImpl.java:45',' java.lang.reflect.Constructor:newInstance:Constructor.java:423&# 39;,' org.apache.hadoop.hive.metastore.MetaStoreUtils:newInstance:MetaStoreUtils.java:1564',' * org.apache.hadoop.hive.metastore.api.MetaException:GC超出限额:30:4',' org.apache.hadoop.hive.metastore.RetryingHMSHandler :: RetryingHMSHandler.java:82',' org.apache.hadoop.hive.metastore .RetryingHMSHandler:getProxy:RetryingHMSHandler.java:91',' org.apache.hadoop.hive.metastore.HiveMetaStore:newRetryingHMSHandler:HiveMetaStore.java:6463',' org.apache.hadoop .hive.metastore.HiveMetaStoreClient :: HiveMetaStoreClient.java:206',' org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient :: SessionHiveMetaStoreClient.java:76',' * java.lang.OutOfMemoryError:超出GC开销限制:0:-1'],sqlState = None, errorCode = 0,errorMessage ='无法打开新会话:java.lang.RuntimeException:java.lang.RuntimeException:无法实例化org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient'),serverProtocolVersion = 7,sessionHandle =无,配置=无)

0 个答案:

没有答案