Question

我正在尝试使用jaydebeapi python lib连接到配置单元服务器，我收到错误。我没有JAVA的经验。可能是什么问题，或者我该如何调试它？

./beeline-jdbc.py 

CLASSPATH is :/usr/hdp/2.4.3.0-227/hadoop/conf:/usr/hdp/2.4.3.0-227/hadoop/lib/*:/usr/hdp/2.4.3.0-227/hadoop/.//*:/usr/hdp/2.4.3.0-227/hadoop-hdfs/./:/usr/hdp/2.4.3.0-227/hadoop-hdfs/lib/*:/usr/hdp/2.4.3.0-227/hadoop-hdfs/.//*:/usr/hdp/2.4.3.0-227/hadoop-yarn/lib/*:/usr/hdp/2.4.3.0-227/hadoop-yarn/.//*:/usr/hdp/2.4.3.0-227/hadoop-mapreduce/lib/*:/usr/hdp/2.4.3.0-227/hadoop-mapreduce/.//*::mysql-connector-java-5.1.37:mysql-connector-java-5.1.37-bin.jar:mysql-connector-java-5.1.37.zip:mysql-connector-java.jar:/usr/hdp/2.4.3.0-227/tez/*:/usr/hdp/2.4.3.0-227/tez/lib/*:/usr/hdp/2.4.3.0-227/tez/conf:/usr/hdp/current/hadoop-client/*:/usr/hdp/current/hive-client/lib/*:/usr/hdp/current/hadoop-client/client/*

Traceback (most recent call last):
  File "./beeline-jdbc.py", line 18, in <module>
    , hivejar)
  File "/home/dev/lib/JayDeBeApi-1.0.0-py2.6.egg/jaydebeapi/__init__.py", line 380, in connect
    jconn = _jdbc_connect(jclassname, url, driver_args, jars, libs)
  File "/home/dev/lib/JayDeBeApi-1.0.0-py2.6.egg/jaydebeapi/__init__.py", line 199, in _jdbc_connect_jpype
    return jpype.java.sql.DriverManager.getConnection(url, *dargs)
RuntimeError: No matching overloads found. at native/common/jp_method.cpp:117

脚本：

#!/usr/bin/env python

import jaydebeapi
import os

principal='hive/dev15-namenode-01.example.com@example.COM'
server='localhost'
port=10000
database='tsz'
hivejar='/usr/hdp/current/hive-client/lib/hive-jdbc.jar'

url=("jdbc:hive2://" + server + ":" + str(port) + "/default;principal="
    + principal + ";")

print 'CLASSPATH is '+os.environ['CLASSPATH']

c=jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver", [ url, '', '']
    , hivejar)

cursor=c.cursor()

Answer 1

...最后，我终于明白了。但是我认为使用此组件连接配置单元不是一个好主意，它依赖于大量的jar包，并且您必须使用python-java debug REPL安装jar包（大多数情况下您是在猜测），这很糟糕。

# -*- coding: utf8 -*-

import jaydebeapi
import os


class Jdbc:
    base_sql = [
        "set hive.mapred.mode=nonstrict",
        "set hive.strict.checks.cartesian.product=false",
        "set hive.execution.engine=tez"
    ]

    def query(self, sql, db = 'tap_ods'):  
        url = 'jdbc:hive2://60.205.213.178:10000/' + db
        dirver = 'org.apache.hive.jdbc.HiveDriver'
        DIR = os.getcwd() + '/lib/'
        jarFile = [
            DIR + 'hive-jdbc-3.1.1.jar',
            DIR + 'commons-logging-1.2.jar',
            DIR + 'hive-service-3.1.1.jar',
            DIR + 'hive-service-rpc-3.1.1.jar',
            DIR + 'libthrift-0.12.0.jar',
            DIR + 'httpclient-4.5.9.jar',
            DIR + 'httpcore-4.4.11.jar',
            DIR + 'slf4j-api-1.7.26.jar',
            DIR + 'curator-framework-4.2.0.jar',
            DIR + 'curator-recipes-4.2.0.jar',
            DIR + 'curator-client-4.2.0.jar',
            DIR + 'commons-lang-2.6.jar',
            DIR + 'hadoop-common-3.2.0.jar',
            DIR + 'httpcore-4.4.11.jar',
            DIR + 'hive-common-3.1.1.jar',
            DIR + 'hive-serde-3.1.1.jar',
            DIR + 'guava-28.0-jre.jar'
        ]
        conn = jaydebeapi.connect(dirver, url, ['hadoop', ''], jarFile)
        curs = conn.cursor()
        for _sql in self.base_sql:
            curs.execute(_sql)
        curs.execute(sql)
        result = curs.fetchall()
        curs.close()
        conn.close()
        return result

注意：

1.JPype1版本使用0.6.3-> pip3 install JPype1 == 0.6.3

2.pip3安装Jaydebeapi == 1.1.1

3。确保hiv2或配置单元？

使用代码，从https://mvnrepository.com/下载jar包

使用python的Jaydebeapi

1 个答案: