Question

我只是想尝试运行一个示例hivejdbc客户端程序，但它给了我一个内存不足的错误。

import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;

public class HiveJdbcClient {
  private static String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";

  /**
 * @param args
 * @throws SQLException
   */
  public static void main(String[] args) throws SQLException {
      try {
      Class.forName(driverName);
    } catch (ClassNotFoundException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
      System.exit(1);
    }
    // "jdbc:hive://host:port/dbname"
    System.out.println("going to get connection");
    Connection con = DriverManager.getConnection("jdbc:hive://titan-edgenode.int.thomsonreuters.com:22/default", "", "");
    System.out.println("got connection!");
    Statement stmt = con.createStatement();
    ResultSet res;
    // show tables
    String sql = "show tables";
    System.out.println("Running: " + sql);
    res = stmt.executeQuery(sql);
    if (res.next()) {
      System.out.println(res.getString(1));
    }

  }
}

这是我得到的输出。

going to get connection
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
    at org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:116)
    at org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:103)
    at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
    at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
    at org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:133)
    at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:122)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:106)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at HiveJdbcClient.main(HiveJdbcClient.java:24)

错误的有趣之处在于我在获得连接后耗尽了堆空间。这是不寻常的，因为获取连接本身不需要太多空间，因为我没有执行查询或类似的事情。

修改

以下是来自另一个用户的此堆栈的另一个变体（更改最大堆大小没有显着差异）：

java.lang.OutOfMemoryError: Java heap space
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:181)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:288)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at [myapp]

Answer 1

您可以增加堆空间。使用命令行参数-Xms -Xmx。如果这不能解决问题，那么驱动程序可能会出错。

Answer 2

我可以通过转移到x64应用程序来解决此错误。无论我如何配置java的最大堆大小，Win32似乎都没有足够的内存。

hive jdbc连接给出内存不足错误

2 个答案: