Question

我在Hortonworks Hive工作。我见过同样的错误。但是，在使用exitCode 1的应用程序错误的情况下，基础MapReduce错误似乎有所不同。

在Hive中，声明

Select * from SomeTable;

......工作正常，但

Select colName from SomeTable;

......不工作。

应用程序错误日志

2014-03-17 12:49:15,557 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1395039411618_0001 State change from ACCEPTED to FAILED
2014-03-17 12:49:15,558 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application appattempt_1395039411618_0001_000002 is done. finalState=FAILED
2014-03-17 12:49:15,559 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: Application application_1395039411618_0001 requests cleared
2014-03-17 12:49:15,559 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Application removed - appId: application_1395039411618_0001 user: asande queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0
2014-03-17 12:49:15,559 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1395039411618_0001 user: asande leaf-queue of parent: root #applications: 0
2014-03-17 12:49:15,559 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=asande   OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED   PERMISSIONS=Application application_1395039411618_0001 failed 2 times due to AM Container for appattempt_1395039411618_0001_000002 exited with  exitCode: 1 due to: Exception from container-launch: 
org.apache.hadoop.util.Shell$ExitCodeException:

这是Hive.log。（但似乎日志中没有任何错误。）

<code>
2014-03-17 10:45:37,322 INFO  server.HiveServer2 (HiveStringUtils.java:startupShutdownMessage(604)) - STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting HiveServer2
STARTUP_MSG:   host = ASANDE1/16.155.82.203
STARTUP_MSG:   args = [-hiveconf, hive.hadoop.classpath=c:\hdp\hive-0.12.0.2.0.6.0-0009\lib\*.............................................., -hiveconf, hive.querylog.location=c:\hadoop\logs\hive\history, -hiveconf, hive.log.dir=c:\hadoop\logs\hive]
STARTUP_MSG:   version = 0.12.0.2.0.6.0-0009
STARTUP_MSG:   classpath = c:\hdp\hadoop-2.2.0.2.0.6.0-0009\etc\hadoop;c:\hdp\hadoop-;;
STARTUP_MSG:   build = git://sijenkins-vm3/cygdrive/d/w/bw/project/hive-monarch -r a7f54db5645b645500778b92e7fad8fab7738080; compiled by 'jenkins' on Fri Dec 20 18:29:58 PST 2013
************************************************************/
2014-03-17 10:45:39,622 INFO  service.CompositeService (SessionManager.java:init(60)) - HiveServer2: Async execution thread pool size: 100
2014-03-17 10:45:39,622 INFO  service.CompositeService (SessionManager.java:init(62)) - HiveServer2: Async execution wait queue size: 100
2014-03-17 10:45:39,623 INFO  service.CompositeService (SessionManager.java:init(64)) - HiveServer2: Async execution thread keepalive time: 10
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:OperationManager is inited.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:SessionManager is inited.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:CLIService is inited.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:ThriftBinaryCLIService is inited.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:HiveServer2 is inited.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:OperationManager is started.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:SessionManager is started.
2014-03-17 10:45:39,628 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:CLIService is started.
2014-03-17 10:45:39,925 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://ASANDE1:9083
2014-03-17 10:46:02,239 WARN  hive.metastore (HiveMetaStoreClient.java:open(307)) - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:2822)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:2808)

Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
    ... 14 more
2014-03-17 10:46:02,280 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
2014-03-17 10:46:03,280 INFO  hive.metastore (HiveMetaStoreClient.java:open(332)) - Connected to metastore.
2014-03-17 10:46:08,455 ERROR hive.log (MetaStoreUtils.java:logAndThrowMetaException(960)) - Got exception: org.apache.thrift.TApplicationException get_databases failed: out of sequence response
org.apache.thrift.TApplicationException: get_databases failed: out of sequence response
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:500)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:487)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:722)
    at org.apache.hive.service.cli.CLIService.start(CLIService.java:83)
    at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
    at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:73)
    at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:103)
2014-03-17 10:46:08,455 ERROR hive.log (MetaStoreUtils.java:logAndThrowMetaException(961)) - Converting exception to MetaException
2014-03-17 10:46:08,621 ERROR service.CompositeService (CompositeService.java:start(74)) - Error starting services HiveServer2
org.apache.hive.service.ServiceException: Unable to connect to MetaStore!
    at org.apache.hive.service.cli.CLIService.start(CLIService.java:85)
    ..........................
Caused by: MetaException(message:Got exception: org.apache.thrift.TApplicationException get_databases failed: out of sequence response)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:962)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:724)
    at org.apache.hive.service.cli.CLIService.start(CLIService.java:83)
    ... 3 more
2014-03-17 10:46:08,627 INFO  service.AbstractService (AbstractService.java:stop(125)) - Service:OperationManager is stopped.
2014-03-17 10:46:08,627 INFO  service.AbstractService (AbstractService.java:stop(125)) - Service:SessionManager is stopped.
2014-03-17 10:46:08,627 INFO  service.AbstractService (AbstractService.java:stop(125)) - Service:CLIService is stopped.
2014-03-17 10:46:08,627 FATAL server.HiveServer2 (HiveServer2.java:main(105)) - Error starting HiveServer2
org.apache.hive.service.ServiceException: Failed to Start HiveServer2
    at org.apache.hive.service.CompositeService.start(CompositeService.java:80)
    at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:73)
    at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:103)
Caused by: org.apache.hive.service.ServiceException: Unable to connect to MetaStore!
    at org.apache.hive.service.cli.CLIService.start(CLIService.java:85)
    at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
    ... 2 more
Caused by: MetaException(message:Got exception: org.apache.thrift.TApplicationException get_databases failed: out of sequence response)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:962)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:724)
    at org.apache.hive.service.cli.CLIService.start(CLIService.java:83)
    ... 3 more
2014-03-17 10:46:08,791 INFO  server.HiveServer2 (HiveStringUtils.java:run(622)) - SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down HiveServer2 at ASANDE1/16.155.82.203
************************************************************/
2014-03-17 10:46:22,805 INFO  server.HiveServer2 (HiveStringUtils.java:startupShutdownMessage(604)) - STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting HiveServer2
STARTUP_MSG:   host = ASANDE1/16.155.82.203
STARTUP_MSG:   args = [-hiveconf, hive.hadoop.classpath=c:\hdp\hive-0.12.0.2.0.6.0-0009\lib\*, -hiveconf, hive.querylog.location=c:\hadoop\logs\hive\history, -hiveconf, hive.log.dir=c:\hadoop\logs\hive]
STARTUP_MSG:   version = 0.12.0.2.0.6.0-0009
STARTUP_MSG:   classpath = c:\hdp\hadoop-2.2.0.2.0.6.0-0009\etc\hadoop;c:\hdp\hadoop-2.2.0.2.0.6.0-0009\share\hadoop\common\lib\activation-1.1.jar;c:\hdp\hadoop-2.2.0.2.0.6.0-
...............................................................
STARTUP_MSG:   build = git://sijenkins-vm3/cygdrive/d/w/bw/project/hive-monarch -r a7f54db5645b645500778b92e7fad8fab7738080; compiled by 'jenkins' on Fri Dec 20 18:29:58 PST 2013
************************************************************/
2014-03-17 10:46:23,677 INFO  service.CompositeService (SessionManager.java:init(60)) - HiveServer2: Async execution thread pool size: 100
2014-03-17 10:46:23,677 INFO  service.CompositeService (SessionManager.java:init(62)) - HiveServer2: Async execution wait queue size: 100
2014-03-17 10:46:23,678 INFO  service.CompositeService (SessionManager.java:init(64)) - HiveServer2: Async execution thread keepalive time: 10
2014-03-17 10:46:23,682 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:OperationManager is inited.
2014-03-17 10:46:23,682 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:SessionManager is inited.
2014-03-17 10:46:23,682 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:CLIService is inited.
2014-03-17 10:46:23,683 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:ThriftBinaryCLIService is inited.
2014-03-17 10:46:23,683 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:HiveServer2 is inited.
2014-03-17 10:46:23,683 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:OperationManager is started.
2014-03-17 10:46:23,683 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:SessionManager is started.
2014-03-17 10:46:23,683 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:CLIService is started.
2014-03-17 10:46:23,694 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://ASANDE1:9083
2014-03-17 10:46:24,093 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
2014-03-17 10:46:25,093 INFO  hive.metastore (HiveMetaStoreClient.java:open(332)) - Connected to metastore.
2014-03-17 10:46:25,118 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:ThriftBinaryCLIService is started.
2014-03-17 10:46:25,122 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:HiveServer2 is started.
2014-03-17 10:46:25,351 INFO  thrift.ThriftCLIService (ThriftBinaryCLIService.java:run(76)) - ThriftBinaryCLIService listening on 0.0.0.0/0.0.0.0:10001
2014-03-17 12:27:04,409 INFO  server.HiveServer2 (HiveStringUtils.java:startupShutdownMessage(604)) - STARTUP_MSG: 
/************************************************************</code>

Answer 1

适合您的查询不会启动Map Reduce作业。启动Map Reduce Jobs时看起来有问题。您可以检查配置单据日志（默认位置，假设您正在运行hiveserver2作为用户配置单元） - /tmp/hive/hive.log以查看它是否包含完整的错误消息。

您可能在日志中发现的一个常见错误是权限被拒绝错误。大多数情况下，ODBC驱动程序以hadoop用户身份登录，该用户没有写访问权限，从而导致启动Map Reduce Job失败。如果将此用户更改为hdfs，则可能会解决您的问题。

Answer 2

您运行的Select * from SomeTable;和Select colName from SomeTable;查询之间的区别在于，后者Select colName from SomeTable;需要来自 Hive Metastore 服务的信息。由于您要查找特定列，因此hive必须首先检查存储在 Hive Metastore 中的hive架构定义。

您的问题是由于您无法连接到Hive Metastore，如日志文件中的以下消息所示：

org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out

尝试重新启动 Hive Metastore 服务。

使用Application exitCode 1选择语句错误

2 个答案: