hive.exec.parallel坏了吗?

时间:2014-01-08 15:26:40

标签: hadoop hive

显然,hive.exec.parallel默认为false是有原因的。 当我将其设置为true时(根据my previous question的回答建议),我的流程会因此消息而死:

MapReduce Jobs Launched:
Job 0: Map: 2  Reduce: 1   Cumulative CPU: 6.43 sec   HDFS Read: 556 HDFS Write: 96 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.15 sec   HDFS Read: 475 HDFS Write: 96 SUCCESS
Job 2: Map: 1  Reduce: 1   Cumulative CPU: 3.36 sec   HDFS Read: 475 HDFS Write: 96 SUCCESS
Job 3: Map: 1  Reduce: 1   Cumulative CPU: 2.19 sec   HDFS Read: 475 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 15 seconds 130 msec
OK
normalized_keyword      pixel_id        count   sum_log events
Time taken: 72.419 seconds

...
14.98user 0.62system 1:16.79elapsed 20%CPU (0avgtext+0avgdata 851392maxresident)k
8inputs+2096outputs (0major+83271minor)pagefaults 0swaps
text: java.io.EOFException
        at java.io.DataInputStream.readShort(Unknown Source)
        at org.apache.hadoop.fs.shell.Display$Text.getInputStream(Display.java:113)
        at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:81)
        at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:306)
        at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:278)
        at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:260)
        at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:244)
        at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
        at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:254)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:304)

没有产生有用的数据。 set hive.exec.parallel.thread.number=2无效(同样失败)

建议?

编辑:hive --version不起作用,但在启动时会打印

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hive/lib/hive-common-0.10.0-cdh4.4.0.jar!/hive-log4j.properties

所以,我想,版本是0.10.0

0 个答案:

没有答案