显然,hive.exec.parallel
默认为false
是有原因的。
当我将其设置为true
时(根据my previous question的回答建议),我的流程会因此消息而死:
MapReduce Jobs Launched:
Job 0: Map: 2 Reduce: 1 Cumulative CPU: 6.43 sec HDFS Read: 556 HDFS Write: 96 SUCCESS
Job 1: Map: 1 Reduce: 1 Cumulative CPU: 3.15 sec HDFS Read: 475 HDFS Write: 96 SUCCESS
Job 2: Map: 1 Reduce: 1 Cumulative CPU: 3.36 sec HDFS Read: 475 HDFS Write: 96 SUCCESS
Job 3: Map: 1 Reduce: 1 Cumulative CPU: 2.19 sec HDFS Read: 475 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 15 seconds 130 msec
OK
normalized_keyword pixel_id count sum_log events
Time taken: 72.419 seconds
...
14.98user 0.62system 1:16.79elapsed 20%CPU (0avgtext+0avgdata 851392maxresident)k
8inputs+2096outputs (0major+83271minor)pagefaults 0swaps
text: java.io.EOFException
at java.io.DataInputStream.readShort(Unknown Source)
at org.apache.hadoop.fs.shell.Display$Text.getInputStream(Display.java:113)
at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:81)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:306)
at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:278)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:260)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:244)
at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:254)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:304)
没有产生有用的数据。 set hive.exec.parallel.thread.number=2
无效(同样失败)
建议?
编辑:hive --version
不起作用,但在启动时会打印
Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hive/lib/hive-common-0.10.0-cdh4.4.0.jar!/hive-log4j.properties
所以,我想,版本是0.10.0
。