Question

我使用下面的命令将HBase表导出到HDFS中。

 hbase org.apache.hadoop.hbase.mapreduce.Driver export "Table-name" "hdfs-path"

此命令可以很好地执行小型表。但是无法导出大型数据表。

错误日志：

2015-09-22 14:48:58,814 INFO  [main] mapreduce.Job: Task Id :     attempt_1442911480092_0002_m_000000_2, Status : FAILED
Container [pid=3575,containerID=container_1442911480092_0002_01_000004] is     running beyond virtual memory limits. Current usage: 23.6 MB of 1 GB physical     memory used; 4.8 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1442911480092_0002_01_000004 :
    |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
    |- 3575 3573 3575 3575 (bash) 0 0 108609536 334 /bin/bash -c /opt/java/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN  -Xmx3024m -Djava.io.tmpdir=/opt/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1442911480092_0002/container_1442911480092_0002_01_000004/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/hadoop/logs/userlogs/application_1442911480092_0002/container_1442911480092_0002_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 10.127.128.149 44859 attempt_1442911480092_0002_m_000000_2 4 1>/opt/hadoop/logs/userlogs/application_1442911480092_0002/container_1442911480092_0002_01_000004/stdout 2>/opt/hadoop/logs/userlogs/application_1442911480092_0002/container_1442911480092_0002_01_000004/stderr
    |- 3591 3575 3575 3575 (java) 10 2 5018701824 5704 /opt/java/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx3024m -Djava.io.tmpdir=/opt/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1442911480092_0002/container_1442911480092_0002_01_000004/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/hadoop/logs/userlogs/application_1442911480092_0002/container_1442911480092_0002_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 10.127.128.149 44859 attempt_1442911480092_0002_m_000000_2 4

Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

2015-09-22 14:49:07,892 INFO  [main] mapreduce.Job:  map 100% reduce 0%
2015-09-22 14:49:07,911 INFO  [main] mapreduce.Job: Job     job_1442911480092_0002 failed with state FAILED due to: Task failed task_1442911480092_0002_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

2015-09-22 14:49:08,105 INFO  [main] mapreduce.Job: Counters: 12
    Job Counters
            Failed map tasks=4
            Launched map tasks=4
            Other local map tasks=3
            Rack-local map tasks=1
            Total time spent by all maps in occupied slots (ms)=13484
            Total time spent by all reduces in occupied slots (ms)=0
            Total time spent by all map tasks (ms)=13484
            Total vcore-seconds taken by all map tasks=13484
            Total megabyte-seconds taken by all map tasks=13807616
    Map-Reduce Framework
            CPU time spent (ms)=0
            Physical memory (bytes) snapshot=0
            Virtual memory (bytes) snapshot=0

Hbase导出问题

0 个答案: