Question

每当我在PIG grunt shell中触发LOAD命令后跟STORE命令时。一旦MapReduce阶段在PIG中启动，我的系统就会崩溃。

在MapReduce模式下启动PIG grunt shell之前，我已经触发了以下命令。

start-dfs.sh  
start-yarn.sh  
mr-jobhistory-daemon.sh start historyserver

以下是上述操作的日志：

16/08/11 10:56:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
Starting namenodes on [localhost]  
localhost: starting namenode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-namenode-devesh-ThinkPad-L412.out  
localhost: starting datanode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-datanode-devesh-ThinkPad-L412.out  
Starting secondary namenodes [0.0.0.0]  
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop/logs/hadoop-hadoop-secondarynamenode-devesh-ThinkPad-L412.out  
16/08/11 10:56:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
starting yarn daemons  
starting resourcemanager, logging to /home/hadoop/hadoop/logs/yarn-hadoop-resourcemanager-devesh-ThinkPad-L412.out  
localhost: starting nodemanager, logging to /home/hadoop/hadoop/logs/yarn-hadoop-nodemanager-devesh-ThinkPad-L412.out  
starting historyserver, logging to /home/hadoop/hadoop/logs/mapred-hadoop-historyserver-devesh-ThinkPad-L412.out

在此之后我开始PIG grunt shell。以下是PIG grunt shell ::

的启动日志

16/08/11 10:58:07 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL  
16/08/11 10:58:07 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE  
16/08/11 10:58:07 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType  
2016-08-11 10:58:07,102 [main] INFO  org.apache.pig.Main - Apache Pig version 0.16.0 (r1746530) compiled Jun 01 2016, 23:10:49  
2016-08-11 10:58:07,103 [main] INFO  org.apache.pig.Main - Logging error messages to: /home/hadoop/pig_1470893287101.log  
2016-08-11 10:58:07,145 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found  
2016-08-11 10:58:07,845 [main] WARN  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
2016-08-11 10:58:07,876 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address  
2016-08-11 10:58:07,876 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS  
2016-08-11 10:58:07,876 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000  
2016-08-11 10:58:08,427 [main] INFO  org.apache.pig.PigServer - Pig Script ID for the session: PIG-default-a50ec406-53e1-42d7-bb8c-ff49c88e9118  
2016-08-11 10:58:08,427 [main] WARN  org.apache.pig.PigServer - ATS is disabled since yarn.timeline-service.enabled set to false

在此之后，我在grunt shell中触发以下命令。

data = LOAD '/pig/data.txt' USING PigStorage(',');  
STORE data INTO '/pig/dataout.txt' USING PigStorage(',');

此后，当Pig启动MapReduce阶段并重新启动系统时，我的系统崩溃。附：我在伪分布式模式下运行hadoop。

我现在正在使用带有4 GB RAM和Intel i5处理器的Lenovo Thinkpad L412。

操作系统详细信息：
Ubuntu 16.04 64位版本
内核版本：4.4.0-31-generic

Hadoop版本

Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41
由jenkins编制于2016-01-26T00：08Z
用protoc 2.5.0编译来自校验和的来源d0fda26633fa762bff87ec759ebe689c
此命令使用/home/hadoop/hadoop/share/hadoop/common/hadoop-common-2.7.2.jar

运行

PIG版本详细信息

Apache Pig版本0.16.0（r1746530）
编辑2016年6月1日，23：10：49

先谢谢。

在PIG grunt shell中运行存储操作时，Ubuntu 16.04崩溃

0 个答案: