Question

我正在尝试使用-f usecatalog选项运行pig脚本，但它给了我一个问题。它说脚本不存在，而我可以看到该文件存在于hdfs文件系统中。见下文。

[hdfs@ip-xx-xx-xx-x-xx ec2-user]$ pig -useHCatalog -f   /user/admin/pig/scripts/hcat1.pig  
    WARNING: Use "yarn jar" to launch YARN applications.  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType  
    2016-04-01 13:44:13,645 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 20                     15, 04:30:33  
    2016-04-01 13:44:13,645 [main] INFO  org.apache.pig.Main - Logging error messages to: /tmp/hsperfdata_hdfs/pig_1459532653643.log  
    2016-04-01 13:44:14,184 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File /user/admin/pig/scripts/hca                     t1.pig does not exist  
    Details at logfile: /tmp/hsperfdata_hdfs/pig_1459532653643.log  
    2016-04-01 13:44:14,203 [main] INFO  org.apache.pig.Main - Pig script completed in 753 milliseconds (753 ms)

[hdfs@ip-xxx-xx-xx-xx ec2-user]$ hadoop fs -cat /user/admin/pig/scripts/hcat1.pig  
    a = load 'trucks' using org.apache.hive.hcatalog.pig.HCatLoader();  
    b = filter a by truckid == 'A1';  
    store b INTO '/user/admin/pig/scritps/outputb1';

Answer 1

您需要指定完整的HDFS URI以运行存储在HDFS中的脚本。

以下是您的需求：

$pig -useHCatalog hdfs://namenode_hostname:port/user/admin/pig/scripts/hcat1.pig

猪脚本不存在错误，即使我可以在hdfs中看到它

1 个答案: