我想使用PIG join 3表然后加载到Hive表中。在存储数据之前,我可以使用dump在屏幕上显示数据。我使用下面的命令来存储数据,它失败了。
store jnr_mas_ins INTO 'table_name' USING org.apache.hive.hcatalog.pig.HCatStorer();
错误是:
2017-01-25 10:58:48,487 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1002: Unable to store alias jnr_mas_ins
Details at logfile: /export/home/pig_1485359450881.log
我搜索了这个网站,有人建议使用以下命令:
store jnr_mas_ins INTO 'hdfs://fullpath' USING org.apache.hive.hcatalog.pig.HCatStorer();
001 : Error setting output information. Cause : NoSuchObjectException(message:default.hdfs://fullpath table not found)
Details at logfile: /export/home/pig_1485359450881.log
请您帮忙解决这个问题,提前谢谢 下面是我用来填充表格的脚本。
account_master_ALL = LOAD' ACCT'使用org.apache.hive.hcatalog.pig.HCatLoader(); account_master_fil =过滤帐号_master_ALL by croft_source_sys_cd =='' ABC'' account_master = foreach account_master_fil生成acct_master_key,(trans_date为空?' -999':trans_date)为trans_date,(acct_data_1为空?' -999':acct_data_1)为acct_data_1,(acct_data_2为null?' -999':acct_data_2)作为acct_data_2,(acct_data_3为空?' -999':acct_data_3)为acct_data_3,(acct_data_4为空?' -999&#39 ;:acct_data_4)作为acct_data_4,(acct_data_5为空?' -999':acct_data_5)为acct_data_5,acct_m_ref_key,croft_source_sys_cd; acct_detail_all = LOAD' acct_detail_tb'使用org.apache.hive.hcatalog.pig.HCatLoader(); acct_detail = foreach acct_detail_all生成c_number,b_number,a_number,c_coded,a_type,c_digit,s_number,r_type,r_cd,s_qty,pipes_cost,trade_qty,t_cost,p_date,load_date; jnr_all =通过(a_number,b_number,c_coded,a_type,c_number,c_digit),account_master通过(acct_data_1,trans_date,acct_data_2,acct_data_3,acct_data_4,acct_data_5)加入acct_detail; ins_master_all = LOAD' ins_acct'使用org.apache.hive.hcatalog.pig.HCatLoader(); jnr_mas_ins_all =通过(acct_detail :: s_number)加入jnr_all,通过(acct_data_1)加入ins_master_all; jnr_mas_ins = foreach jnr_mas_ins_all生成acct_master_key为acct_master_key,ins_master_key为ins_master_key,' ABC'作为code_dev,c_number作为c_number,b_number作为b_number,a_number作为src_a_number,c_coded作为c_coded,a_type作为c_digit,c_digit作为c_digit,s_number作为s_number,r_type作为r_type,r_cd作为r_cd,s_qty作为s_qty,pipes_cost作为pipes_cost,trade_qty作为trade_qty ,t_cost为t_cost,p_date为p_date,CurrentTime()为load_date,' PIG_USER' as create_userid,CurrentTime()as create_date,CurrentTime()as f_date; 存储jnr_mas_ins INTO' tartet_table'使用org.apache.hive.hcatalog.pig.HCatStorer();
答案 0 :(得分:0)
您似乎错过了关系jnr_mas_ins_all
。这来自哪里?jnr_mas_ins
将是emtpy,因为没有关系jnr_mas_ins_all
。它应该是ins_master_all
?
ins_master_all = LOAD 'ins_acct' USING org.apache.hive.hcatalog.pig.HCatLoader();
jnr_mas_ins = foreach jnr_mas_ins_all generate acct_master_key as acct_master_key, ins_master_key as ins_master_key, 'ABC' as code_dev, c_number as c_number, b_number as b_number, a_number as src_a_number, c_coded as c_coded, a_type as a_type, c_digit as c_digit, s_number as s_number, r_type as r_type, r_cd as r_cd, s_qty as s_qty, s_b_cost as s_b_cost, trade_qty as trade_qty, t_cost as t_cost, p_date as p_date, CurrentTime() as load_date, 'PIG_USER' as create_userid, CurrentTime() as create_date, CurrentTime() as f_date;