ERROR 1070 PIG toDate

时间:2017-04-19 08:23:52

标签: apache hadoop apache-pig

使用pig将数据投射到datetime时出现问题。导入以下数据集,制表符分隔,我添加了\ t来说明。

5000001 \ t 1133938 \ t 1273719 \ t 2008-06-01 00:03:35.0

我想取最后一个值(2008-06-01 00:03:35.0)并提取月份。将其作为DateTime加载会生成一个空白字段。所以我想将它作为chararray导入,然后使用toDate将其转换为dateTime字段,我可以使用getMonth。

当我尝试使用shell时,我得到了这个:

grunt> orders = load '/home/cloudera/Desktop/orders1' using PigStorage('\t') AS (orderID:int, cusID:int, productId:chararray, time:chararray);
grunt> to_date = FOREACH orders GENERATE toDate(time,'yyyy-MM-dd HH:mm:ss.S') as (dTime:DateTime), cusID;
2017-04-19 01:13:46,206 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve toDate using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /home/cloudera/pig_1492588751037.log

日志文件包含:

  

错误1070:无法使用导入解析toDate:[,java.lang。,   org.apache.pig.builtin。,org.apache.pig.impl.builtin。]

     

无法解析:Pig脚本无法解析:   无法生成逻辑计划。嵌套异常:   org.apache.pig.backend.executionengine.ExecException:ERROR 1070:   无法使用导入解析toDate:[,java.lang。,   org.apache.pig.builtin。,org.apache.pig.impl.builtin。] at   org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)     在org.apache.pig.PigServer $ Graph.validateQuery(PigServer.java:1660)     在org.apache.pig.PigServer $ Graph.registerQuery(PigServer.java:1633)     在org.apache.pig.PigServer.registerQuery(PigServer.java:587)处   org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)     在   org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)     在   org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)     在   org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)     在org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)at   org.apache.pig.Main.run(Main.java:547)at   org.apache.pig.Main.main(Main.java:158)at   sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at   sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)     在   sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     在java.lang.reflect.Method.invoke(Method.java:606)at   org.apache.hadoop.util.RunJar.run(RunJar.java:221)at   org.apache.hadoop.util.RunJar.main(RunJar.java:136)引起:无法生成逻辑计划。嵌套异常:   org.apache.pig.backend.executionengine.ExecException:ERROR 1070:   无法使用导入解析toDate:[,java.lang。,   org.apache.pig.builtin。,org.apache.pig.impl.builtin。] at   org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:1507)     在   org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:9372)     在   org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:11051)     在   org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:10810)     在   org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:10159)     在   org.apache.pig.parser.LogicalPlanGenerator.flatten_generated_item(LogicalPlanGenerator.java:7488)     在   org.apache.pig.parser.LogicalPlanGenerator.generate_clause(LogicalPlanGenerator.java:17590)     在   org.apache.pig.parser.LogicalPlanGenerator.foreach_plan(LogicalPlanGenerator.java:15982)     在   org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:15849)     在   org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1933)     在   org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)     在   org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)     在   org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)     在   org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)     ... 16更多引起:   org.apache.pig.backend.executionengine.ExecException:ERROR 1070:   无法使用导入解析toDate:[,java.lang。,   org.apache.pig.builtin。,org.apache.pig.impl.builtin。] at   org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:653)     在   org.apache.pig.impl.PigContext.getClassForAlias(PigContext.java:769)     在   org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:1504)     ... 29更多

1 个答案:

答案 0 :(得分:1)

Pig函数名称区分大小写。它是ToDate()

grunt> to_date = FOREACH orders GENERATE ToDate(time,'yyyy-MM-dd HH:mm:ss.S') as (dTime:DateTime), cusID;