错误2997:无法从支持的错误重新创建异常。 这里我已经解析了apache日志文件但是当我尝试将其导出为csv格式时会发生此错误。代码和错误:
grunt> STORE logs INTO '/home/cloudera/workspace/Test_log.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage(',', 'NO_MULTILINE','NOCHANGE');
2015-12-24 10:50:44,821 [主要] INFO org.apache.pig.tools.pigstats.ScriptState - 使用的Pig功能 脚本:UNKNOWN
2015-12-24 10:50:44,830 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - 文件级联阈值:100乐观?假
2015-12-24 10:50:44,937 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - 未设置mapred.job.reduce.markreset.buffer.percent,设置为默认值0.3
2015-12-24 10:50:49,055 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - 设置单店作业
2015-12-24 10:50:49,056 [main] INFO org.apache.pig.data.SchemaTupleFrontend - 键[pig.schematuple]是 false,不会生成代码。
2015-12-24 10:50:49,056 [main] INFO org.apache.pig.data.SchemaTupleFrontend - 开始移动的过程 生成的代码到分布式缓存
2015-12-24 10:50:49,056 [main] INFO org.apache.pig.data.SchemaTupleFrontend - 设置密钥 [pig.schematuple.classes]包含要反序列化[]
的类2015-12-24 10:50:49,158 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 等待提交的1个map-reduce作业。
2015-12-24 10:50:49,158 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker 已弃用。相反,请使用mapreduce.jobtracker.address
2015-12-24 10:50:49,159 [JobControl] INFO org.apache.hadoop.yarn.client.RMProxy - 连接到ResourceManager 在/0.0.0.0:8032
2015-12-24 10:50:49,177 [JobControl] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name是 弃用。而是使用fs.defaultFS
2015-12-24 10:50:49,428 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - 总输入 处理的路径:1
2015-12-24 10:50:49,431 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - 总计 输入路径(组合)处理:2
2015-12-24 10:50:49,467 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - 拆分次数:2
2015-12-24 10:50:49,518 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - 提交作业的令牌: job_1450979216927_0004
2015-12-24 10:50:49,578 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - 已提交 application application_1450979216927_0004
2015-12-24 10:50:49,581 [JobControl] INFO org.apache.hadoop.mapreduce.Job - 跟踪作业的网址: http://quickstart.cloudera:8088/proxy/application_1450979216927_0004/
2015-12-24 10:50:49,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId:job_1450979216927_0004
2015-12-24 10:50:49,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 处理别名日志
2015-12-24 10:50:49,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 详细位置:M:logs [7,7],null [-1,-1] C:R:
2015-12-24 10:50:49,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 有关详情,请访问:http://localhost:50030/jobdetails.jsp?jobid=job_1450979216927_0004
2015-12-24 10:50:49,702 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 完成0%
2015-12-24 10:51:17,030 [主要] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 完成50%
2015-12-24 10:52:04,848 [主要]警告 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 哎呀!有些工作失败了!如果希望Pig在失败时立即停止,请指定-stop_on_failure。
2015-12-24 10:52:04,848 [主要] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 工作job_1450979216927_0004失败了!停止运行所有相关作业
2015-12-24 10:52:04,848 [主要] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100%完成
2015-12-24 10:52:05,039 [主要]错误 org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997:无法 从支持的错误重新创建异常: AttemptID:attempt_1450979216927_0004_m_000001_3信息:错误: org.apache.pig.data.Tuple.isNull()z
2015-12-24 10:52:05,040 [主要]错误 org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce work(s) 失败! 2015年12月24日 10:52:05,040 [main] INFOorg.apache.pig.tools.pigstats.SimplePigStats - 脚本统计:
HadoopVersion PigVersion UserId StartedAt FinishedAt功能 2.6.0-cdh5.4.0 0.12.0-cdh5.4.0 cloudera 2015-12-24 10:50:44 2015-12-24 10:52:05 UNKNOWN
失败!
失败的工作:JobId别名功能消息输出 job_1450979216927_0004记录MAP_ONLY消息:作业 失败! /home/cloudera/workspace/Test_log.csv,
输入:无法从" /myhdfs/project/TestLog.txt"
读取数据输出:无法生成结果 " /home/cloudera/workspace/Test_log.csv"
计数器:写入的总记录数:0写入的总字节数:0可溢出 内存管理器泄漏计数:0主动溢出的行李总数:0总计 主动泄漏的记录:0
Job DAG:job_1450979216927_0004
2015-12-24 10:52:05,040 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 失败了!
答案 0 :(得分:1)
似乎是您的数据处理链中的错误。此日志条目指出元组为空。只是一个想法:这可能发生,例如当您使用自定义UDF时,在某些情况下返回Null。
2015-12-24 10:52:05,039 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backed error: AttemptID:attempt_1450979216927_0004_m_000001_3 Info:Error: org.apache.pig.data.Tuple.isNull()Z