ParameterSubstitutionException:将bash变量传递给Pig参数

时间:2014-07-30 21:51:33

标签: bash hadoop mapreduce apache-pig

我收到错误:

  

2014-07-30 14:34:07,104 [main] ERROR org.apache.pig.Main - ERROR 2997:遇到IOException。 org.apache.pig.tools.parameters.ParameterSubstitutionException:未定义的参数:尝试查找输入值时找到的年份。

我做./run_pig_script 2014-07-30后?

我正在尝试将年,月和日传递给Pig中的输入参数。

以下是我的shell脚本的精简版,名为run_pig_script

#!/bin/bash

# Hadoop Job Queue to use
JOB_QUEUE=LongRun

array=( "$@" )
time=${array[0]}

IFS='-' read -ra timestamp <<< "$time"
year=${timestamp[0]}
month=${timestamp[1]}
day=${timestamp[2]}

echo $year
echo $month
echo $day

pig -Dmapred.job.queue.name=$JOB_QUEUE -param "input=hdfs:///raw/tool/year=$year/month=$month/day=$day/hour=00/min=00/" -f process_data.pig

错误消息显示在org.apache.pig.tools.parameters.PreprocessorContext.java

    while (bracketKeyMatcher.find()) {
        if ( (bracketKeyMatcher.start() == 0) || (line.charAt( bracketKeyMatcher.start() - 1)) != '\\' ) {
            key = bracketKeyMatcher.group(1);
            if (!(param_val.containsKey(key))) {
                String message;
                if (parentKey == null) {
                    message = "Undefined parameter : " + key;
                } else {
                    message = "Undefined parameter : " + key + " found when trying to find the value of " + parentKey + "."; 
                }
                throw new ParameterSubstitutionException(message);
            }
            val = param_val.get(key);
            if (val.contains("$")) {
                val = val.replaceAll("(?<!\\\\)\\$", "\\\\\\$");
            }
            replaced_line = replaced_line.replaceFirst("\\$\\{"+key+"\\}", val);
        }
    }

0 个答案:

没有答案