Question

我有一个主bash脚本运行其他几个bash脚本和hql文件。 hql文件有hive查询。配置单元查询有一个where子句，它位于日期字段中。我正在尝试自动化一个进程，我需要根据今天的日期（从主bash脚本获得）更改where子句。

例如.hql文件如下所示：这是selectrows.hql

DROP TABLE IF EXISTS tv.events_tmp;
CREATE TABLE tv.events_tmp
(   origintime STRING,
 deviceid STRING,
 clienttype STRING,
 loaddate STRING)
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '\u0001'
 LINES TERMINATED BY '\n'
 STORED AS TEXTFILE LOCATION 'hdfs://nameservice1/data/full/events_tmp';

INSERT INTO TABLE tv.events_tmp SELECT origintime, deviceid, clienttype, loaddate FROM tv.events_tmp WHERE origintime >= '2015-11-02 00:00:00' AND origintime < '2015-11-03 00:00:00';

由于今天是2015-11-11，我希望能够将date - 9 days和date-8 days从bash脚本传递给.hql脚本。有没有办法将这两个变量从bash脚本传递到.hql文件。

所以主要的bash脚本如下所示：

#!/bin/bash
# today's date
prodate=`date +%Y-%m-%d`
echo $prodate
dateneeded=`date -d "$prodate - 8 days" +%Y-%m-%d`
echo $dateneeded

# CREATE temp table
beeline -u 'jdbc:hive2://datanode:10000/;principal=hive/datanode@HADOOP.INT.BELL.CA' -d org.apache.hive.jdbc.HiveDriver -f /home/automation/tv/selectrows.hql
echo "created table"

提前感谢。

Answer 1

You can use beeline -e option to execute queries using strings. Then pass the date parameters to the strings.

#!/bin/bash
# today's date
prodate=`date +%Y-%m-%d`
echo $prodate
dateneeded8=`date -d "$prodate - 8 days" +%Y-%m-%d`
dateneeded9=`date -d "$prodate - 9 days" +%Y-%m-%d`
echo $dateneeded8
echo $dateneeded9

hql="
DROP TABLE IF EXISTS tv.events_tmp;
CREATE TABLE tv.events_tmp
(   origintime STRING,
 deviceid STRING,
 clienttype STRING,
 loaddate STRING)
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '\u0001'
 LINES TERMINATED BY '\n'
 STORED AS TEXTFILE LOCATION 'hdfs://nameservice1/data/full/events_tmp';

INSERT INTO TABLE tv.events_tmp SELECT origintime, deviceid, clienttype, loaddate FROM tv.events_tmp WHERE origintime >= '"

echo "$hql""$dateneeded9""' AND origintime < '""$dateneeded8""';"

# CREATE temp table
beeline -u 'jdbc:hive2://datanode:10000/;principal=hive/datanode@HADOOP.INT.BELL.CA' -d org.apache.hive.jdbc.HiveDriver -e "$hql""$dateneeded9""' AND origintime < '""$dateneeded8""';"
echo "created table"

来自bash脚本的hive（.hql）文件中的命令行参数

1 个答案: