使用Scala作为脚本运行Hive

时间:2017-09-04 06:29:25

标签: scala hadoop hive apache-spark-sql

我正在浏览Scala的processBuilder API,为了运行shell命令,就像我们在Shell脚本中运行一样,我可以运行一些脚本但是遇到一种Hive查询执行问题。

当我运行以下命令时,它完全成功运行,但是一种格式失败:

运行Shell命令(成功):

scala> import sys.process._
scala> "ls -lrt /home/cloudera/Desktop".!
total 164
-rwxrwxr-x  1 cloudera cloudera    237 Apr  5  2016 Parcels.desktop
-rwxrwxr-x  1 cloudera cloudera    238 Apr  5  2016 Kerberos.desktop
-rwxrwxr-x  1 cloudera cloudera    259 Apr  5  2016 Express.desktop

使用文件选项运行Hive查询(成功):

 scala> "hive -f /home/cloudera/hi.hql" !!
 warning: there was one feature warning; re-run with -feature for 
 details
 ls: cannot access /usr/lib/spark/lib/spark-assembly-*.jar: No such 
 file or directory
 2017-09-03 23:20:34,392 WARN  [main] mapreduce.TableMapReduceUtil: The 
 hbase-prefix-tree module jar containing PrefixTreeCodec is not 
 present.  Continuing without it.

 Logging initialized using configuration in 
 file:/etc/hive/conf.dist/hive-log4j.properties
 OK
Time taken: 0.913 seconds, Fetched: 2 row(s)
res20: String =
"100    Amit                    12000   10
 101    Allen                   22000   20 .                                   

"

使用-e选项运行Hive查询(失败):

如果我在终端上运行了以下查询,我可以在下面给出的格式中运行。

bash$ hive -e "select * staging.from employee_canada;"

在scala终端中运行相同查询时出现的问题由于双引号(“”)@ Select查询而失败。如何逃避这些并成功运行。尝试使用三引号,以及转义“\”序列,但仍然无法执行。

scala> "hive -e select * staging.from employee_canada; "!!

以下是错误:

FAILED: ParseException line 1:6 cannot recognize input near '<EOF>' 
'<EOF>' '<EOF>' in select clause
java.lang.RuntimeException: Nonzero exit value: 64
at scala.sys.package$.error(package.scala:27)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(
ProcessBuilderImpl.scala:102)
... 32 elided   

0 个答案:

没有答案