Spark Sql:'like'语法错误

时间:2015-09-29 08:36:31

标签: apache-spark-sql

我执行时

sqlcontext.sql("select * from logs where content like '%stream%' ")

在我的火花应用程序中,错误显示如下:

  

java.lang.RuntimeException:[1.39]失败:``'''预期但'%'   结果

     

从日志中选择*,其中内容如%stream%                                         ^           在scala.sys.package $ .error(package.scala:27)           在org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(AbstractSparkSQLParser.scala:40)           at org.apache.spark.sql.SQLContext $$ anonfun $ 2.apply(SQLContext.scala:130)           at org.apache.spark.sql.SQLContext $$ anonfun $ 2.apply(SQLContext.scala:130)           在org.apache.spark.sql.SparkSQLParser $$ anonfun $ org $ apache $ spark $ sql $ SparkSQLParser $$ others $ 1.apply(SparkSQLParser.scala:96)           在org.apache.spark.sql.SparkSQLParser $$ anonfun $ org $ apache $ spark $ sql $ SparkSQLParser $$ others $ 1.apply(SparkSQLParser.scala:95)           在scala.util.parsing.combinator.Parsers $ Success.map(Parsers.scala:136)           在scala.util.parsing.combinator.Parsers $ Success.map(Parsers.scala:135)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ map $ 1.apply(Parsers.scala:242)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ map $ 1.apply(Parsers.scala:242)           在scala.util.parsing.combinator.Parsers $$ anon $ 3.apply(Parsers.scala:222)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ append $ 1 $$ anonfun $ apply $ 2.apply(Parsers.scala:254)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ append $ 1 $$ anonfun $ apply $ 2.apply(Parsers.scala:254)           在scala.util.parsing.combinator.Parsers $ Failure.append(Parsers.scala:202)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ append $ 1.apply(Parsers.scala:254)           在scala.util.parsing.combinator.Parsers $ Parser $$ anonfun $ append $ 1.apply(Parsers.scala:254)           在scala.util.parsing.combinator.Parsers $$ anon $ 3.apply(Parsers.scala:222)           在scala.util.parsing.combinator.Parsers $$ anon $ 2 $$ anonfun $ apply $ 14.apply(Parsers.scala:891)           在scala.util.parsing.combinator.Parsers $$ anon $ 2 $$ anonfun $ apply $ 14.apply(Parsers.scala:891)           在scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)           在scala.util.parsing.combinator.Parsers $$ anon $ 2.apply(Parsers.scala:890)           在scala.util.parsing.combinator.PackratParsers $$ anon $ 1.apply(PackratParsers.scala:110)           在org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(AbstractSparkSQLParser.scala:38)           at org.apache.spark.sql.SQLContext $$ anonfun $ parseSql $ 1.apply(SQLContext.scala:134)           at org.apache.spark.sql.SQLContext $$ anonfun $ parseSql $ 1.apply(SQLContext.scala:134)           在scala.Option.getOrElse(Option.scala:120)           在org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:134)           在org.apache.spark.sql.SQLContext.sql(SQLContext.scala:915)           at louis.spark.LogAnalysis $ 4.call(LogAnalysis.java:218)           在louis.spark.LogAnalysis $ 4.call(LogAnalysis.java:1)           在org.apache.spark.streaming.api.java.JavaDStreamLike $$ anonfun $ foreachRDD $ 1.apply(JavaDStreamLike.scala:302)           在org.apache.spark.streaming.api.java.JavaDStreamLike $$ anonfun $ foreachRDD $ 1.apply(JavaDStreamLike.scala:302)           在org.apache.spark.streaming.dstream.DStream $$ anonfun $ foreachRDD $ 1.apply(DStream.scala:534)           在org.apache.spark.streaming.dstream.DStream $$ anonfun $ foreachRDD $ 1.apply(DStream.scala:534)           在org.apache.spark.streaming.dstream.ForEachDStream $$ anonfun $ 1.apply $ mcV $ sp(ForEachDStream.scala:42)   在   org.apache.spark.streaming.dstream.ForEachDStream $$ anonfun $ 1.适用(ForEachDStream.scala:40)           在org.apache.spark.streaming.dstream.ForEachDStream $$ anonfun $ 1.apply(ForEachDStream.scala:40)           在scala.util.Try $ .apply(Try.scala:161)           在org.apache.spark.streaming.scheduler.Job.run(Job.scala:32)           在org.apache.spark.streaming.scheduler.JobScheduler $ JobHandler $$ anonfun $ run $ 1.apply $ mcV $ sp(JobScheduler.scala:176)           在org.apache.spark.streaming.scheduler.JobScheduler $ JobHandler $$ anonfun $ run $ 1.apply(JobScheduler.scala:176)           在org.apache.spark.streaming.scheduler.JobScheduler $ JobHandler $$ anonfun $ run $ 1.apply(JobScheduler.scala:176)           在scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)           在org.apache.spark.streaming.scheduler.JobScheduler $ JobHandler.run(JobScheduler.scala:175)           在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)           at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:615)           在java.lang.Thread.run(Thread.java:745)

如何在spark sql中使用'like'语法?有什么建议吗?

1 个答案:

答案 0 :(得分:0)

可能您使用的是旧版本的spark,我已经在Apache Spark 1.4.0中测试了您的查询,它是正确的但是根据该行 - >

java.lang.RuntimeException: [1.39] failure: ``*'' expected but `%' found

你应该试试这个 - >

sqlcontext.sql("select * from logs where content like '*stream*' ")