Question

我有一个DataFrame（Apache Spark 1.5）。我想使用spark sql context添加新列以获取所有raw包含单引号的新列。

我的代码：

df.registerTempTable("tempdf");
df = df.sqlContext().sql("SELECT *, \" \\\" \" as quoteCol FROM tempdf");

执行后Spark抛出下一个异常：

Exception in thread "main" java.lang.RuntimeException: [1.44] failure: ``union'' expected but ErrorToken(end of input) found

SELECT *, " \" " as quoteCol FROM tempdf
                                        ^
    at scala.sys.package$.error(package.scala:27)
    at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:36)
    at org.apache.spark.sql.catalyst.DefaultParserDialect.parse(ParserDialect.scala:67)
    at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
    at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
    at org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:114)
    at org.apache.spark.sql.execution.SparkSQLParser$$anonfun$org$apache$spark$sql$execution$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:113)
    at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:137)
    ...

下一代码正常工作，并添加一个包含单个字符的新行：

df.registerTempTable("tempdf");
df = df.sqlContext().sql("SELECT *, \" q \" as quoteCol FROM tempdf");

我做错了什么？

Answer 1

SQL字符串应使用单引号：

sqlContext().sql("SELECT *, '\"' AS quoteCol FROM tempdf");

Spark SQL单引号错误

1 个答案: