Question

The above code is resulting in issues as it has a column name as keyword

限制。如果我从选择列表中删除“限制”列，则脚本为工作正常。表A具有以下内容 \ ****表A ******* \\\\

There are two tables A , B Table A as follows
 ID    Day      Name  Description  limit  
 1   2016-09-01  Sam   Retail       100
 2   2016-01-28  Chris Retail       200
 3   2016-02-06  ChrisTY Retail      50
 4   2016-02-26  Christa Retail      10
 3   2016-12-06  ChrisTu Retail     200
 4   2016-12-31  Christi Retail     500

Table B has following contents
\\\**** Table B *****\\\\\\\
 Table B

ID SkEY
1  1.1
2  1.2
3  1.3

上面的代码由于将列名作为关键字而导致出现问题限制。如果我从选择列表中删除“限制”列，则脚本为工作正常。

\\\**** Tried Code *****\\\\\\\
 from pyspark.sql import sparksession
 from pyspark.sql import functions as F
 from pyspark.sql.functions import lit
 from pyspark import HiveContext
 hiveContext= HiveContext(sc)
 ABC2 = spark.sql(
"select * From A where day ='{0}'".format(i[0])
 )
Join = ABC2.join(
 Tab2,
 (
    ABC2.ID == Tab2.ID
)
)\
.select(
Tab2.skey,
ABC2.Day,
ABC2.Name,
ABC2.limit,)
withColumn('newcol1, lit('')),
withColumn('newcol2, lit('A'))
ABC2 .show()

ABC = spark.sql（ “从A中选择不同的日期，其中day ='2016-01-01'” ）

\\\**** Expected Result *****\\\\\\\
How can we amend the code so that the limit is also selected

Answer 1

It worked this wasy. not sure functional reason but is successful, Renaming   
the limit as alias before and there after getting it back
\\**** Tried Code *****\\\\\\\ 
from pyspark.sql import sparksession 
from pyspark.sql import functions as F 
from pyspark.sql.functions import lit 
from pyspark import HiveContext 
hiveContext= HiveContext(sc) 
ABC2 = spark.sql( "select Day,Name,Description,limit as liu From A where day
 ='{0}'".format(i[0]) ) 
 Join = ABC2.join( Tab2, ( ABC2.ID == Tab2.ID ) )\ 
.selectexpr( "skey as skey", 
            "Day as Day", 
            "Name as Day", 
            "liu as limit",) 
 withColumn('newcol1, lit('')), 
 withColumn('newcol2, lit('A')) 
 ABC2 .show()

无法选择表中的列，因为列名之一是限制

1 个答案: