Azure Databricks SparkSQL是否支持递归查询

时间:2019-10-07 22:09:14

标签: azure pyspark databricks

我正在将数据从SQL Server移动到Azure数据湖gen2,并使用递归查询转换SQL查询。

这是使用CTE(公用表表达式)进行递归的示例SQL查询

 WITH RECURSIVE AS BOM
          (SELECT p.MItemId AS RootPartNumber,
                  p.MItemId AS PartNumber,
                  NULL AS ParentPartNumber,
                  0    AS BomLevel,
                  1.0  AS Quantity
           FROM   PartItem p

           UNION ALL
           SELECT BOM.RootPartNumber,
                 CAST(BSM.ChildItem AS string) AS PartNumber,
                 CAST(DB.PartNumber AS string) AS ParentPartNumber,
                 BOM.BomLevel + 1  as BomLevel,
                 BSM.Quantity AS Quantity 
           FROM  PartItemBomList BSM
           INNER JOIN BOM  ON BOM.PartNumber = BSM.ParentItem
           INNER JOIN PartItem p           ON p.MItemId = BSM.ChildItem
           WHERE BSM.IsDeleted = 0 
  )
  SELECT * FROM BOM

我尝试更改将递归嵌入到FROM子句中的查询,如下所示,但没有成功。

 SELECT * FROM 
          (SELECT p.MItemId AS RootPartNumber,
                  p.MItemId AS PartNumber,
                  NULL AS ParentPartNumber,
                  0    AS BomLevel,
                  1.0  AS Quantity
           FROM   PartItem p
           WHERE p.PartType =    'Cloud-OrderableAssembly' 
           UNION ALL
           SELECT BOM.RootPartNumber,
                 CAST(BSM.ChildItem AS string) AS PartNumber,
                 CAST(DB.PartNumber AS string) AS ParentPartNumber,
                 BOM.BomLevel + 1  as BomLevel,
                 BSM.Quantity AS Quantity 
           FROM  PartItemBomList BSM
           INNER JOIN BOM  ON BOM.PartNumber = BSM.ParentItem
           INNER JOIN PartItem p           ON p.MItemId = BSM.ChildItem
           WHERE BSM.IsDeleted = 0 
  ) as BOM

这是我从Azure Databricks会话中得到的错误。

SQL语句中的错误:AnalysisException:找不到表或视图:BOM;第16行pos 22

1 个答案:

答案 0 :(得分:1)

问题在这里

Generating a RSA private key
......................................................................................................................................................................+++++
..........................+++++
writing new private key to 'MOK.priv'
-----
problems making Certificate Request
140510219141568:error:0D07A097:asn1 encoding routines:ASN1_mbstring_ncopy:string too long:../crypto/asn1/a_mbstr.c:107:maxsize=2

这是内部查询,据我了解,BOM是在外部定义的,因此这部分查询运行的BOM不存在。

如果我是我,则可以尝试通过直接在SQL上运行来修复以下查询。 下面引用BOM的方式不正确

INNER JOIN BOM ON BOM.PartNumber = BSM.ParentItem