Spark数据集/数据框/ Spark SQL中的多级循环

时间:2019-09-19 05:04:22

标签: sql apache-spark recursion

如何识别Spark中的多级循环?对于单级自我连接可以完成

Select * from input a
join input b on a.source = b.target and a.target=b.source

但是如何查找数据中包含循环的单个节点的所有记录?

Day-1
Source | Target
A | B
B | C  

Output
Source | Target
A|B | NO Cycle
B|C | No Cycle
A|C | No Cycle


Day-2
Input
Source | Target
A|B | 
B|C | 
A|C | 
C|A | New Record
P|Q | New REcord

Output
Source | Target
A|B | cycle  
B|C | cycle   
A|C | cycle
C|A | cycle
P|Q | no cycle

A->B->C  (As A->B->C->A is cycle so all nodes will be considered in cycle)
 \__/

0 个答案:

没有答案