从另一个数据帧列值过滤数据帧

时间:2018-05-24 00:28:55

标签: scala apache-spark apache-spark-sql

我们有两个数据帧,我们需要过滤 一个数据帧中的数据与另一个数据帧列中的数据

df1
-------------------------------
name paid_amount date_paid
-------------------------------
aaa   10          2017-10-10
aba   10          2017-01-10
aac   10          2017-10-10
daa   10          2017-16-10

df2
-----------------------------
start_date end_date
-----------------------------
2017-01-01 2018-01-01
------------------------------

we need to create third dataframe by checking  
(date_paid) field in df1 falls in between df2(start_date) & df2(end_date)

df1.where($date_paid).isin(df2.start_date && df2.end_date)

0 个答案:

没有答案