结合两个rdd - pyspark

时间:2017-07-20 12:34:25

标签: join pyspark rdd

我有两个RDD,如下所示

r1 = [(u'5971', u'COLOR > RED', 599),(u'5131', u'MEN > BOW TIES > ALL IN COLLECTION', 599)]  # id, category, price

r2 = [(u'5131', 1), (u'5971', 1), (u'8347', 1)] # id, quantity

我希望结果如下:

r3 = [(u'5131', ('MEN > BOW TIES > ALL IN COLLECTION',599)), (u'5971', ('COLOR > RED',599)]

我在下面尝试过:

r3 = r1.join(r2)

但结果r3 rdd中缺少价格。

1 个答案:

答案 0 :(得分:0)

r1 = [(u'5971', u'COLOR > RED', 599),(u'5131', u'MEN > BOW TIES > ALL IN COLLECTION', 599)]  # id, category, price

r2 = [(u'5131', 1), (u'5971', 1), (u'8347', 1)] # id, quantity

r1_modified = r1.map(lambda (l,m,n):(l,(m,n)))

r3 = r1_modified.join(r2).map(lambda (l,m,n):(l,m))