当我查询ObjectId类型的主键时,我正在使用mongo连接器(mongo-spark-connector_2.11:2.3.1
)。永远不会推送到数据库
示例(ObjectId):
|-_id:结构(nullable = true)
| |-oid:字符串(nullable = true)
dataframe1.filter($"_id.oid" === ("58115596d12d37d780353ed7"))
== Physical Plan ==
*(1) Filter ((_id#491.oid = 58115596d12d37d780353ed7) && isnotnull(_id#491))
+- *(1) Scan MongoRelation(MongoRDD[73] at RDD ... PushedFilters: [IsNotNull(_id)] ...
,但是如果它是另一种类型。
示例(字符串):
_id:字符串(可为空,为true)
dataframe2.filter($"_id" === ("LEM"))
== Physical Plan ==
*(1) Filter (isnotnull(_id#455) && (_id#455 = LEM))
+- *(1) Scan MongoRelation(MongoRDD[52] at RDD at ...... PushedFilters: [IsNotNull(_id), EqualTo(_id,LEM)] ..
有人知道有什么方法可以使用spark和mongodb在数据库级别执行这些查询吗?