如果我想在spark中使用MongoDB
我应该使用spark2-submit --packages org.mongodb.spark:mongo-spark-connector_2.11:2.2.0 ...
但是这个命令需要maven和internet
manualBookingReservations = spark.read.format("com.mongodb.spark.sql.DefaultSource"
).option("uri",uri
).option("partitioner" , "MongoPaginateBySizePartitioner"
).load()
spark2-submit cmd:
spark2-submit --master yarn --deploy-mode cluster --files /etc/hive \
/conf/hive-site.xml --executor-cores 1 --driver-cores 1 \
--num-executors 1 --driver-memory 1G --executor-memory 1G \
--jars ./mongo-spark-connector_2.11.jar \
spark-awesome-app.spark2.py
*假设当前目录中存在“mongo-spark-connector_2.11.jar”,其中我执行spark2-sumbit
如何在没有互联网或专家的情况下添加Mongodb火花连接器?