Question

我编写了一个简单的 pyspark 脚本来将数据保存在我的本地数据库中

spark = SparkSession.builder.master('local[*]').getOrCreate()
df=spark.read.csv('test.csv'
                  ,inferSchema=True
                  ,header=True)

df.write.format('jdbc').option('url','jdbc:mysql://127.0.0.1:3306/?user=root')\
    .option('dbtable','test.table1')\
    .option('driver','com.mysql.cj.jdbc.Driver')\
    .option('password','12345678').save()
spark.stop()

这段代码大部分时间都可以正常工作。但是当我连接到工作 VPN 并再次运行代码时，它给出了错误：

Traceback (most recent call last):
  File "/Users/opt/anaconda3/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-1f64172ba870>", line 1, in <module>
    runfile('/Users/Desktop/spark_process/main.py', wdir='/Users/Desktop/spark_process')
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/Desktop/spark_process/main.py", line 13, in <module>
    df.write.format('jdbc').option('url','jdbc:mysql://127.0.0.1:3306/?user=root')\
  File "/Users/opt/anaconda3/lib/python3.8/site-packages/pyspark/sql/readwriter.py", line 1107, in save
    self._jwrite.save()
  File "/Users/opt/anaconda3/lib/python3.8/site-packages/py4j/java_gateway.py", line 1304, in __call__
    return_value = get_return_value(
  File "/Users/opt/anaconda3/lib/python3.8/site-packages/pyspark/sql/utils.py", line 111, in deco
    return f(*a, **kw)
  File "/Users/opt/anaconda3/lib/python3.8/site-packages/py4j/protocol.py", line 326, in get_return_value
    raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o37.save.
: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

如何解决这个问题？

使用 VPN 连接到本地 MySQL 服务器时 PySpark 失败

0 个答案: