将Spark插入EMR集群中的Hive表时发生java.net.UnknownHostException

时间:2018-12-27 14:11:07

标签: amazon-web-services apache-spark pyspark amazon-emr

我正在尝试将值插入到现有的配置单元表中。值已插入配置单元表中,但在EMR群集中运行时却出现异常。

df.write.mode("append").insertInto(staging.table)

Sql语句

   spark.sql("insert into table staging.table  values ('81253157746','CZK','','Dest','Neth','-1','','CZK;Dest;EST','2018-12-27 14:01:19','')")

例外

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/spark/python/pyspark/sql/session.py", line 708, in sql
    return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)
  File "/usr/lib/spark/python/lib/py4j-0.10.6-src.zip/py4j/java_gateway.py", line 1160, in __call__
  File "/usr/lib/spark/python/pyspark/sql/utils.py", line 69, in deco
    raise AnalysisException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: u'java.lang.IllegalArgumentException: java.net.UnknownHostException: ip-10-XXX-XXX-XXX;'

0 个答案:

没有答案