方法col([class java.util.ArrayList])不存在

时间:2018-08-02 10:44:51

标签: pyspark apache-spark-sql

在以下代码中获取此异常。

代码:

when(INV['VFA_EXTRA_AM'].isNull(), None)
                    .otherwise((INV['VFA_EXTRA_AM'].cast(DecimalType(12,2))/INV['EXTRAADULT_COUNT_CHECK']
                                ).cast(DecimalType(12,2))).alias("PER_PRSN_VFA_EXTRA_AM"),
                    when(INV['VFC_EXTRA_AM'].isNull(), None)
                    .otherwise((INV['VFC_EXTRA_AM'].cast(DecimalType(12, 2))/INV['EXTRACHILD_COUNT_CHECK']
                                ).cast(DecimalType(12, 2))).alias("PER_PRSN_VFC_EXTRA_AM"),
                    when(INV['VFI_EXTRA_AM'].isNull(), None)
                    .otherwise((INV['VFI_EXTRA_AM'].cast(DecimalType(12, 2))/INV['EXTRAINFANT_COUNT_CHECK']
                                ).cast(DecimalType(12, 2))).alias("PER_PRSN_VFI_EXTRA_AM"))

INV是DataFrame名称。

错误日志:

  File "/mnt/dclrms-cogs/resbaseline/Integration.py", line 52, in execueIntegration
    ).cast(DecimalType(12, 2))).alias("PER_PRSN_VFI_EXTRA_AM"))\
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1040, in select
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 895, in _jcols
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 882, in _jseq
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/column.py", line 60, in _to_seq
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/column.py", line 48, in _to_java_column
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/column.py", line 41, in _create_column_from_name
  File "/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
  File "/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 323, in get_return_value
py4j.protocol.Py4JError: An error occurred while calling z:org.apache.spark.sql.functions.col. Trace:
py4j.Py4JException: Method col([class java.util.ArrayList]) does not exist
        at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
        at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:339)
        at py4j.Gateway.invoke(Gateway.java:274)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:214)
        at java.lang.Thread.run(Thread.java:748)

0 个答案:

没有答案