java.lang.UnsupportedClassVersionError-在MAC OSX上-不支持的类文件主要版本56

时间:2019-04-22 16:15:22

标签: java apache-spark

我尝试在Mac OSX 10.14.4上运行引用Apache Spark的Python代码,但是Java类版本似乎有问题,并且无法解决问题。它可能与Java版本有关,但我不认识这个问题以及如何解决它。

I installed version of Spark: spark-2.4.1-bin-hadoop2.7, 
I installed the latest version of Java JDK

终端窗口中的Java版本给出以下结果:

java -version

>java version "12.0.1" 2019-04-16
>Java(TM) SE Runtime Environment (build 12.0.1+12)
>Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)

javac -version

>javac 12.0.1

Jupiter Notebook for Python启动期间的警告

在“终端”窗口中启动

$ pyspark

[I 20:47:14.841 NotebookApp] JupyterLab beta preview extension loaded from /Users/myname/anaconda3/lib/python3.6/site-packages/jupyterlab
[I 20:47:14.841 NotebookApp] JupyterLab application directory is /Users/myname/anaconda3/share/jupyter/lab
[I 20:47:14.852 NotebookApp] Serving notebooks from local directory: /Users/myname
[I 20:47:14.852 NotebookApp] 0 active kernels
[I 20:47:14.852 NotebookApp] The Jupyter Notebook is running at:
[I 20:47:14.852 NotebookApp] http://localhost:8888/?token=0666955663ecb246ee569680e4e8e7afbdd3d64b0efcfc75
[I 20:47:14.852 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 20:47:14.896 NotebookApp] 

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8888/?token=0666955663ecb246ee569680e4e8e7afbdd3d64b0efcfc75&token=0666955663ecb246ee569680e4e8e7afbdd3d64b0efcfc75
[I 20:47:15.555 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1
[I 20:47:24.920 NotebookApp] Creating new notebook in 
[I 20:47:27.600 NotebookApp] Kernel started: 2ff964f7-da2a-4427-9aaa-ff9f96967512

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/Applications/spark-2.4.1-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.1.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

19/04/22 20:47:37 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[W 20:47:37.658 NotebookApp] Timeout waiting for kernel_info reply from 2ff964f7-da2a-4427-9aaa-ff9f96967512
19/04/22 20:47:39 WARN MacAddressUtil: Failed to find a usable hardware address from the network interfaces; using random bytes: a1:17:d6:b3:ce:11:0b:4c
[I 20:47:41.088 NotebookApp] Adapting to protocol v5.1 for kernel 2ff964f7-da2a-4427-9aaa-ff9f96967512
[I 20:49:27.467 NotebookApp] Saving file at /Untitled1.ipynb

只要不引用Spark,我就能在Notebook中运行Python代码。在配置Jupyter Notebook之前,在Terminal窗口中启动Spark时,也会发生警告。

这是我在笔记本电脑中使用的python代码:

sc = SparkContext.getOrCreate()
import numpy as np
TOTAL = 10000
dots = sc.parallelize([2.0 * np.random.random(2) - 1.0 for i in range(TOTAL)]).cache()
print("Number of random points:", dots.count())

错误报告

运行代码时,我收到以下错误消息。目的是能够运行该程序而不会出现此错误。

---------------------------------------------------------------------------
Py4JJavaError                             Traceback (most recent call last)
/Applications/spark-2.4.1-bin-hadoop2.7/python/pyspark/sql/utils.py in deco(*a, **kw)
     62         try:
---> 63             return f(*a, **kw)
     64         except py4j.protocol.Py4JJavaError as e:

/Applications/spark-2.4.1-bin-hadoop2.7/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py 
in get_return_value(answer, gateway_client, target_id, name)
    327   "An error occurred while calling {0}{1}{2}.\n".
--> 328   format(target_id, ".", name), value)
    329   else:
Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: java.lang.IllegalArgumentException: Unsupported class file major version 56
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:166)
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:148)
at org.apache.xbean.asm6.ClassReader.<init>(ClassReader.java:136)

More error lines not copied here

我的问题

我对Java和Apache Spark环境不太熟悉。您能否提供有关如何解决此错误的说明?您知道木星笔记本启动过程中的电子警告是否可以解决?如果您需要更多信息,请告诉我。

感谢您的帮助。

1 个答案:

答案 0 :(得分:0)

TLDR

通过安装this之一并通过this设置env var来切换到Java 1.8