如何使用pyspark和jupyter获得评估指标？

Question

我是机器学习和python方面的新手。我正在使用Logistic回归进行任务。当我试图找出如何使用PySpark和Jupiter笔记本获取评估指标时，出现以下错误。

from pyspark.mllib.util import MLUtils
from pyspark.mllib.evaluation import MulticlassMetrics
# Compute raw scores on the test set
predictionAndLabels = test.rdd.map(lambda lp: (float(lr_model.predict(lp.features)), lp.label))
metrics = MulticlassMetrics(predictionAndLabels)

错误

PicklingError: Could not serialize object: Py4JError: An error occurred while calling o123.__getstate__. Trace:
py4j.Py4JException: Method __getstate__([]) does not exist
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
    at py4j.Gateway.invoke(Gateway.java:274)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.lang.Thread.run(Thread.java:748)