静态成员变量如何在Spark中分配给工人?

时间:2018-11-30 15:30:57

标签: java apache-spark serializable

public class MapMatchingMain {

    public static void main(String[] args) {
        ...

        MapMatching mapMatching = new MapMatching(hopper, opts);

        JavaRDD<String> matchedTraces = traceData.map(
                new Function<String, String>() {
                    @Override
                    public String call(String s) throws Exception {
                        return singleTraceMatch(getTraceFromString(s), mapMatching));
                    }
                }
        );
        }
    }

当我在mapMatching方法中实例化main()实例时,我将得到错误消息

  

线程“ main” org.apache.spark.SparkException中的异常:任务不可序列化

public class MapMatchingMain {

    private static MapMatching mapMatching = new MapMatching();

    public static void main(String[] args) {
        ...
        JavaRDD<String> matchedTraces = traceData.map(
                new Function<String, String>() {
                    @Override
                    public String call(String s) throws Exception {
                        return singleTraceMatch(getTraceFromString(s));
                    }
                }
        );
        }
    }

如果我将mapMatching实例实例化为类MapMatchingMain的静态成员变量,则没有问题。

我已经阅读了一些答案,说静态字段值未序列化。那么如何将其分配给每个工人呢?

0 个答案:

没有答案