Question

此代码是方法中的一部分。该代码使用两个for循环遍历两个列表。我想看看是否有可能使用多线程来加快两个循环的速度。我关心的是如何使它线程安全。

已编辑：更完整的代码

static class Similarity {
        double similarity;
        String seedWord;
        String candidateWord;

        public Similarity(double similarity, String seedWord, String candidateWord) {
            this.similarity = similarity;
            this.seedWord = seedWord;
            this.candidateWord = candidateWord;
        }

        public double getSimilarity() {
            return similarity;
        }

        public String getSeedWord() {
            return seedWord;
        }

        public String getCandidateWord() {
            return candidateWord;
        }
    }

    static class SimilarityTask implements Callable<Similarity> {
        Word2Vec vectors;
        String seedWord;
        String candidateWord;
        Collection<String> label1;
        Collection<String> label2;

        public SimilarityTask(Word2Vec vectors, String seedWord, String candidateWord, Collection<String> label1, Collection<String> label2) {
            this.vectors = vectors;
            this.seedWord = seedWord;
            this.candidateWord = candidateWord;
            this.label1 = label1;
            this.label2 = label2;
        }

        @Override
        public Similarity call() {
            double similarity = cosineSimForSentence(vectors, label1, label2);
            return new Similarity(similarity, seedWord, candidateWord);
        }
    }

现在，此“计算”线程安全吗？涉及3个变量：

1) vectors;
  2) toeknizerFactory;
  3) similarities;

public static void compute() throws Exception {

        File modelFile = new File("sim.bin");
        Word2Vec vectors = WordVectorSerializer.readWord2VecModel(modelFile);

        TokenizerFactory tokenizerFactory = new TokenizerFactory()

        List<String> seedList = loadSeeds();
        List<String> candidateList = loadCandidates();

        log.info("Computing similarity: ");

        ExecutorService POOL = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
        List<Future<Similarity>> tasks = new ArrayList<>();
        int totalCount=0;
        for (String seed : seedList) {
            Collection<String> label1 = getTokens(seed.trim(), tokenizerFactory);
            if (label1.isEmpty()) {
                continue;
            }
            for (String candidate : candidateList) {
                Collection<String> label2 = getTokens(candidate.trim(), tokenizerFactory);
                if (label2.isEmpty()) {
                    continue;
                }
                Callable<Similarity> callable = new SimilarityTask(vectors, seed, candidate, label1, label2);
                tasks.add(POOL.submit(callable));
                log.info("TotalCount:" + (++totalCount));
            }
        }

        Map<String, Set<String>> similarities = new HashMap<>();
        int validCount = 0;
        for (Future<Similarity> task : tasks) {
            Similarity simi = task.get();
            Double similarity = simi.getSimilarity();
            String seedWord = simi.getSeedWord();
            String candidateWord = simi.getCandidateWord();

            Set<String> similarityWords = similarities.get(seedWord);
            if (similarity >= 0.85) {
                if (similarityWords == null) {
                    similarityWords = new HashSet<>();
                }
                similarityWords.add(candidateWord);
                log.info(seedWord + " " + similarity + " " + candidateWord);
                log.info("ValidCount: "  + (++validCount));
            }

            if (similarityWords != null) {
                similarities.put(seedWord, similarityWords);
            }
        }
}

添加了另一种相关方法，供call（）方法使用：

public static double cosineSimForSentence(Word2Vec vectors, Collection<String> label1, Collection<String> label2) {
        try {
            return Transforms.cosineSim(vectors.getWordVectorsMean(label1), vector.getWordVectorsMean(label2));
        } catch (Exception e) {
            log.warn("OOV: " + label1.toString() + " " + label2.toString());
            //e.getMessage();
            //e.printStackTrace();
            return 0.0;
        }
    }

Answer 1

（已更新问题的答案。）

通常，在尝试优化代码之前，应对它进行概要分析，尤其是在代码非常复杂的情况下。

对于线程，您需要确定线程之间共享的可变状态。理想情况下，在求助于锁和并发数据结构之前，应尽可能地多。这样，一个线程中包含的可变状态就不是问题。不变的东西很棒。

我认为传递给您的任务的任何内容都不会修改。很难说。在字段上使用final是个好主意。可以将集合放置在不可修改的包装中，尽管这并不能阻止它们通过其他引用进行修改，并且现在可以以静态类型显示。

假设您不破坏内部循环，则唯一的共享可变状态似乎是similarities及其包含的值。

您可能会或可能不会发现您最终仍然做过多的串行操作，需要更改similarities才能并发

    ConcurrentMap<String, Set<String>> similarities = new ConcurrentHashMap<>();

get中的put和similarities必须是线程安全的。我建议始终创建Set。

        Set<String> similarityWords = similarities.getOrDefault(seed, new HashSet<>());

或

        Set<String> similarityWords = similarities.computeIfAbsent(seed, key -> new HashSet<>());

您可以使用线程安全的Set（例如，使用Collections.synchronizedSet），但是我建议为整个内部循环持有一个相关的锁。

synchronized (similarityWords) {
    ...
}

如果您想懒惰地创建similarityWords，那将是“更加有趣”。

如何使这段代码线程安全？

1 个答案: