Question

对于我的工作，我必须开发一个小型Java应用程序来解析非常大的文本文件（~6800 kb），因此我尝试对其进行一些优化以进一步处理。使用equalto（=）作为第一个fork任务拆分行，并使用第二个fork任务拆分逗号和一些计算。这会为单个应用程序带来多个fork任务的良好性能。

1）第一个分叉和加入过程：

 @Override
protected Map < String, String > compute() {
SplitString lineSplit = new SplitString();
Map < String, String > splitUrl = new HashMap < > ();
// list size is less than limit process actual task.

if (readAllLines.size() < Constant.LIMIT) {
    // splitUrl map stores the splitter lines with equal_to
    splitUrl = lineSplit.splittingLines(readAllLines);

} else { // list size greater than limit splits task in to two and process it.
    // split task into sub task and stores it in list
    List < String > left = readAllLines.subList(Constant.ZERO, readAllLines.size() / 2);
    List < String > right = readAllLines.subList(readAllLines.size() / 2, readAllLines.size());
    SplitTask firstTask = new SplitTask(left);
    SplitTask secondTask = new SplitTask(right);
    // forks the firstTask
    firstTask.fork();
    // forks the secondTask
    secondTask.fork();
    // join the task in splitUrl map
    splitUrl.putAll(firstTask.join());
    splitUrl.putAll(secondTask.join());
}
return splitUrl;
}

2）第二次分叉和加入过程：

 @Override
protected Map < String, Map < String, Integer >> compute() {
    SplitString ruleSplit = new SplitString();
    Map < String, Map < String, Integer >> response = new HashMap < > ();
    // list size is less than limit process actual task.

    if (KeyList.size() < Constant.LIMIT) {
        // splitUrl map stores the splitter lines with equal_to
        response = ruleSplit.calculatingRuleTime(result, KeyList);

    } else { // list size greater than limit splits task in to two and process it.
        // split task into sub task and stores it in list.
        List < String > left = KeyList.subList(Constant.ZERO, KeyList.size() / 2);
        List < String > right = KeyList.subList(KeyList.size() / 2, KeyList.size());
        SplitRuleTask firstTask = new SplitRuleTask(result, left);
        SplitRuleTask secondTask = new SplitRuleTask(result, right);
        // forks the firstTask
        firstTask.fork();
        // forks the firstTask
        secondTask.fork();
        // join the task in response map
        response.putAll(firstTask.join());
        response.putAll(secondTask.join());
    }
    return response;
}

任何人都可以帮帮我吗？

Answer 1

Fork也在创造一个沉重的重量过程。我建议你阅读线程概念，然后对它们实现多线程。更高级的方法是创建线程池来工作。我建议你阅读以下链接。

使用多个fork和join类来分割任务，是不是一个好的做法？

1 个答案: