分割流

Question

我有一个文件。我使用Files.lines获得流。该文件很大。我需要循环遍历并形成几个数组。在遍历文件期间，必须将每个生成的数组传递给将对其进行处理的方法。我知道有PartitioningBy和GroupingBy方法，但是我不知道如何将它们应用于我的任务。我正在尝试这样做：

@Test
public void myTest() {
    Stream<String> lines = Stream.of(
            "some row from my file 1",
            "some row from my file 2",
            "some row from my file 3",
            "some row from my file 4",
            "some row from my file 5",
            "some row from my file n",
            "some row from my file 750000"
    );
    lines.parallel()
            .unordered()
            .collect(Collectors.partitioningBy(s -> s == 3).supplier(it -> {
                myParser(it);
            }));
}

public void myParser(List<String> myList){
    //this piece of code should give the length of the transmitted array
    System.out.println(myList.size()); 
}

在myParser方法中，我想获取3个元素的数组并对其进行处理

Answer 1

我将详细介绍这个选项

    ArrayList<String> list = new ArrayList<>();

    lines.forEach(it -> {
        list.add(it);
        if (list.size() > 0 && list.size() % 3 == 0) {
            myParser(list);
            list.clear();
        }
    });

Answer 2

您可以尝试使用此方法拆分流：

public class T30SplitStream {

public static void main(String[] args) {
    Stream<String> lines = Stream.of("some row from my file 1", "some row from my file 2",
            "some row from my file 3", "some row from my file 4", "some row from my file 5",
            "some row from my file n", "some row from my file 750000");
    AtomicInteger i = new AtomicInteger(0);
    Map<Integer, List<String>> map = lines.parallel().unordered().map(s -> new Pair(i.incrementAndGet(), s))
            .collect(Collectors.groupingBy(p -> p.i % 3, Collectors.mapping(p -> p.s, Collectors.toList())));
    System.out.println(map);

}

public static class Pair {
    public final Integer i;
    public final String s;

    public Pair(int i, String s) {
        this.i = i;
        this.s = s;
    }
}
}

Answer 3

解决方案：

List<List<String>> partition = ListUtils.partition(lines, 3);
partition.parallelStream().forEach(this::myParser);

行家：

    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-collections4</artifactId>
        <version>4.1</version>
    </dependency>