Question

我正在编写一些代码，这些代码将读取日志行并在后台对该数据进行一些处理。这种处理可能有利于并行化，例如Stream.parallel方法提供的内容，我试图使用它。这是我开始使用的代码完美无缺。

public static void main(String[] args) {
    try (Socket socket = new Socket(ADDRESS, PORT); 
         BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()))) {
        socket.getOutputStream().write(QUERY);
        reader.lines().forEach(System.out::println);
    } catch (IOException e) {
        e.printStackTrace();
    }

}

此代码连接并打印出我的所有数据。我非常想重新构建此代码，如下所示：

public static void main(String[] args) {
    try {
        final Socket socket = new Socket(ADDRESS, PORT);
        final BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
        socket.getOutputStream().write(QUERY);
        reader.lines().parallel().forEach(System.out::println);
    } catch (IOException e) {
        e.printStackTrace();
    }

}

但遗憾的是，这并不奏效。更糟糕的是，回到原始代码，这甚至不起作用：

try(A a = new A()) {}

这里添加的所有内容都是。并行调用，这完全不起作用。它只是坐在那里没有打印出来。

如果没有使用修改后的Full thread dump OpenJDK 64-Bit Server VM (25.112-b15 mixed mode): "Attach Listener" #9 daemon prio=9 os_prio=0 tid=0x00007fd4f4001000 nid=0x4907 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Service Thread" #8 daemon prio=9 os_prio=0 tid=0x00007fd5280be000 nid=0x48d2 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007fd5280bb000 nid=0x48d1 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fd5280b9800 nid=0x48d0 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fd5280b6800 nid=0x48cf waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fd5280b5000 nid=0x48ce runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fd528082000 nid=0x48cd in Object.wait() [0x00007fd515c6d000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000ec008e98> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) - locked <0x00000000ec008e98> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fd52807d800 nid=0x48cc in Object.wait() [0x00007fd515d6e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000ec006b40> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) - locked <0x00000000ec006b40> (a java.lang.ref.Reference$Lock) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) "main" #1 prio=5 os_prio=0 tid=0x00007fd528008000 nid=0x48c2 runnable [0x00007fd52fd9f000] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) - locked <0x00000000ec086790> (a java.net.SocksSocketImpl) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.<init>(Socket.java:434) at java.net.Socket.<init>(Socket.java:211) at com.gravypod.Test.main(Test.java:48) "VM Thread" os_prio=0 tid=0x00007fd528075800 nid=0x48ca runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fd52801d800 nid=0x48c4 runnable "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fd52801f000 nid=0x48c5 runnable "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007fd528021000 nid=0x48c6 runnable "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007fd528022800 nid=0x48c7 runnable "VM Periodic Task Thread" os_prio=0 tid=0x00007fd5280c0800 nid=0x48d3 waiting on condition JNI global references: 18的第二个版本，我可以很好地生活，因为在这种情况下看起来并不太好。我无法生活的是弄清楚为什么这个。并行呼叫会打破一切。

我假设修改过的try语句在我退出流后立即关闭流（在我们启动forEach之后），因此它们在操作之前被杀死并且GC被删除。我不能为我的生活弄清楚。并行电话到底是怎么回事。

这里要求的是在此代码的.parellel（）版本上运行的jstack的输出。

Socket socket = new Socket

Test.java:48行是Full thread dump OpenJDK 64-Bit Server VM (25.112-b15 mixed mode): "Attach Listener" #9 daemon prio=9 os_prio=0 tid=0x00007f9048001000 nid=0x4982 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Service Thread" #8 daemon prio=9 os_prio=0 tid=0x00007f90800be800 nid=0x496f runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007f90800bb000 nid=0x496e waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f90800b9800 nid=0x496d waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f90800b6800 nid=0x496c waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f90800b5000 nid=0x496b runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f9080082000 nid=0x496a in Object.wait() [0x00007f907018d000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000ec008e98> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) - locked <0x00000000ec008e98> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f908007d800 nid=0x4969 in Object.wait() [0x00007f907028e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000ec006b40> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) - locked <0x00000000ec006b40> (a java.lang.ref.Reference$Lock) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) "main" #1 prio=5 os_prio=0 tid=0x00007f9080008000 nid=0x4961 runnable [0x00007f90884c3000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) - locked <0x00000000ec08e890> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) - locked <0x00000000ec08e890> (a java.io.InputStreamReader) at java.io.BufferedReader.readLine(BufferedReader.java:389) at java.io.BufferedReader$1.hasNext(BufferedReader.java:571) at java.util.Iterator.forEachRemaining(Iterator.java:115) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) at com.gravypod.Test.main(Test.java:51) "VM Thread" os_prio=0 tid=0x00007f9080075800 nid=0x4968 runnable "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f908001d800 nid=0x4963 runnable "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f908001f000 nid=0x4964 runnable "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007f9080021000 nid=0x4965 runnable "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007f9080022800 nid=0x4966 runnable "VM Periodic Task Thread" os_prio=0 tid=0x00007f90800c1000 nid=0x4970 waiting on condition JNI global references: 319行。这是完全工作的非并行代码（仅使用.lines（））的结果。

reader.lines().forEach

Test.java:51行是{{1}}行。

Answer 1

我想并行流上的parallel（）或forEach（）等待在并行化任务之前读取所有输入。因为服务器永远不会关闭连接，所以它将永远等待。

您的任务无法实现并行化。数据按顺序通过线路传输，因此并行读取数据无法正常工作。

Answer 2

看来，在执行可观察的工作之前，您的应用程序在技术上并不是悬挂，而是等待大量输入。这是两个实现细节的组合。当您开始并行流操作时，在实际开始处理元素之前，它将首先尝试拆分工作负载，直到每个CPU核心都有事情要做。这与Reader#lines() parallelizes badly due to nonconfigurable batch size问题相结合。

简单地说，当Stream具有未知大小时，实现将尝试缓冲多个1024倍数的批量，在每次拆分时增长。 This great answer显示了如何对具有多个核心的未知大小的流进行拆分，这表明多个1024元素将在此过程中被缓冲。

1。在调用传递给forEach的消费者之前，这可能需要很长时间。

请注意，通过非短路forEach处理无限来源无论如何都不在Stream API的范围之内。假设及时的副作用是关于流的处理顺序的假设，但没有保证。

This answer指导您进行解决方案。你可以使用像

这样的东西

try(Socket socket = new Socket(ADDRESS, PORT);
    BufferedReader reader = new BufferedReader(
        new InputStreamReader(socket.getInputStream()))) {

    socket.getOutputStream().write(QUERY);
    Stream.generate(() -> {
        try { return reader.readLine(); }
        catch (IOException ex) { throw new UncheckedIOException(ex); }
    }).parallel().forEach(System.out::println);
} catch(IOException|UncheckedIOException e) {
    e.printStackTrace();
}

但是，如上所述，这不是Stream API的预期用例......

使用BufferedReader.lines（）并行中断

2 个答案: